jump to navigation

Pope of debunkers February 15, 2009

Posted by dissident93 in Iraq mortality.

Tim Lambert runs a “science blog” which at times successfully debunks rightwing punditry. At other times, Lambert seems out of his depth. For example, his attempt at “analysis” of media inaccuracy (over Iraq Body Count’s figures) was riddled with errors. He grudgingly corrected this after I pointed out the mistakes, but only partially (it still contains obvious howlers). Remarkably, Lambert admitted that he hadn’t read to the end of the media articles that he was supposedly analyzing.

More recently, Lambert seemed out of his depth when attempting to debunk an award-winning paper published by the Journal of Peace Research (Bias in Epidemiological Studies of Conflict Mortality, by Neil F. Johnson et al). This paper is better known as the “main street bias” work which was critical of the 2006 Lancet study on Iraqi deaths (and which has been expanded in a new paper for the European Physics Letters journal).

Lambert’s attempts to debunk this research (which he’d previously dismissed as “bogus”) boil down to three banal observations:

  1. That it’s possible to disagree with the example parameter values used in the paper (values used to suggest possible bias in the Lancet Iraq study – nobody knows the actual values, partly because the Lancet authors won’t release necessary information). (Main blog post)
  2. That you can construct a hypothetical case which ignores the premises of the main street bias model in order to “demonstrate” that you can ignore those premises. (Comment #55)
  3. That you can falsely impute assumptions to the MSB paper if you can’t come up with anything better to debunk it with. (Comment #144)

Lambert wouldn’t see it that way, of course, but his attempts to debunk the study are themselves pretty convincingly debunked (eg see comments #111, #153, #180, and my own #3, #132, #150, etc) after which he doesn’t have much more to say. Still, his blog is sometimes worth reading – for the scrutiny of spurious punditry, and for the hilarious outbursts of pompous credentialism (such as when a poster who called himself Expert Statistician wrote: “Go get a PhD in stats, then we’ll talk”).

Postscript – Lambert on Main Street Bias

[The following was originally a separate post, from 22/2/09 – I’ve incorporated it into the above to remove clutter]

Tim Lambert wrote a second piece attempting to debunk MSB. This time he drew a corner of a map, hoping to show that there may be no sampling bias from “main” streets in the 2006 Lancet Iraq study.

I’ve drawn a map (see below) illustrating the problems with Lambert’s assumptions, although you probably need to be familiar with this debate for it to make any sense.

Click to see full size
Click to see full-size

Please click on the map to see it at readable size.

Lambert has now replied to me that the road he redesignated as a “main” street (but which wasn’t shown in his own map) was the one which runs upwards through junction A (on my map). In his blog, Lambert describes this as an “obvious” main street, but it’s nothing of the sort (it’s half the width of Lambert’s other designated “main” street, for example).

The best that Lambert can claim for one of his secondary (or “cross”) roads (road 1 in my map) is that it joins another cross road (2) at junction A, and that the road which he has arbitrarily designated as a “main” street runs through the same junction.

In other words, even if a survey team agreed with Lambert’s arbitrary designation of the junction A road as a “main” street, it’s debatable whether (using the published Lancet sampling methodology) they would select road 1 as a cross street. Looking at Junction A, it seems equally (or perhaps more) likely that they’d class road 2 as the secondary street, with road 1 as a tertiary road leading off road 2. But nobody actually knows, since nobody knows how the Lancet sampling scheme worked in reality.

What this ambiguity over classes of road shows, at this level of detail, is that Lambert is misleading his readers when he claims that the MSB map (which he redrew) is “wrong”. The most he can say is that he has a different subjective designation of roads, which has its own problems in terms of plausibility.

Lambert’s map left out the least plausible part of his selection scheme.