Misrepresenting Iraq Body Count October 22, 2008Posted by dissident93 in Iraq, Iraq mortality, Media watchdogs.
Note: An extended, updated version of this post
was published as a featured article by ZNet.
David Edwards and David Cromwell (editors of Media Lens) have published several articles attacking Iraq Body Count (IBC). Their claims have been widely circulated as part of a sustained, vigorous (and at times aggressive) campaign against IBC. But Media Lens’s case against IBC is riddled with errors, and is discredited by recent research (WHO/IFHS, CRED, etc).
Basic errors by Media Lens
• One of the main premises of Media Lens’s campaign against IBC is that “IBC is not primarily an Iraq Body Count, it is not even an Iraq Media Body count, it is an Iraq Western Media Body Count” (Media Lens 14/3/06, my emphasis).
This is entirely false. IBC use non-Western media sources and non-media sources (eg hospital, morgue and NGO data). They monitor 72 major “non-Western” media on a daily basis, along with 120 “Western” sources. (IBC)
Many incidents/deaths in IBC’s database are from the major wire agencies. This merely reflects the fact that, for example, Reuters covers by far the highest percentage – approximately 50% of documented incidents, compared to 35% from Al Sharqiyah TV (another IBC source), and much lower coverage by other media sources, “Western” or “non-Western” (IBC). Note also that at the level of reporting utilized by IBC, the dichotomy of “Western” vs “non-Western” is false, as agencies such as Reuters employ (for example) Iraqi journalists in covering Iraqi incidents (“We mainly use local reporters, Arab reporters can go out and talk to people” – Reuters’ Baghdad bureau chief).
• In their first article targeting IBC, Media Lens wrote:
“Whereas the Lancet report estimated around 100,000 civilian deaths in October 2004, IBC reported 17,000 at that time.”
This is incorrect in two ways, and is typical of the lack of thoroughness in Media Lens’s research. First, the Lancet study didn’t estimate “civilian” deaths as Media Lens claim (its estimate includes “combatants” as well as civilians). Second, IBC record only violent deaths, so the comparison should be between 57,600 and 17,687 (57,600 being the Lancet study’s estimate of violent deaths, according to Lancet co-author Richard Garfield). But even that isn’t comparing like with like, since IBC do not include combatant deaths, whereas the Lancet study does.
• One of Media Lens’s main claims is that IBC captures only “5-10% of the true death toll”. One can see immediately that this isn’t supported by their comparison of Lancet 2004 and IBC. (IBC’s count of violent civilian deaths is 30% of Lancet 2004’s estimate of total violent deaths. In other words, IBC is capturing much more than 30% of the “true death toll” of violent civilian deaths, given Lancet 2004 as a measure).
(It’s interesting to note that later estimates, eg from IFHS and CRED, show that IBC is capturing at least around a third of violent civilian deaths – contrary to the claims of Media Lens. An earlier estimate, from ILCS, shows IBC capturing well over a half of violent civilian deaths.)
Latest errors by Media Lens
Media Lens continue their attack on IBC in a more recent article, Iraq Body Count: “A Very Misleading Exercise”. This contains several misrepresentations and errors, which I list below:
1. Media Lens write: “IBC’s response to the suggestion that violence prevents journalists from capturing many deaths has been, in effect, ‘Prove it!'”
This is plainly false. IBC have always stated that “many if not most civilian casualties will go unreported by the media. That is the sad nature of war.” Media Lens are aware of this (they’ve quoted IBC’s statement) and cannot claim ignorance.
2. Media Lens: “It is striking that IBC link to a high-profile media report that so badly misrepresents its figures”.
This is misleading. The purpose of IBC’s link (titled “Lists of victims or victim categories to signal the pervasive impact on every sector of Iraqi society”) is to provide an example of how media have used IBC’s data on individual victims (see lower section of the cited article, which is clearly titled “Victims’ Stories“).
Whether Media Lens’s assertion that the article “misrepresents” IBC figures has any merit or not is irrelevant to the point of the link. IBC doesn’t endorse misrepresentations of its figures.
(Given Media Lens’s advocacy for the Lancet studies on Iraq mortality, it’s “striking” that they fail to mention a similar misrepresentation of IBC’s figures by the Lancet study’s authors, in an article for Slate magazine: “Today, IBC estimates there have been 45,000 to 50,000 violent deaths”).
3. Media Lens: “Whereas IBC have responded vigorously, indeed tirelessly, in responding [sic] to the 2004 and 2006 Lancet studies…”
In fact IBC released only two documents commenting on Lancet 2006 (both mildly critical) and one on Lancet 2004 (uncritical):
http://www.iraqbodycount.org/analysis/beyond/state-of-knowledge/ (only part of this document deals with Lancet 2006).
That Media Lens is now condemning IBC for “responding” to the Lancet studies is itself an ironic turn of events. One of the main complaints of Media Lens’s earlier articles targeting IBC was that IBC were “Refusing to Respond”.
— Medialens alert, 14 March 2006—
4. Media Lens: “It was [Marc] Herold’s Afghan Victim Memorial Project that inspired John Sloboda to set up IBC. Herold’s ‘most conservative estimate’ of Afghan civilian deaths resulting from American/NATO operations is between 5,700 and 6,500. But, he cautions, this is ‘probably a vast underestimate’
[...] There is no reason to believe that the application of the same methodology in Iraq is generating very different results.”
Again this is mistaken and misleading. IBC use the same general approach as Marc Herold has used for Afghanistan, but they don’t use the same methodology. One of three reasons listed by Herold in support of his comment in the same article is that his count includes civilian victims directly killed by US/NATO bombings and military action, while excluding victims of the Taliban or other perpetrators. IBC of course includes killings by any perpetrators in Iraq. There are several other differences in the methodologies, and there are also reasons to believe the approach in Iraq is generating somewhat different results than in Afghanistan. But it is unlikely that Media Lens have looked into the matter in enough depth to know the reasons. They have not looked into the matter closely enough even to know that there are differences in the methodologies, or even to know that it is not Herold’s “Afghan Victim Memorial Project” (begun in 2004) that inspired IBC, but rather his “Daily Casualty Count of Afghan Civilians Killed by U.S. Bombing” – begun in 2001), two wholly different projects.
In any case, Herold has now written to ZNet stating that the paragraph written by Edwards and Cromwell has inaccuracies which need to be corrected, and that the inference drawn from it regarding IBC is unwarranted.
5. Media Lens: “…what IBC is doing to promote or reduce the confusion”.
This is an unworthy insinuation, suggesting IBC are “promoting” confusion, but providing no examples of this.
6. Media Lens: “Well, the bureau chief of one of three Western media agencies providing a third of IBC’s data from Iraq sent this email to a colleague last year (the latter asked us to preserve the sender’s anonymity)”.
Media Lens also cited an “anonymous epidemiologist” in their earlier pieces targeting IBC. It was noteworthy then, as it is now with this anonymous “bureau chief” and “colleague”, that these unnamed sources weren’t able to send their comments directly to IBC (who would, of course, have treated them in confidence), or stand behind them publicly. In effect it amounts to 3rd-hand rumour-mongering.
7. Media Lens: “…a new ORB poll revealing that 1.2 million Iraqis had been murdered since the 2003 invasion”.
This is inaccurate. ORB estimated 1.2 million murders. They did not “reveal” any number of actual murders. Note also that the ORB poll wasn’t peer-reviewed science. According to ORB’s publicity literature, the person conducting ORB’s poll, Munqith Daghir, began his polling career in 2003, with little in the way of formal training or field experience. The ORB poll doesn’t have the scientific standing of major studies such as ILCS, which Media Lens failed to mention.
8. Media Lens: “Why is it important for IBC [...] to challenge the methodology and conclusions of epidemiological studies published in the Lancet…”.
IBC didn’t “challenge” Lancet 2004 (see IBC’s uncritical press release on Lancet 2004), so Media Lens are incorrect to write “studies” (plural). And other leading researchers besides IBC have expressed scepticism over the Lancet 2006 estimates: Jon Pedersen of the UNDP Iraq study, demographer Beth Osborne Daponte, Fritz Scheuren, a past president of the American Statistical Association, Professor Hans Rosling and Dr Johan Von Schreeb at the Karolinska Institute in Stockholm, Oxford physicists Neil Johnson and Sean Gourley, Debarati Guha-Sapir, Director of the WHO Collaborating Centre for Research on the Epidemiology of Disasters (CRED), among many others.
9. Media Lens: “Secondly, while IBC’s self-described task does indeed require only “care and literacy”, does not the task of challenging peer-reviewed science published by some of the world’s leading epidemiologists require very much more? Does it not, in fact ‘require statistical analysis or extrapolations,’…”.
In fact, it does not require “statistical analysis” to observe that the Lancet 2006 figure implies that half a million death certificates are missing. It does not require “extrapolations” to observe contradictions in the accounts of the Lancet 2006 team’s description of sampling, or to note that the sampling methodology as published wouldn’t give you “random” street selection. You don’t need “world’s leading epidemiologists” to appreciate how important random sampling is.
The rhetorical basis of Media Lens’s campaign against IBC is: “how dare these data collectors tirelessly and vigorously criticise an epidemiological study”. It’s a weak and misleading argument, and an appeal to crass credentialism. It’s noteworthy that Media Lens don’t apply the same credentialist standards to the ORB poll which they endorse (and which, as noted above, is not peer-reviewed science). It’s noteworthy also that (to date) Media Lens have ignored a large body of science (from leaders in the fields of demography and epidemiology) which tends to support and confirm the data collected by IBC.
Bear in mind that Media Lens went as far as writing (in a letter to New Statesman magazine, 16/10/06) that, “to our knowledge, IBC has not been able to demonstrate support for its methods from a single professional epidemiologist”. Presumably they weren’t paying attention to the views of leading epidemiologists such as Debarati Guha-Sapir, Olivier Degomme, Mohamed M. Ali, Colin Mathers, J. Ties Boerma, etc.