Violating “fundamental standards of science”?

In what ABC News calls a “highly unusual rebuke”, the American Association for Public Opinion Research (AAPOR) has accused Gilbert Burnham, lead author of the 2006 Lancet study on Iraqi deaths, of violating its code of professional ethics.

According to ABC, the Association last brought such a charge of ethics violation 12 years ago, against rightwing pollster Frank Luntz. Burnham is accused of repeatedly refusing “to make public essential facts about his research” (on Iraqi deaths). AAPOR holds that researchers must make available for public disclosure essential information such as the wording of survey questions and other basic methodological details.

Burnham isn’t a member of AAPOR, but that isn’t particularly relevant given that the charge is of a serious breach of widely accepted professional ethics. In fact AAPOR seem to go out of their way to stress the gravity of the case. In their press release, AAPOR’s President, Richard Kulka, says the following:

“When researchers draw important conclusions and make public statements and arguments based on survey research data, then subsequently refuse to answer even basic questions about how their research was conducted, this violates the fundamental standards of science, seriously undermines open public debate on critical issues, and undermines the credibility of all survey and public opinion research. These concerns have been at the foundation of AAPOR’s standards and professional code throughout our history, and when these principles have clearly been violated, making the public aware of these violations is in integral part of our mission and values as a professional organization.” [My emphasis – RS]

David Marker (chair of the American Statistical Association’s Scientific and Public Affairs Advisory Committee), in a Methodological Review of the Lancet study, raises the issue of the AAPOR-published “Best Methods” for prevention of interviewer falsification in survey research. He comments:

A few years ago, 35 leading survey researchers issued a consensus statement on how to minimize interviewer falsification of data (AAPOR 2003). This statement has been endorsed by the American Association for Public Opinion Research and the Survey Research Methods Section of the American Statistical Association. They listed eight factors that could affect falsification rates. Inadequate supervision, poor quality control and off-site isolation of interviewers were three of those factors that are present in this [Lancet] study. The remaining five factors (training on falsification, interviewer motivation, inadequate compensation, piece-rate compensation, and excessive workload) are harder to assess in this situation due to the limited information available on these topics.

In a paper to be published in Defence and Peace Economics, Professor Michael Spagat, of Royal Holloway, had previously documented areas in which the Lancet study appears to violate AAPOR’s code of professional ethics and practices: Ethical and Data-Integrity Problems in the Second Lancet Survey of Mortality in Iraq

See also:
http://www.mediahell.org/lancet_unspeak.htm
http://news.bbc.co.uk/1/hi/world/middle_east/7869317.stm