First, Do Some Harm, Part IV: Fake Submission to Fake Conference Yields Fake Charge of Misconduct

Professor Jim Vander Putten, who spent six years as chair of the University of Arkansas Little Rock (UALR) IRB, is now charged with violating university rules by conducting research without that board’s approval. The case highlights several problems with the current system, most notably its failure to provide standards for studies designed to expose misbehavior.

[Peter Schmidt, “A Scholar’s Sting of Education Conferences Stirs a Hornet’s Nest,” Chronicle of Higher Education, March 14, 2016, paywalled.]

Scamming the scammers

The Chronicle gives the basic facts of the study:

Jim Vander Putten suspected that some education conferences accepted any study pitched by someone willing to pay a registration fee. He worried that the gatherings enabled scholars to pad their publishing records while tainting research in the field.

To test his hypothesis, he sent fake research-paper summaries larded with unforgivable methodological errors to the organizers of 15 conferences he believed to have lax standards. All responded by offering to let him present his findings and to publish his papers as part of their proceedings.

Before proceeding with the study, Vander Putten spent 18 months trying to get approval from the UALR IRB, but it eventually ruled that, in the Chronicle’s words, the proposal “had failed to assuage its concerns over issues such as potential harm to reviewers, whom conference organizers or people willing to investigate the review process might be able to identify.” Rather than accept defeat, he turned to Solutions IRB, a commercial firm in Little Rock, which approved the study.

Issue 1: Can researchers choose their IRBs?

By seeking a second opinion on his proposal, Vander Putten engaged in what is called “IRB shopping.” As Michelle Meyer notes in the story, this is rare among academics, because university policies usually require their affiliates to use university or other designated boards. And it’s now going to be rarer, in that both UALR and Solutions IRB have changed their policies to block future researchers from repeating Vander Putten’s gambit. I myself would like to see more IRB shopping, not less, since it competition could require higher standards at university IRBs.

Issue 2: Should IRBs protect their institutions?

The Chronicle paraphrases a university administrator on the mission of an IRB:

Scott L. Thomas, dean of Claremont Graduate University’s School of Educational Studies and president of the Association for the Study of Higher Education, said it is a mistake to focus entirely on the question of whether a given study protected human subjects. University review boards, he said, function not just to protect human subjects, but to protect university interests and to verify that the research is insured. “There are liabilities and risks associated, for both the university and the researchers,” he said.

Federal law states that IRBs must be established “to review biomedical and behavioral research involving human subjects conducted at or sponsored by such entity in order to protect the rights of the human subjects of such research,” not to protect university interests. As Sanjay Srivastava notes, it’s a bit unusual for an administrator to be so blunt about the ways universities have repurposed the boards to protect themselves from embarrassment. But not unprecedented. And everyone knows this is how things really work.

Issue 3: Should IRBs protect powerful malfeasors?

To me, the most interesting question is how the UALR IRB should have reviewed Vander Putten’s proposal, given that it was largely designed to expose wrongdoing.

The Chronicle calls Vander Putten’s project a “sting;” the fancier term “audit study” is often used to denote the submission of fake job applications, housing applications, and the like, to see if the people on the receiving end are playing by the rules. (Does “audit study” imply a controlled experiment, e.g., sending actors of different races to apply for the same apartment? I’m not sure.) This technique has a long history in social science and journalism.

Some IRBs have found a way to approve these kinds of studies. For example, the IRBs at Columbia and Penn approved the sending of fake, urgent queries to university professors to see if prospective PhD students with names suggesting they are ethnic or racial minorities were less likely to get responses than those with names suggesting they were white men. And the IRB system itself has been the target of such a sting; in 2009, the Government Accountability Office used undercover tests to show that “the IRB system is vulnerable to unethical manipulation.”

But other IRBs have blocked research that could expose misbehavior, as when a team wanted to send San Francisco residents into stores to request loose cigarettes, and then record if the stores offered to make those illegal sales. The UCSF IRB forbad that practice for unclear reasons. The Chronicle article does not provide any documents or statements from the UALR IRB, but I suspect they would provide the same muddled thinking as the UCSF IRB in the loosies case.

If so, it wouldn’t be the individual IRB’s fault; it would be the fault of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. At its January 1978 meeting, staffer Bradford Gray raised this very issue. And, as it did whenever confronted with a challenging case of social science ethics, the commission failed to come to a resolution, and instead published the Belmont Report with its blanket statements about “do no harm.”

As I’ve noted before, in 2010 Canada’s Panel on Research Ethics addressed this problem by declaring:

Some research, involving critical assessments of public, political or corporate institutions and associated public figures, for example, may be legitimately critical and/or opposed to the welfare of those individuals in a position of power, and may cause them some harm. There may be a compelling public interest in this research. Therefore, it should not be blocked through the use of risk-benefit analysis. Such research should be carried out according to the professional standards of the relevant discipline(s) or field(s) of research.

The scammy conference organizers were in positions of power. Had the UALR IRB followed this guidance, Vander Putten could have conducted his research without fuss.