Can community partners replace IRBs for field experiments?

In her contribution to the PS symposium, Melissa Michelson argues that “real-world practitioners” will often know more about relevant ethics and law than will the members of an IRB.

[Melissa R. Michelson, “The Risk of Over-Reliance on the Institutional Review Board: An Approved Project Is Not Always an Ethical Project,” PS: Political Science & Politics 49, no. 02 (April 2016): 299–303, doi:10.1017/S104909651600024X.]

Michelson’s essay is a commentary on the Montana postcard fiasco, also mentioned by Yanow and Schwartz-Shea. Michelson notes that a New Hampshire version of the experiment had been deemed exempt by the Dartmouth IRB, and that generally IRBs are the wrong tools to monitor such research.

There were clear potential consequences that the researchers should have considered. Sending postcards to a large portion of the electorate in such a close race should have raised red flags about the possibility of altering the outcome of the election. Sending postcards with partisan information in a nonpartisan judicial race in a state that had recently endured dark-money scandals should have raised red flags about how the recipients of the mailers would react to outside money coming into their state. Sending postcards linked to elite out-of-state universities should have raised red flags about how the voters of Montana would react to being the guinea pigs in a research experiment.

None of these concerns was likely to be revealed through the IRB process. IRB members are not likely to be experts in electoral law, and it is unreasonable to assume that they would have noted the possible illegal aspects of the experimental proposal. They were also unlikely to know much about Montana politics or that the size of the experiment raised the possibility of changing the outcome of the election.

Instead of relying on IRBs, Michelson advises that researchers (who know most about the proposed research) consult with experts.

Seeking IRB approval should not be equated with a decision that a proposed experiment is ethical. Instead, researchers should start by asking themselves whether they are behaving ethically. As discussed previously, this can be difficult; our enthusiasm for our own ideas can blind us to possible risk and harm.

Another route is for researchers to supplement their own internal check on the ethics of a [get-out-the-vote] experiment by involving partners. This may mean working with candidates, local election officials, or a local community organization. Involving actors who are not political scientists but rather real-world practitioners provides a different perspective and, usually, relevant expertise about legal constraints and possible unintended effects. Working with a partner in Montana, for example, might have helped the Stanford–Dartmouth team to better understand the local political context regarding dark money or led them to design postcards that did not break local laws. This type of pracademic [sic] work has other advantages as well. Of course, partnering with candidates and elected officials also can introduce disadvantages and ethical concerns, particularly if those partners have something to gain from the research. When possible, scholars should seek to partner with independent local actors, such as nonprofit and nonpartisan organizations.

In other words, rather than asking a group of non-experts assembled from the university and community, why not consult people who know something?

While Michelson generally seems to understand that “IRB procedures focus on possible harm to individuals, not election outcomes,” she also writes that “Despite guidelines mandating avoidance of harm to society overall, the IRB process as interpreted at many institutions is solidly for the protection of individuals rather than broader populations.”

What “guidelines”? The Common Rule (§46.111) asks IRBs to consider only risks to subjects. It explicitly states that “the IRB should not consider possible long-range effects of applying knowledge gained in the research (for example, the possible effects of the research on public policy) as among those research risks that fall within the purview of its responsibility.”