Thursday, December 15, 2011

Halavais Calls for "Open Publication of IRB Protocols or Ethics Reflections"

In an essay in Nature and on his blog, Alexander Halavais, president of the Association of Internet Researchers, calls for funders to require "the open publication of IRB protocols or ethics reflections."

[Alexander Halavais, "Social Science: Open Up Online Research," Nature 480 (8 December 2011): 174–175, doi:10.1038/480174a; Alexander Halavais, "IRBs and Clean Secrets, A Thaumaturgical Compendium, 8 December 2011.]

The Problem: Secrecy

In his Nature essay, Halavais details the problems that Internet researchers face when submitting protocols to inexpert boards.

High levels of scrutiny are clearly necessary for a drug trial. But scrutinizing whether gamers would be traumatized by being asked questions about dressing up as characters for conventions — to take an example from my students' research — is an issue best addressed by the researchers, who have had much more exposure to the participants and the culture being examined.

For those who research online interactions, it can be especially frustrating to have a board filled with members who have never used Facebook or played World of Warcraft. Although IRBs can, and sometimes do, bring in experts who can address the context of the research more directly, this happens more rarely than it should . . .

The decisions of IRBs seem to be idiosyncratic and, by extension, capricious, especially when multi-site research is approved by several boards, yet held up by others. In the case of a colleague, each of two review boards insisted on having the other approve a protocol first.

Looking for the source of IRB caprice, he finds it in IRB secrecy:

The greatest problem faced by the ethics system is secrecy. Review boards must make decisions with limited access to previous cases. Their decisions are rarely available to other IRBs or to researchers who could make productive use of their precedents — particularly in new areas of research such as online social science, in which review boards tend to have little experience and would therefore benefit most from the experience of others.

This, of course, has been a problem since at least 1973, when Jay Katz testified that "The review committees work in isolation from one another, and no mechanisms have been established for disseminating whatever knowledge is gained from their individual experiences."

The Solution: Transparency

Katz's proposed remedy was "the publication of the important [IRB] decisions rendered at the local and national level. This would radically change the current uninformed and secretive climate which pervades research decisionmaking. At present, decisionmaking remains divorced from pertinent prior decisions of other committees or from scholarly and public evaluation and criticism." He wanted Congress to establish this system, with a permanent National Human Investigation Board at its apex.

[Jay Katz, testimony, U.S. Senate, Quality of Health Care—Human Experimentation, 1973: Hearings before the Subcommittee on Health of the Committee on Labor and Public Welfare, Part 3 (93d Cong., 1st sess., 1973), 1053.]

Halavais, too, wants research ethics to be made transparent, but he doesn't count on Congress or IRBs to lead the way. As he notes in Nature, "The IRB system is conservative by design; we must look elsewhere for change."

So instead of seeking the publication of IRB decisions (the outputs), Halavais wants the publication of researcher applications (the inputs). And, he blogs, even "if you don’t have to have IRB approval, I think funders should still require you to talk about the ethical considerations of your research, and journals should require you to publish this online when you do not have an IRB-approved protocol to provide." He elaborates (in Nature):

Most government funding agencies and private foundations prominently promote open data sharing and collaboration. Were they to make the open publication of IRB protocols or ethics reflections a requirement for receiving funding, it would provide a crack in an otherwise too-secretive process. Many journals expect social-science research to have been inspected by an IRB before submission; a few even require authors to sign a statement of IRB approval. None requires that the approved protocol be provided to the journal or published. They should. Although some information would need to be redacted from such documents — including anything that might violate the privacy of participants or of the researchers — they would open a new window on ethics considerations.

Some Questions

I applaud Halavais for this bold reimagining of a system of research ethics, and I hope he will flesh out some needed details.

1. Is boilerplate OK?

Halavais wants researchers to "talk about the ethical considerations of your research," but he doesn't explain what that would look like. As Rena Lederman noted in 2007, neuroscientists using an fMRI machine

developed a standard set of very detailed responses to the IRB questionnaire (e.g., about what subjects will experience, the safety risks they will face, and the measures that will be taken to minimize those risks). The only parts of the questionnaire that are individualized—apart from the researchers’ names, ranks, and contact information—are the brief project descriptions (what they are looking for this time and what they hope to learn from the data). Generally, since the basic responses are all, by now, familiar to our IRB, the main ethical dilemma faced in discussing fMRI protocols concerns the possibility that a pathology might show up in a brain scan and the consequent question of whether and how the researchers ought to inform subjects.

Lederman used this model to write "standardized descriptions of anthropological research," noting, for example, the inappropriateness of consent forms. She then posted this for others to use. Would such standardized descriptions satisfy Halavais?

(Side note. In the comments section to Lederman's post, one researcher reports trying out Lederman's application and being told by the IRB to get written consent anyway.)

2. Is an approved protocol the same as an ethical reflection?

L. L. Wynn surveyed ethnographers and found that "32 percent of the respondents actually believed that the [committee-required] modifications were 'detrimental to the welfare of the informants or research participants.'" If an IRB forces a researcher to change her plans in ways that endanger research participants, it seems unfair to require that researcher to publish the flawed protocol unless it is accompanied by a disclaimer.

Now, that could lead to some interesting results if researchers routinely declared, in writing, that they thought their IRBs were endangering research participants. But I wonder if Halavais intends to put researchers in so awkard a spot.

3. Why keep the IRB?

In a throwaway line in his Nature piece, Halavais writes, "The solution is not to do away with the IRB, but rather to make amendments that render its dysfunctions less acute."


In sketching a requirement for public declarations of ethics, Halavais envisions a complex system that brings researchers, funders, journals, and readers into dialogue about research ethics, without necessarily involving IRBs. If such a system gets off the ground, why would we want to maintain a system of "idiosyncratic and, by extension, capricious" review by ignorant IRBs?

Just because the IRB system has been around a long time is no reason to assume it should persist.

No comments: