Schrag Responds to Responses to Schrag

The June 2012 issue of Research Ethics features four responses to my December 2011 essay, "The Case Against Ethics Review in the Social Sciences." Three scholars based in Canada wrote a joint response, while three in Britain wrote individual replies. I am grateful to all of the respondents for their attention, kind words, and challenging critiques.

• Nicholls, Stuart G., Jamie Brehaut, and Raphae Saginur. “Social Science and Ethics Review: A Question of Practice Not Principle.” Research Ethics 8, no. 2 (June 2012): 71–78. doi:10.1177/1747016112445435
• Hedgecoe, Adam. “The Problems of Presumed Isomorphism and the Ethics Review of Social Science: A Response to Schrag.” Research Ethics 8, no. 2 (June 2012): 79–86. doi:10.1177/1747016112445437
• Jennings, Sean. “Response to Schrag: What Are Ethics Committees for Anyway? A Defence of Social Science Research Ethics Review.” Research Ethics 8, no. 2 (June 2012): 87–96. doi:10.1177/1747016112445423
• Bond, Tim. “Ethical Imperialism or Ethical Mindfulness? Rethinking Ethical Review for Social Sciences.” Research Ethics 8, no. 2 (June 2012): 97–112. doi:10.1177/1747016112445423

Since the responses overlap somewhat in their themes, I think it best for me to respond to them collectively.

Benefits

In my essay, I argued that "ethics review has few success stories." Two respondents reply by asserting that there are success stories; I just don't know about them. Sean Jennings claims that "anyone who has worked with ethics committees for any length of time has reason to believe that while there are many very good and ethical researchers, there are also a number who are just blind to significant ethical questions that their work gives rise to." And Tim Bond writes that "each year the review process [at the University of Bristol] has discovered and prevented a small number of potential situations in which participants were vulnerable to significant harm."

Neither respondent offers examples of such flawed proposals. I would suggest that this opacity is in itself a great flaw in the present system, for it prevents the kind of discussion of research ethics advocated by Stuart Nicholls, Jamie Brehaut, and Raphae Saginur, and instead leads ethics committees and resarchers to talk past each other. Somewhere out there, ethics committee members are shaking their heads over the researcher so blind to ethical questions that she didn't know you need signed consent forms from every barista who serves you coffee. If ethics committees rely on the "if you only knew" defense, they will never have such misguided beliefs challenged.

They will also fail to persuade researchers and critics like me of the need for ethics review. If Jennings and Bond were to write up in detail some of the cases in which ethics review prevented a flawed social science project from proceeding, they would have a much better chance of persuading me of the system's benefits, and they would provide researchers at large a chance to think through the ethics of similar projects before they ever face a committee. Alexander Halavais has offered suggestions along these lines.

In the absence of evidence of ethics review's ability to protect people from harm, two responses offer alternative justifications. Nicholls et al. propose that ethics committees can serve as peer review bodies, denying funding and publication to "poorly conducted" research. They do not explain why these decisions should be made by interdisciplinary committees that may have no representatives from the researcher's discipline, rather than the disciplinary groups--funding panels, scholarly conferences and journals, and the like--that have peformed these functions for a century or so.

Bond suggests another justification for mandatory ethics review: he directs his committee to ask, "what is ethically required to ensure public trust in, and willingness to participate in, social science research"?

I am intrigued by this idea, but it raises many questions. Why does Bond think that the public is apt to distrust social scientists and refuse to participate in social science research? Why does he think ethics review builds that trust? As Bond himself notes, "Seeking written consent in cultural contexts where oral transactions are the accepted norm may actively communicate mistrust and alarm participants." Thus, some ethics review decisions work against the goal he sets for ethics review.

We could also test Bond's claims by comparing various regimes over time and space. Did the public trust social scientists more after ethics review was imposed in various countries? Did it trust them less in the United States after social science won broad exemptions in 1981, and more after review became stricter in the 1990s? Are people in countries without ethics review of the social science—e.g., France and Germany—less trustful? Obviously these are empirical questions, and again, I suggest the burden of proof is on the defenders of ethics review.

Finaly, how far is Bond willing to go with this argument? Would he require ethics review for marketers and, especially, journalists? We know that some journalists abuse the public trust, but I know that Americans, at least, would consider mandatory ethics review of journalism an unconstitutional burden on the freedom of the press.

David Hyman has argued that "there is no empirical evidence that IRBs have any benefit whatsoever." I disagree, for I have read many (well, several) accounts by researchers who, like Bond, are grateful to ethics committees for the guidance they offered. But those accounts are much harder to find than are stories of research damaged by inappopriate requirements. The evidence of an overall benefit from ethics review is thin, and the respondents have done little to bulk it up.

Costs

Just as they emphasizes the purported benefits of ethics review, the responses downplay the costs by labelling my arguments as merely pragmatic, not principled. Nicholls et al. argue that "it is not a principle that social science research should not undergo ethical review, but rather that the current process of ethical review can be inappropriate for some social science studies." Similiarly, Jennings argues that "Even if we grant that there are problems with the way that ethics review works in practice for social science, we can maintain that the appropriate response would be to improve the system of review, rather than scrap the system entirely."

I take this to mean that with enough time, money, and effort, ethics review of the social sciences could work a good deal better than it does now. I suppose I agree, and in that sense, my objections are indeed pragmatic rather than principled. But I don't think that distinction is as important as Jennings and Nicholls et al. charge.

I offer, as a comparison, the Constellation Program, a proposal to develop spacecraft that could send six-astronaut teams to the moon. In 2009, the Review of U.S. Human Spaceflight Plans Committee concluded that while such a mission was possible, pursuing it would require the abandonment of more compelling projects, and that "if resources are not available to match established goals, new goals need to be adopted." This was not a rejection of Constellation on principle—a moon mission required no physical impossibilities, and it would achieve desirable results—but its pragmatic reasoning convinced the Obama administration to scrap Constellation entirely. (Congress later revived portions.)

Similarly, with enough resources, ethics committees could perhaps be trained to review social sciences proposals with the same sensitivity that Nicholls et al. say they bring to cluster randomized trials. For example, Philip Rubin and Joan Sieber have suggested that "each department [could] nominate to the IRB those of its members who have extensive and up-to-date methodological expertise and research experience. In turn, those members could assist faculty who teach methodology to include critical curriculum at the intersection of ethics and methodology. For this service, they should be compensated with release time from teaching and recognition in promotion decisions." I have no doubt that this would reduce the problems with ethics committees, but it would also require a substantial shift of resources away from both teaching and research, the primary functions of a university. Since governments and universities are unlikely to support this shift, new goals need to be adopted.

Exemplars

Nicholls et al. caution against "throwing the baby out with the bathwater." Having recently made a similar argument in the course of defending scholarly editing, I am sympathetic to this stance. But in the case of ethics review of the social sciences, it can be awfully hard to find the baby.

In accusing me of "assumed isomorphism," Adam Hedgecoe suggests two, competing answers to this problem. At one level, he suggests that my critique applies only to "one, specific, jurisdiction (i.e. the US IRB system)." That is, the United States has the bathwater, and everyone else has the baby.

Hedgecoe acknowledges that I provided examples of dirty bathwater from Canada, the United Kingdom, and Australia, but thinks five are too few. How many are enough?

As Hedgecoe knows from reading The Seduction of Ethics, Will van den Hoonaard has compiled plenty of horror stories from Canada. Robert Dingwall is probably the best source for British versions, and Hedgecoe's compatriots, Bond and Jennings, had no trouble recognizing the British variants of the problems I outlined. For example, Hedgecoe seems to think that "institutional protection" is a mission unique to U.S. IRBs. He may want to consult Bond, whose ethics committee is charged with protecting not only research participants and academic freedom, but also "the reputation of the University [of Bristol] as a centre for properly conducted, high quality research."

For Australia, I have noted works by Robert Cribb, Greg Bamber and Jennifer Sappey, and Anthony Langlois, as well as Australia's National Statement on Ethical Conduct in Human Research.

Finally, since Hedgecoe thinks that we should trust ethnographic observations of ethics committees at work, I would refer him to the findings of
Maureen H. Fitzgerald, Paul A. Phillips, and Elisa Yule, who observed "29 ethics committee meetings in five countries (13 in Australia, 2 in Canada, 3 in New Zealand, 4 in the United Kingdom, and 7 in the United States)" and interviewed "213 people (79 in Australia, 52 in Canada, 16 in New Zealand, 15 in the United Kingdom, and 51 in the United States) from a total of 37 cities or metropolitan areas." They found that

Despite the fact that the ethics review process in each of the countries involved is based on different sets of regulations or guidelines and laws, our observations suggest that the actual meetings and the narratives within them are similar enough in this case to deal with our body of data as one body of data. Thus, although there are differences across countries and committees, we focus here on a remarkable commonality, the narratives involved. When asked by some committee members after the meetings how they compared to other committees, we often joked that if we could modify the accents of the people involved and a few of the acronyms and terms, people probably could not tell in which country the meeting occurred.

Elsewhere, Fitzgerald and Martin Tolich have written that "In our own research projects, as well as Fitzgerald’s extensive study of ethics committees in five countries (Australia, Canada, New Zealand, United Kingdom, and United States), we have yet to find an ethics committee that reflects qualitative epistemological assumptions."

Thus, Hedgecoe is quite right that my critique follows other scholars in regarding the differences among Anglophone counties as less striking than the similarities. But the isomorphism is observed, not assumed. The bathwater, while filthiest in the United States, is everywhere.

Hedgecoe's second answer is to find a baby within the British bath; he claims that National Health Service (NHS) RECs are "capable of reviewing qualitative social science in a reasonable and supportive manner." He is correct that I found his article an unpersuasive rebuttal of the work of Sue Richardson and Miriam McMullan, who noted sociologists' complaints about the same committees. Perhaps a longer account of his observations would yield examples of serious ethical lapses set right by an NHS committee.

I do share Hedgecoe's wish to find working variants of the ethics review system. Maybe Macquarie University. Maybe Macalester College. But these are at best islands of sanity in a sea of madness. Perhaps my final challenge to my readers should have been to find me an institution whose researchers in the social sciences and humanities broadly agree that the ethics review system is doing more good than harm, and where few scholars are outraged by the mistreatment they have received.

Coercion

In the final sentence of my article, I referred to the "coercive nature of ethics review." More eloquently than I, Bond expresses this central problem. What he wants is

not primarily a coercive relationship but one that is supported by reviewers modelling the ethical qualities they seek to foster in the researchers and working in dialogue with researchers in a timely fashion. Asking the right questions in Socratic style to promote the ethical development of researchers and their engagement with ethical issues that they face has many advantages. It acknowledges the characteristic ethical commitment of most social science researchers rather than undermines it. Listening to the responses to questions with sufficient humility to empathically engage with and demonstrate respect for the researchers, and especially their subjects, provides the best possible basis for constructive outcomes to ethical review.

Yet Bond recognizes that "the legal framework underpinning ethical review in some of the jurisdictions is so protective of the independence of the review process that it prevents any meaningful communication between applicant and reviewer during the research. This may be a further factor that accounts for some of Schrag’s cases. Some systems of review appear to be inherently coercive by design."

I would put that last bit more strongly: almost all systems of review with which I am familiar appear to be inherently coercive by design. And it is ethics committees' reliance on coercion, rather than suasion, that is at the root of the problem, that robs committees of the "sufficient humility" to do their jobs.

Murray Dyck and Gary Allen recently made this point well, when they called for moving "ethical review from approving a proposed project to providing guidance and feedback on submitted projects," that is, reducing ethics committees to advisory bodies. This is consistent with Bond's call for "greater emphasis on suitably thorough and wide-ranging ethics training, relatively limited review at the proposal stage but greater availability of ethical support as the research proceeds, rather than a high stakes review prior to starting the research."

I suppose, then, that my article was incompletely titled. It is mandatory ethics review that should be scrapped. If ethics committees want researchers' respect, let them earn it.