Wynn Calls for Department-Level Review of Student Research

L. L. Wynn, an anthropologist at Macquarie University and a member of that university’s Human Research Ethics Committee, spoke to 40 teachers and administrators at 14 Australian universities. She finds that “opportunities for independent undergraduate human research are being eroded by expanding ethics bureaucracies” and that “the ethics review process [is] a significant obstacle to universities and teachers who wish to incorporate original human research into the curriculum.” (7) She calls for the devolution of ethics review to individual departments.

[L. L. Wynn, “The Impact of Ethics Review on a Research-Led University Curriculum Results of a Qualitative Study in Australia,” Journal of Empirical Research on Human Research Ethics, Published online before print, March 16, 2016, doi:10.1177/1556264616636234.]

University of Queensland Punishes Researchers, Won't Say Why

The University of Queensland demoted a professor and blocked him and another researcher from publishing findings, based on charges that they had not obtained necessary ethics clearances. But the university will not explain its conduct.

[Jorge Branco. “UQ Suppressed Bus Racism Study: Academics.” Brisbane Times, February 27, 2015. Thanks to Michelle Meyer for tweeting this to my attention.]

Report from the National Academies Workshop

Last week I attended the Revisions to the “Common Rule” in Relation to Behavioral and Social Sciences Workshop sponsored by the National Academies.

I live-tweeted the event on my @IRBblog account, and I have collected those tweets on Storify.

What follows are what I consider some of the key messages from selected presenters. The statements following each name represent my summary of the remarks, not necessarily a quotation or paraphrase.

Can Macalester's Divisional Review Work Elsewhere?

In his contribution to the Professional Geographer special issue, Dan Trudeau of Macalester College writes that "IRBs can be a pedagogical asset, particularly if institutional review practices cultivate the habits of mind and strategies necessary for engaged and reflexive research." The key, his article suggests, may be the devolution of review to specialized committees rather than the general-purpose IRBs that are the norm. But Trudeau does not stress the degree to which Macalester's sucess depends on its departure from federal standards.

[Dan Trudeau, "IRBs as Asset for Ethics Education in Geography," Professional Geographer 64, no. 1 (2012): 25-33, DOI: 10.1080/00330124.2011.596786.]

Anxious Pessimism on UK's New Framework for Research Ethics

In March 2010, the United Kingdom's Economic and Social Research Council (ESRC) released its Framework for Research Ethics (FRE) as a successor to its 2005 Research Ethics Framework (REF).

David Erdos kindly alerted me to the November 2010 (Volume 15, Issue 4) issue of Sociological Research Online, which devotes a special section to essays about the new framework.

The six essays in the section suggest that British sociologists are wary of their research ethics committees and the expanded authority granted to them by the new framework.

Princeton IRB Delays Student Research

The Daily Princetonian reports a sociology major's difficulties getting IRB approval for her senior thesis on Brazilian immigrants' changing perceptions of gender roles.

"It's such a long process that it thwarts your field work efforts," [Christine] Vidmar said, noting that the review board does not meet to approve proposals during the summer. "I've been waiting since I got back to school. The first deadline that I could apply for was in October. It's November now, and I still can't officially go do my interviews."

. . .

Vidmar noted that a well-researched thesis may require up to a year of field work, adding that review board hurdles make it more challenging to complete sufficient research. "If you're a senior and you don't have a thesis chosen by the spring of junior year then you can't start field research until November or December of senior year, which is really late," she said. "You need to be in the field in order to know what questions you're going to ask, but in order to be in the field you need to have given the IRB your questions ahead of time."

As horror stories go, this one is mild. But consider the following:

• While details are lacking, Vidmar's proposed research sounds to be exempt under federal regulations; she's just interviewing adults about their perceptions of gender.
  
• Princeton demands full board review for "almost all proposals," offering expedited review only on "an exception basis."
  
• The IRB does not meet for three and a half months in the summer and requires proposals to be submitted two weeks in advance of the meeting. Hence, a student who misses the late-May deadline must wait almost four months until late September for review.
  

Put these together, and it seems that Princeton has built a substantial impediment to students who would like to interact with people as a capstone to their undergraduate training but are unable to write detailed research protocols six months in advance.

This is not to say that undergraduates should be sent into the field without training or supervision. But review by at the department level, as suggested by Felice Levine and Paula Skedsvold; subcommittee review, as practiced at Macquarie University; or researcher certification as permitted at the University of Pennsylvania, might well achieve the same or better levels of oversight as full-board review without delaying the work and discouraging the curiosity of a student researcher.

Can We Patch This Flat Tire?

The fourth article in the PS symposium is Felice J. Levine and Paula R. Skedsvold, “Where the Rubber Meets the Road: Aligning IRBs and Research Practice.” Both authors been involved in IRB debates for several years, and this article reflects their sophisticated understanding of some of the issues involved. But for an article published in a political science journal, it is disappointingly insensitive to the power dynamics that govern IRB-researcher relationships.

Unlike symposium participants Tony Porter, Dvora Yanow and Peregrine Schwartz-Shea, Levine and Skedsvold do not question the premise that IRBs help promote ethical research. Instead, they assert that there is no fundamental conflict between IRBs and social science researchers: "federal regulations, professional ethics codes, and research practice may have shared goals but tend to speak with different languages—creating frustration and skepticism in a system that could potentially work quite well if transformations are made." (502) Based on that assertion, they suggest four such transformations, ranging from the bold to the timid.

Law & Society Review, continued

As I noted earlier, the December 2007 issue of Law & Society Review features five items concerning IRBs and the social sciences.

Malcolm M. Feeley, "Legality, Social Research, and the Challenge of Institutional Review Boards"

The section on IRBs begins with Malcolm M. Feeley's 2006 presidential address to the Law & Society Association. Feeley presents an impassioned critique of IRBs, complaining, "in the name of minimizing risks, IRBs subject researchers to petty tyranny. Graduate students and junior scholars are particularly likely to be caught in their web—and for them IRB tyranny is often more than petty. Senior scholars are generally more adept at avoidance, evasion, and adaptation, but they too are hardly exempt from this tyranny. A number of prominent social scientists, including some members of this Association, know all too well the harms of running afoul of campus IRBs. . . Entire research areas and methodologies are in jeopardy, insofar as the difficulties of obtaining IRB approval affect research priorities for funding agencies and universities' willingness to support researchers.”

Feeley then raises a number of specific problems, such as the ill fit between the beneficence encoded in regulation and the kind of social research that aspires to produce "tarnished reputations and forced resignations" of evil-doers.

To remedy this situation, Feely proposes three modes of action:

1. "Join [IRBs]; subvert them—or at least curtail them. Serve on them and do all you possibly can to facilitate the research of your colleagues rather than act as a censor."

2. Follow Richard Schweder's call to get your university to apply federal regulations only to federally funded research.

3. "Ask about estimates of how much actual harm to subjects in social science research has been prevented by IRB actions. And ask for documentation."

I am a bit skeptical about the first suggestion, for two reasons. First, few universities have IRBs strictly for the social sciences. This means that a sociologist, anthropologist, political scientist, or historian would spend most of her time on an IRB reviewing (or abstaining from reviewing) psychological experiments. That's an unfair price to pay to have some power over one's own research. Second, it assumes that IRBs are run by IRB members. As Caroline H. Bledsoe et al. report in "Regulating Creativity: Research and Survival in the IRB Iron Cage," the size of human protections staffs has ballooned in recent years. If the staff have the real power, IRB members will have little chance to facilitate research.

Laura Stark, "Victims in Our Own Minds? IRBs in Myth and Practice."

The first comment is Laura Stark's. It draws in part on Stark's 2006 Princeton dissertation, "Morality in Science: How Research Is Evaluated in the Age of Human Subjects Regulation." I am glad to learn of this work, and I hope to comment on it in a later post.

Stark suggests trying to improve, rather than restrict, IRBs, because “ethics review in some form is here to stay because of institutional inertia, and [because of her] belief as a potential research subject that ethics review is not an entirely bad idea, even for social scientists.” She advocates "changing local practices to suit the local research community, rather than refining federal regulations."

One intriguing example is the establishment of "IRB subcommittees, which can review lower-risk studies [and] have moved ethics review into academic departments. In so doing, these subcommittees of faculty members (who presumably understand the methods in question) have taken over the task of evaluating low-risk studies from board administrators." This sounds a lot like the departmental review that the AAUP suggested as an alternative to IRB control, and like the Macquarie model I described in August. I hope that Stark will publicize the name of the university that uses such subcommittees, so that it can better serve as an example to others. Stark does not explain why this model is appropriate only for low-risk studies. It seems to me the higher the risk, the more reason to have research reviewed by people who understand its methods.

Significantly, neither in her article nor in her dissertation does Stark take up Feeley's challenge to document cases in which IRBs have prevented actual harm to participants in social science research. Her research offers important insights about how IRBs reach decisions, but no evidence that those decisions do more good than harm, or that they are consistent with norms of academic freedom.

Finally, Stark claims, "the social science victim narrative—by which I mean the story that human subjects regulations were not meant to apply to us—is pervasive among academics, and it is particularly central to qualitative researchers as a justification for their criticisms of IRBs. Yet this victim narrative does not stand up to historical scrutiny, as I have shown." Yes and no. Stark's use of the passive voice (were not meant to apply) is telling; the question is who meant the regulations to apply to social scientists, and who did not. I am working on a full-scale history of the imposition of human subjects regulations on the social scientists, and I can tell Stark that more scrutiny will complicate her story.

Robert Dingwall, "Turn off the oxygen …"

The second comment is Robert Dingwall's "Turn off the oxygen …," the oxygen here referring to the legitimacy granted to IRBs by university faculty.

Dingwall is skeptical of legal challenges, given the cost, the possibility of failure, and the fact that the First Amendment only applies to the United States (Dingwall works in the UK.) He argues instead that “if we can show that ethical regulation does not actually contribute to a better society, but to a waste of public funds, serious information deficits for citizens, and long-term economic and, hence, political decline, then we may have identified a set of arguments that might lead to a more skeptical approach to the self-serving claims of the philosopher kings who sustain that system.” For example, we must continue to document ethical wrongs like the insistence by a British medical journal that two historians falsify the names of their oral history narrators, despite the wishes of most of the narrators to be named. [Graham Smith and Malcolm Nicolson, "Re-expressing the Division of British Medicine under the NHS: The Importance of Locality in General Practitioners' Oral Histories," Social Science & Medicine 64 (2007): 938–48.] I hope Professor Dingwall has a chance to read Scott Atran's essay, "Research Police – How a University IRB Thwarts Understanding of Terrorism," posted on this blog in May. It is an excellent example of the way that IRB interference can disrupt vitally important work.

Jack Katz, "Toward a Natural History of Ethical Censorship"

The third comment, by Jack Katz, is the most shocking, for it is the most thoroughly documented. (It even cites this blog, thanks.) Katz lists several cases, all recent, in which IRBs have derailed potentially important social research. Unlike the 2006 AAUP report, he gives names, universities, dates and citations for most of his horror stories. Among them:

* "In Utah, Brigham Young University's IRB blocked an inquiry into the attitudes of homosexual Mormons on their church. When the same anonymous questionnaire study design was transferred to another researcher, the IRB at Idaho State University found the study unproblematic."

* "A proposed study of university admissions practices [was] blocked by an IRB at a Cal State campus. The study had the potential to reveal illegal behavior, namely affirmative action, which was prohibited when Proposition 209 became California law."

* "At UCLA, a labor institute developed a white paper lamenting the health benefits that Indian casinos offered their (largely Mexican and Filipino) workers. Despite the university's support for the labor institute when anti-union legislators at the state capitol have sought to eliminate its funding, publication was banned by the IRB after a complaint by an advocate for Indian tribes that the study had not gone through IRB review."

Stark would have us believe that "the local character of board review does not mean that IRB decisions are wrong so much as that they are idiosyncratic." But Katz shows that IRBs' idiosyncracies can be hard to distinguish from viewpoint-based censorship.

In contrast to these identifiable harms, Katz finds "no historical evidence that the social science and humanistic research now pre-reviewed by IRBs ever harmed subjects significantly, much less in ways that could not be redressed through post hoc remedies." I don't think I would go quite this far, given Carole Gaar Johnson's description of the harms caused to the residents of "Plainville" by the inept anonymization of their town ("Risks in the Publication of Fieldwork," in Joan E. Sieber, ed., The Ethics of Social Research: Fieldwork, Regulation, and Publication (New York: Springer, 1982). But the rarity of such cases means we should weigh IRB review against other methods of prevention, such as departmental review of projects or better certification of researchers.

Katz reiterates his call, previously set forth in the American Ethnologist, for a "culture of legality," in which IRBs would be forced to explain their decisions and "publicly disseminate proposed rules before they take the force of law." He believes that "were IRBs to recognize formally that they cannot properly demand the impossible, were they to invite public discussion of policy alternatives, and were they to open their files to public oversight, they would fundamentally alter the trajectory of institutional development by forcing confrontation with the central value choices currently ignored in the evolution of ethical research culture."

But what do we do when we confront those value choices? We get statements like Stuart Plattner's: “no one should ever be hurt just because they were involved in a research project, if at all possible," a position clearly at odds with Katz's applause for "the American tradition of critical social research." (Plattner, “Human Subjects Protection and Cultural Anthropology,” Anthropological Quarterly, 2003) The problem with IRBs' value choices is not that they are hidden, but that they are often wrong. The Belmont Report is the most public and widely cited rule used by IRBs, and it is a terrible guide for the kind of critical research Feeley and Katz want done.

Feeley, "Response to Comments"

The most interesting part of Feeley's response comes at the very end. Noting that, with the AAUP's encouragement, some universities have ceased promising to review all human subjects research in favor of the regulatory minimum of federally funded research, he points out that we will soon know if the lack of IRB review of social science at those universities yields a flood of unethical research. "If there are few reports of negative consequences . . . they might encourage national officials to rethink the need for such an expansive regulatory system . . . On the other hand, if opt-out results in increased problems, the findings might help convince Katz, Dingwall, me, and still others of the value of IRBs." This strikes me a very fair bet, and the experiment can't begin soon enough.

IRBs vs. Departmental Review

In comments on this blog's introduction, bioethicist David Hunter of the University of Ulster asked me about my preferred alternative to IRB review, and I mentioned my hopes for departmental review (hopes shared by the AAUP). Lest our conversation get lost in the comments, I am moving it to this new posting:

DAVID HUNTER:

I'd disagree on departmental review being best for two reasons.

1. While a committee which has some knowledge and expertise in the area of the project, too much expertise and it becomes too close to the subject matter. This can mean that it misses significant ethical issues because they are standard practice within a specific discipline. To give one example, psychologists often want to give part of their students grade (10%) for being involved in their research. Most RECs I am involved in don't allow this practice because it is felt it is unduly coercive. I imagine if a REC/IRB was entirely composed of psychologists they may disagree.

2. It is important for a REC to be substantially independent from the researcher, but this doesn't happen in departmental review, instead the REC has an interest in the research being let to go ahead.

My university presently runs on a departmental review model, and while I can't name names I have personally seen examples of both of the above issues coming up.

I've written about these problems here:
Hunter, D. 'An alternative model for research ethics review at UK universities' Research Ethics Review. (2006) Vol 2, No 2, 47-51.
(Which unfortunately isn't available online)

and here: Hunter, D. 'Proportional Ethical Review and the Identification of Ethical Issues Journal of Medical Ethics. (2007);33:241-245.

I certainly agree with you that IRBs shouldn't be dominated by medics and medical concerns, they instead should have a wide range of representation. I'm inclined to think though that the baseline ethical issues are similar and while different rules may be appropriate for different disciplines they flow out of the same background.

In terms of examples here are a few, I can't be too specific with details for reasons of confidentiality.

1. Study of sexual attitudes in school children. Asked very probing questions as one might expect, but didn't intend to get parental consent to carry out the research a parallel can be found here: India Research Ethics Scandal: Students made guinea pigs in sex study
No consideration had been given to what might have been done if there was disclosure of harmful behaviour etc.

2. Historian was going to civil war stricken country to interview dissidents about the war, intended to publish identifying comments (without getting consent for this) which were likely to be highly critical of the current regime.

3. Social scientist wanted to understand children's attitudes towards a particular topic. As a blind so that the participant would not know the questions they wanted to answers to, they proposed to use the becks depression index. This contains questions about self harm, future worth and was potentially very distressing, not at all appropriate as a blind.

4. Student wished to conduct interviews with employees of a company on an issue that could significantly damage the companies profitability. No consideration was given to how to best report this information to minimise harm to the company.

I'm inclined to think that any sort of research involving humans can lead to harm whether that is physical, social, financial, psychological or so on. As such the benefits and the risks need to be balanced, and it needs to be considered how to minimise that harm. That I take it is the job of the researcher. However, having sat on RECs for a while it is a job that sometimes the researchers fail at spectacularly, then it becomes the job of the IRB/REC. The difficulty is how, without full review by a properly constituted REC, do you identify those applications that have serious ethical issues?

ZACHARY SCHRAG:

Thanks for these examples.

First, let me state that I am primarily interested in projects that fit Pattullo's proposal of 1979: “There should be no requirement for prior review of research utilizing legally competent subjects if that research involves neither deceit, nor intrusion upon the subject’s person, nor denial or withholding of accustomed or necessary resources.” Under this formula, the projects invovling children (who are not legally competent) and the project involving undergraduates (whose course credit is an accustomed or necessary resource) would still be subject to review.

That said, I have little confidence that IRBs are the right tool to review such research. As for child research, under U.S. regulations, and, I believe, the rules of most universities, the studies could be approved by three IRB members wholly lacking in expertise on child development. (The regulations encourage but do not require the inclusion of one or more experts when vulnerable populations are involved.) Were I the parent of a child involved in such studies (and I'm proud to say that both my children have furthered the cause of science by participating in language studies), I would greatly prefer that the protocols be reviewed not by a human subjects committee, but by a child subjects committee composed mostly or entirely of people expert in child research.

For the psychology course and the history project, the real question is whether a departmental committee can be trusted to enforce its own discipline's ethical code. The code of the British Psychological Society forbids pressuring students to participate in an experiment. And the ethical guidelines of the the Oral History Society require interviewers "to inform the interviewee of the arrangements to be made for the custody and preservation of the interview and accompanying material, both immediately and in the future, and to indicate any use to which the interview is likely to be put (for example research, education use, transcription, publication, broadcasting)." So yes, those sound like unethical projects.

Perhaps some departments would fail to correct these mistakes, just as some IRBs and RECs get them wrong. At some level this is an empirical question that cannot be answered due to the uniform imposition of IRB review. In the U.S., at least one university (the University of Illinois) had a system of departmental review in psychology that worked without complaint until it was crushed by federal regulation in 1981. With the federal government imposing the same rules nationwide, we can only guess about how well alternatives would work.

Moreover, departmental review would allow committees to bring in considerations unknown to more general ethics committees. For example, the British and American oral history codes require attention to preservation and access to recordings, something that and IRB/REC is unlikely to ask about.

I would also add that something close to departmental review is typical of the standard IRB, i.e., one in a hospital or medical school. It's true that the U.S. regulations require "at least one member whose primary concerns are in nonscientific areas" and "at least one member who is not otherwise affiliated with the institution and who is not part of the immediate family of a person who is affiliated with the institution." But the rest of the members can be biomedical researchers of one stripe or another. If that's good enough for the doctors, how about letting each social science discipline form an IRB of its members, with a community member and a non-researcher thrown in?

Still, if IRBs/RECs limited themselves to holding researchers up to the standards of the researchers' own academic discipline, I wouldn't be complaining.

Where we really disagree, then, is on project 4. You write, a "Student wished to conduct interviews with employees of a company on an issue that could significantly damage the company's profitability. No consideration was given to how to best report this information to minimise harm to the company."

That sounds a lot like this case:

Kobi Alexander's stellar business career began to unravel in early March with a call from a reporter asking why his stock options had often been granted at the bottom of sharp dips in the stock price of the telecom company he headed, Comverse Technology Inc.

According to an affidavit by a Federal Bureau of Investigation agent, unsealed in Brooklyn, N.Y., the call to a Comverse director set off a furious chain of events inside the company that culminated yesterday in criminal charges against Mr. Alexander and two other former executives. Federal authorities alleged the trio were key players in a decade-long fraudulent scheme to manipulate the company's stock options to enrich themselves and other employees.

After the March 3 phone call from a Wall Street Journal reporter, the FBI affidavit said, Mr. Alexander and the other two executives, former chief financial officer David Kreinberg and former senior general counsel William F. Sorin, attempted to hide the scheme. Their actions allegedly included lying to a company lawyer, misleading auditors and attempting to alter computer records to hide a secret options-related slush fund, originally nicknamed "I.M. Fanton." It wasn't until a dramatic series of confessions later in March, the affidavit said, that the executives admitted having backdated options. The trio resigned in May.

That's an excerpt from Charles Forelle and James Bandler, "Dating Game -- Stock-Options Criminal Charge: Slush Fund and Fake Employees," Wall Street Journal, 10 August 2006. As far as I can tell, Forelle and Bandler made no effort to minimize the harms to the companies they studied or the executives they interviewed. Their "Perfect Payday" series won the 2007 Pulitzer Prize for public service.

Your insistence that an interviewer minimize harm is a good example of an effort to impose medical ethics on non-medical research, and a good reason to get RECs away from social science.