The Systematic Threat to Academic Freedom

Lisa Rasmussen kindly alerted me to her essay, "Problems with Minimal-Risk Research Oversight: A Threat to Academic Freedom?" IRB: Ethics & Human Research 31 (May 2009): 11-16. The essay mostly seeks to rebut the AAUP's 2006 report, "Research on Human Subjects: Academic Freedom and the Institutional Review Board." Rasmussen identifies some important shortcomings in that report, and she raises key questions about the relationship between IRBs and academic freedom. But I am unpersuaded by her central arguments.

Before I address them, I should note the repeated disclaimers within the essay. "I will not settle here the fundamental issue of whether a convincing argument exists that IRB review poses a threat to academic freedom," Rasmussen writes. "A longer explanation of the [AAUP report's] failures is beyond the scope of this paper, but a brief outline is possible." I am disappointed by these limits. Rasmussen devotes significant space to matters peripheral to the question of academic freedom, such as her assertion that researchers whose work was approved by a department--rather than a central IRB--would necessarily merit less legal protection, a claim whose weakness she acknowledges in a footnote. Given only six pages, Rasmussen would have done better to focus on the question posed in her title.

Rasmussen's main argument is that the AAUP report "does not demonstrate that IRBs pose a threat to academic freedom." As she notes, such a demonstration would require a definition of academic freedom, something lacking in the AAUP report. So she offers a passage from the AAUP's "1940 Statement of Principles on Academic Freedom and Tenure": "Institutions of higher education are conducted for the common good and not to further the interest of either the individual teacher or the institution as a whole. The common good depends upon the free search for truth and its free exposition." Emphasizing the grounding of this argument in the search for the "common good," Rasmussen then concludes that "there is a prima facie claim that research can be subjected to assessment regarding whether it threatens to harm the common good via harm to individuals."

I believe this is a misreading of the 1940 Statement, for it suggests that any policy aimed at safeguarding the common good is consistent with academic freedom. For example, she could have written, "there is a prima facie claim that research can be subjected to assessment regarding whether it threatens to harm the common good via the promotion of communist overthrow of the government," and that therefore a prohibition on the use of Marxist analysis is consistent with academic freedom.

A more relevant definition of academic freedom can be drawn from the AAUP's 1915 Declaration of Principles on Academic Freedom and Academic Tenure":

The liberty of the scholar within the university to set forth his conclusions, be they what they may, is conditioned by their being conclusions gained by a scholar’s method and held in a scholar’s spirit; that is to say, they must be the fruits of competent and patient and sincere inquiry, and they should be set forth with dignity, courtesy, and temperateness of language . . .

It is, however . . . inadmissible that the power of determining when departures from the requirements of the scientific spirit and method have occurred, should be vested in bodies not composed of members of the academic profession. Such bodies necessarily lack full competency to judge of those requirements; their intervention can never be exempt from the suspicion that it is dictated by other motives than zeal for the integrity of science; and it is, in any case, unsuitable to the dignity of a great profession that the initial responsibility for the maintenance of its professional standards should not be in the hands of its own members. It follows that university teachers must be prepared to assume this responsibility for themselves.

As Matthew W. Finkin and Robert C. Post write in their new book, For the Common Good: Principles of American Academic Freedom, freedom of research depends on "a framework of accepted professional norms that distinguish research that contributes to knowledge from research that does not." (54) While these two experts on academic freedom decline to offer a firm opinion on the legitimacy of IRBs, they take the AAUP's concerns far more seriously than does Rasmussen (69).

The question, then, is whether IRBs, like the boards of trustees that concerned the authors of the 1915 statement, "lack full competency to judge of [scholarly] requirements." Rasmussen suggests that IRBs merely maintain scholarly standards: "The source of the threat to academic freedom via oversight by one’s colleagues is far from clear," she writes, "especially since researchers undergo peer review for research funding and when submitting their manuscripts for publication." But IRB review is not peer review, since it is conducted mostly by people ignorant of the scholarly methods they are reviewing. (See "Why IRBs Are Not Peer Review," and other posts tagged "peer review.")

To make this a bit more concrete, we can examine the exemplary "horror stories" included in the 2006 AAUP report. Rasmussen rejects these as "unelaborated anecdotes with no documenting citations," rather than examining their implications for academic freedom.

Here's one: "A Caucasian PhD student, seeking to study career expectations in relation to ethnicity, was told by the IRB that African American PhD students could not be interviewed because it might be traumatic for them to be interviewed by the student." Or another: "A campus IRB attempted to deny an MA student her diploma because she did not obtain IRB approval for calling newspaper executives to ask for copies of printed material generally available to the public." No peer review process would impose such conditions. If these are not infringements of academic freedom, then nothing is.

Rasmussen is quite right that we should not equate "inconvenience and hassle with abridgement of academic freedom." Yet nor should we dismiss the abridgement of academic freedom as mere inconvenience and hassle. When IRBs impose conditions on research that prevent researchers from conducting the basic tasks of scholarship--talking to people of varied backgrounds, recording interviews, or telephoning for information--they abridge academic freedom. The more interesting questions are how often this occurs, and why it happens.

Rasmussen presents IRB abuse as a somewhat random process: "IRBs can function well or poorly, and which is true for a given IRB depends on many factors, not least of which are institutional support and member training." This suggests that IRB abuses are individual anomalies, rather than a pattern.

By contrast, the AAUP detects a systematic bias toward the infringement of freedom. This is better developed in the AAUP's 2000 report (cited by Rasmussen), "Institutional Review Boards and Social Science Research." That report includes such observations as "no one is likely to get into trouble for insisting that a research proposal is not exempt" and "no university is likely to want to explain to either the government or the public why its commitment to avoid harming the human subjects of research is limited by the source of funding for the research." In these and other cases, the AAUP recognizes that the IRB system punishes individuals and institutions only for approving research, not for restricting it.

The design flaws in the system have yielded a pattern of abuse. Read Maureen Fitzgerald and Laura Stark, both of whom observed repeated abuses by the IRBs they studied. Read Linda Thornton, whose work was thwarted at 15 of 24 institutions she contacted. Read Jack Katz, who shows that IRBs are particularly likely to pounce on controversial topics. IRBs can function well or poorly, but the system is weighted toward poor function.

Rasmussen acknowledges that poorly designed systems can lead to systematic problems. She concedes that the "lack of an [IRB] appeals process may threaten academic freedom." She also details the way that departmental-level review might systematically hamper research. And she ends her essay with a promising proposal for “template review:"

Disciplines at the national level might formulate templates to guide very common research approaches. For example, a research template for oral historians could stipulate that the researcher will interview individuals, record their answers, refer them to counselors if the questions have provoked strong emotions, procure consent forms, lock the transcripts securely, and identify what will happen to the transcripts at the close of research. IRBs at individual institutions would review the template once and approve it (or even decide to accept any templates from given professional societies). Thus, a researcher would simply submit a form to the IRB stating her agreement to abide by the format of the template. Upon receipt of the form, the IRB would approve the protocol.

If IRBs are not threatening academic freedom, why propose this reform? Inside this proposal is an acknowledgment that disciplinary experts and professional societies in the social sciences and humanities have been excluded from the present IRB system. While such exclusion does not automatically threaten academic freedom, we should not be surprised when it does. For all her skepticism of the AAUP report, Rasmussen has presented her own suggestion that the current system is rotten at the core.

Finnish Group Warns Against Unnecessary Bureaucracy

Klaus Mäkelä and Kerstin Stenius kindly alerted me to their paper, "A New Finnish Proposal for Ethical Review in the Humanities and Social Sciences," which they presented in London in April. The paper describes a draft report by a working group of Finland's National Advisory Board on Research Ethics, which examined the need for ethics review in the humanities and social sciences.

In its draft report, issued in January, the working group adopted some principles that would be familiar to ethics committees and regulators in the United States and other countries. The report stresses the importance of voluntary participation, informed consent, the confidentiality of information, the avoidance of "undue risk and harm," and the need for special care when researching minors. It sees ethics review committees as part of a process to effect these goals.

On the other hand, the working group recognizes that too much oversight presents its own problems:

5. It is important to respect the autonomy and good sense of research subjects. In social research, participants usually are fully competent to assess the risks involved without outside expertise. Ethics committees should avoid paternalism.

8. Clear criteria should be formulated for what kinds of projects require ethical review, but it should be up to the researcher to determine whether a project meets these criteria.

10. The work of ethics committees should be as transparent and open as possible and a system of appeals should be put in place.

The second part of principle number 8 is particularly significant. U.S. regulators have, since 1995, insisted that researchers cannot be trusted to determine when their research is subject to review under the Common Rule. Recently, Jerry Menikoff of OHRP noted that institutions are not legally required to strip researchers of the power to make these determinations, but OHRP will continue to recommend that they do so.

The Finnish working group, by contrast, sees a greater danger in giving that power to committee members and staffers who will likely err on the side of too much review:

It is a matter of judgement to decide what kinds of stimuli are 'exceptionally strong'. To avoid unnecessary bureaucracy, it nevertheless should be up to individual researchers to decide whether their project falls into the categories listed above and needs to be submitted to ethical review. It is highly unlikely that this will lead to transgressions, and ex post facto sanctions will be enough to keep any exceptions under control.

In short, the working group understands that in this case, the dangers of too much bureaucracy outweigh the dangers of too little.

I should note that the London conference at which Mäkelä and Stenius presented their work was the Third Working Meeting of the International Study of Ethical Codes and Ethical Control in the Social Sciences, the previous conferences having been held in London in 2007 and 2008. The meetings have brought together scholars from several northern European countries to discuss social science ethics and regulations across international borders. It is splendid that these scholars are at work on so important a topic, and I look forward to learning more from them.

Menikoff to Critics: "Yes, We Hear You"

Theresa Defino kindly alerted me to the streaming video feed of Dr. Jerry Menikoff’s May 14 address at the University of Michigan, “The Legal Assault on the Common Rule." The speech was an impressive acknowledgment of the widespread criticism of the foundations of the IRB system, and it ended with the promise of some substantive improvement. But by listing some of the most common critiques of the IRB system without attempting to rebut them, Menikoff fell short of the dialogue he seeks to foster.

Menikoff began his speech with a word for IRB critics: "Yes, we hear you." He then spent most of his hour summarizing some of the more prominent critiques of the U.S. IRB system. Among them:

• Carl Schneider, Personal Statement, in President's Council on Bioethics, The Changing Moral Focus of Newborn Screening: An Ethical Analysis by the President's Council on Bioethics, 2008.



• Department of Health and Human Services. Office of Inspector General, Institutional Review Boards: A Time for Reform (1998)



• Philip Hamburger, "The New Censorship: Institutional Review Boards" (2005).
  
  
• University of Illinois, The Illinois White Paper. Improving the System for Protecting Human Subjects:. Counteracting IRB “Mission Creep” (2005).



• Northwestern University Law Review, Symposium on Censorship and Institutional Review Boards (2006)



• American Association of University Professors, Research on Human Subjects: Academic Freedom and the Institutional Review Board (2006)

(As Menikoff noted, challenges to the legality of the IRB regime are only one component of this broad critique, and Menikoff declined to go into detail about them. This makes the title of his speech an odd one.)

For most of the speech, Menikoff was carefully respectful of the authors of these documents, noting their prominent positions and accomplishments in other areas of scholarship. The only real rebuttal he made was to suggest that some critics are too quick to assume that an IRB would demand to review, or even deny, a particular study. That's a good point; it's always better to critique the system with a real horror story about thwarted research, rather than a hypothetical one. What Menikoff didn't say is that there are plenty of real horror stories to go around.

Menikoff offered one important concession. In his 2007 article, "Where’s the Law? Uncovering The Truth About IRBs and Censorship," he had suggested that if an institution required IRB review of research exempted by the Common Rule, researchers at that institution shouldn't blame the feds:

An institution can choose to impose rules that are more restrictive than [federal] regulations. But complaints about censorship resulting from such a circumstance would seem more appropriately directed at the specific institutions that are choosing to do this, rather than the IRB system itself, as created by the federal regulations. (792, n. 6)

Such analysis ignored the role of OPRR/OHRP in encouraging institutions to impose requirements beyond those in the regulations. In particular, in 1995, OPRR advised institutions "that investigators should not have the authority to make an independent determination that research involving human subjects is exempt." More recently, OHRP contributed to a 2008 report that recommended expedited review for several hypothetical projects that would seem to merit exemption.

In his Michigan speech, Menikoff did better. He conceded that OHRP continues to recommend that researchers should not make exemption determinations. But he also noted, "it's just a recommendation. You don't have to follow it." He pledged that OHRP would clarify this.

While this concession is welcome, it is pretty small stuff. Even on the narrow issue of the Common Rule exemptions, it ignores the question of why OHRP continues to promulgate guidance that contradicts the intent of the authors of the exemptions and that originally emerged from the panic of the mid-1990s that almost everyone agrees led to the overregulation of human subjects research. Nor can I put much faith in a promise that new guidance from OHRP is just around the corner.

More importantly, it's frustrating that Menikoff missed the opportunity to address the central problem posed by the critics he cited: there is no evidence that the IRB system does more good than harm. The question of self-exemption is important, but in the context of the overall critique of the system outlined in the speech, Menikoff's emphasis on this one point becomes a bit of a red herring.

Finally, I must object to Menikoff's patronizing remarks about the emotional state of IRB critics:

It's not as if everything or even the bulk of what these people are claiming is necessarily true or valid, but nonetheless we again have to be aware of the strength of their feelings and the source of it. There is this heartfelt feeling about at least parts of the human protection system that they are wrong.

Yes, critics are angry, but it is insulting to portray us as so overcome with emotion that we cannot form "true or valid" complaints. It is not the strength of our feelings that should concern Menikoff, but the accuracy of our facts and the logic of our arguments.

Lisa Wynn's Words and Pictures

In April I commented on the ethics training program for ethnographers developed by Lisa Wynn of Macquarie University with some colleagues.

At Culture Matters, the blog of the Macquarie anthropology department, Wynn described the ideas that led her to develop the program.

Now, at Material World, a blog hosted by the the anthropology departments of University College London and New York University, Wynn describes another aspect of the training program: the pictures.

Wynn explains that along with its medical-centered ethics and jargon-laden text, the standard NIH ethics training program suffers from clip art in which people are depicted as faceless cartoons--probably not the best way to get researchers thinking about others as autonomous individuals. So for her program, Wynn offers pictures of real researchers and research participants, from Laud Humphreys to Afghan school administrators.

Gathering these photos--about a hundred in all--wasn't easy, but they contribute meaningfully to the warmth and depth of the site. And it put Wynn in touch with some prominent scholars.

[Side note: Professor John Stilgoe tells his students that it's rare to have enough photos of yourself at work. That's a good admonition; you never know when someone will want to show you doing controversial research.]

In another posting on Culture Matters, Wynn describes her continuing research on research ethics. She notes that ethics-committee oversight of ethnography is a relatively recent phenomenon. While it was debated as early as the mid-1960s, only in the 1990s did it become widespread. Thus, in studying the effect of ethics committees,

We’ve got a perfect “natural” control: an older generation of researchers who spent most of their careers not seeking ethics clearance, a younger generation for whom it is standard operating procedure, and a “middle-aged” group of researchers like myself who started their research under one regime and now live under another (I swear, this is the first time I’ve thought of myself as middle-aged). By correlating responses with different regulatory regimes, we can ask questions like: do researchers who never got ethics clearance have different ideas about what is ethical than researchers who go through ethics review? Does one group consider itself more or less ethical than the other? Or do they feel like ethics oversight hasn’t made any difference to their research practice?

Wynn plans to contact scholars in Australia and the United States to see how the spread of ethics review affected ideas about research ethics. I'm quite excited by this work; in fact, I plan to publish it in a special issue of the Journal of Policy History I am editing on the general topic of the history of human research ethics regulation.

How many pictures should I demand?

A Horror Anthology

Mark Kleiman takes on IRBs at The Reality-Based Community. On April 14 he asked his readers for IRB horror stories, and on May 2 he posted some of the responses.

The saddest concerns a group of law students who wished "to send testers of different races in different styles of clothing to the restaurant over some period of time to test whether they enforced their dress code in a discriminatory manner." Law school administrators told them they would have to secure IRB approval. This discouraged the students, who did not want to go through the time and effort of the approval process.

This was not the intent of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. At its 15 April 1978 meeting, the commission discussed just such a scenario (pp. II-5 to II-21 of the transcript), and all the members seemed to agree that such testing for discrimination should not require IRB review. But, as I've noted before, the commission wrote a definition of human subjects research that plausibly includes a great deal of activity the commission did not seek to regulate. Thirty years later, justice suffers as a result of the commission's sloppiness.

NOTE: In honor of Professor Kleiman's search, I have gone back through this blog to add the "horror stories" tag to some posts that should have had it to begin with. Clicking on that tag now yields more than a dozen posts, with even more documented horror stories.