Reforms for “21st century science” would have been good for the 20th too

A group of 11 researchers and IRB professionals, most of them affiliated with the University of California, San Diego, report on a brainstorming session from early 2015. They argue that readable consent forms, expert review, a less punitive system, and more exemptions would better serve researchers and participants. While they present their ideas as “a human research protections system that is responsive to 21st century science,” the measures they propose are equally valid for research as it has been practiced for decades.

[Cinnamon Bloss et al., “Reimagining Human Research Protections for 21st Century Science,” Journal of Medical Internet Research 18, no. 12 (2016): e329, doi:10.2196/jmir.6634.]

IRB Still Isn't Peer Review

A bit off-topic here (this is a health-related study), but here's an illustration of the benefits of IRB shopping.

[Cordner, Alissa, and Phil Brown. “Moments of Uncertainty: Ethical Considerations and Emerging Contaminants.” Sociological Forum 28, no. 3 (September 2013): 469–94. doi:10.1111/socf.12034.]

The Systematic Threat to Academic Freedom

Lisa Rasmussen kindly alerted me to her essay, "Problems with Minimal-Risk Research Oversight: A Threat to Academic Freedom?" IRB: Ethics & Human Research 31 (May 2009): 11-16. The essay mostly seeks to rebut the AAUP's 2006 report, "Research on Human Subjects: Academic Freedom and the Institutional Review Board." Rasmussen identifies some important shortcomings in that report, and she raises key questions about the relationship between IRBs and academic freedom. But I am unpersuaded by her central arguments.

Before I address them, I should note the repeated disclaimers within the essay. "I will not settle here the fundamental issue of whether a convincing argument exists that IRB review poses a threat to academic freedom," Rasmussen writes. "A longer explanation of the [AAUP report's] failures is beyond the scope of this paper, but a brief outline is possible." I am disappointed by these limits. Rasmussen devotes significant space to matters peripheral to the question of academic freedom, such as her assertion that researchers whose work was approved by a department--rather than a central IRB--would necessarily merit less legal protection, a claim whose weakness she acknowledges in a footnote. Given only six pages, Rasmussen would have done better to focus on the question posed in her title.

Rasmussen's main argument is that the AAUP report "does not demonstrate that IRBs pose a threat to academic freedom." As she notes, such a demonstration would require a definition of academic freedom, something lacking in the AAUP report. So she offers a passage from the AAUP's "1940 Statement of Principles on Academic Freedom and Tenure": "Institutions of higher education are conducted for the common good and not to further the interest of either the individual teacher or the institution as a whole. The common good depends upon the free search for truth and its free exposition." Emphasizing the grounding of this argument in the search for the "common good," Rasmussen then concludes that "there is a prima facie claim that research can be subjected to assessment regarding whether it threatens to harm the common good via harm to individuals."

I believe this is a misreading of the 1940 Statement, for it suggests that any policy aimed at safeguarding the common good is consistent with academic freedom. For example, she could have written, "there is a prima facie claim that research can be subjected to assessment regarding whether it threatens to harm the common good via the promotion of communist overthrow of the government," and that therefore a prohibition on the use of Marxist analysis is consistent with academic freedom.

A more relevant definition of academic freedom can be drawn from the AAUP's 1915 Declaration of Principles on Academic Freedom and Academic Tenure":

The liberty of the scholar within the university to set forth his conclusions, be they what they may, is conditioned by their being conclusions gained by a scholar’s method and held in a scholar’s spirit; that is to say, they must be the fruits of competent and patient and sincere inquiry, and they should be set forth with dignity, courtesy, and temperateness of language . . .

It is, however . . . inadmissible that the power of determining when departures from the requirements of the scientific spirit and method have occurred, should be vested in bodies not composed of members of the academic profession. Such bodies necessarily lack full competency to judge of those requirements; their intervention can never be exempt from the suspicion that it is dictated by other motives than zeal for the integrity of science; and it is, in any case, unsuitable to the dignity of a great profession that the initial responsibility for the maintenance of its professional standards should not be in the hands of its own members. It follows that university teachers must be prepared to assume this responsibility for themselves.

As Matthew W. Finkin and Robert C. Post write in their new book, For the Common Good: Principles of American Academic Freedom, freedom of research depends on "a framework of accepted professional norms that distinguish research that contributes to knowledge from research that does not." (54) While these two experts on academic freedom decline to offer a firm opinion on the legitimacy of IRBs, they take the AAUP's concerns far more seriously than does Rasmussen (69).

The question, then, is whether IRBs, like the boards of trustees that concerned the authors of the 1915 statement, "lack full competency to judge of [scholarly] requirements." Rasmussen suggests that IRBs merely maintain scholarly standards: "The source of the threat to academic freedom via oversight by one’s colleagues is far from clear," she writes, "especially since researchers undergo peer review for research funding and when submitting their manuscripts for publication." But IRB review is not peer review, since it is conducted mostly by people ignorant of the scholarly methods they are reviewing. (See "Why IRBs Are Not Peer Review," and other posts tagged "peer review.")

To make this a bit more concrete, we can examine the exemplary "horror stories" included in the 2006 AAUP report. Rasmussen rejects these as "unelaborated anecdotes with no documenting citations," rather than examining their implications for academic freedom.

Here's one: "A Caucasian PhD student, seeking to study career expectations in relation to ethnicity, was told by the IRB that African American PhD students could not be interviewed because it might be traumatic for them to be interviewed by the student." Or another: "A campus IRB attempted to deny an MA student her diploma because she did not obtain IRB approval for calling newspaper executives to ask for copies of printed material generally available to the public." No peer review process would impose such conditions. If these are not infringements of academic freedom, then nothing is.

Rasmussen is quite right that we should not equate "inconvenience and hassle with abridgement of academic freedom." Yet nor should we dismiss the abridgement of academic freedom as mere inconvenience and hassle. When IRBs impose conditions on research that prevent researchers from conducting the basic tasks of scholarship--talking to people of varied backgrounds, recording interviews, or telephoning for information--they abridge academic freedom. The more interesting questions are how often this occurs, and why it happens.

Rasmussen presents IRB abuse as a somewhat random process: "IRBs can function well or poorly, and which is true for a given IRB depends on many factors, not least of which are institutional support and member training." This suggests that IRB abuses are individual anomalies, rather than a pattern.

By contrast, the AAUP detects a systematic bias toward the infringement of freedom. This is better developed in the AAUP's 2000 report (cited by Rasmussen), "Institutional Review Boards and Social Science Research." That report includes such observations as "no one is likely to get into trouble for insisting that a research proposal is not exempt" and "no university is likely to want to explain to either the government or the public why its commitment to avoid harming the human subjects of research is limited by the source of funding for the research." In these and other cases, the AAUP recognizes that the IRB system punishes individuals and institutions only for approving research, not for restricting it.

The design flaws in the system have yielded a pattern of abuse. Read Maureen Fitzgerald and Laura Stark, both of whom observed repeated abuses by the IRBs they studied. Read Linda Thornton, whose work was thwarted at 15 of 24 institutions she contacted. Read Jack Katz, who shows that IRBs are particularly likely to pounce on controversial topics. IRBs can function well or poorly, but the system is weighted toward poor function.

Rasmussen acknowledges that poorly designed systems can lead to systematic problems. She concedes that the "lack of an [IRB] appeals process may threaten academic freedom." She also details the way that departmental-level review might systematically hamper research. And she ends her essay with a promising proposal for “template review:"

Disciplines at the national level might formulate templates to guide very common research approaches. For example, a research template for oral historians could stipulate that the researcher will interview individuals, record their answers, refer them to counselors if the questions have provoked strong emotions, procure consent forms, lock the transcripts securely, and identify what will happen to the transcripts at the close of research. IRBs at individual institutions would review the template once and approve it (or even decide to accept any templates from given professional societies). Thus, a researcher would simply submit a form to the IRB stating her agreement to abide by the format of the template. Upon receipt of the form, the IRB would approve the protocol.

If IRBs are not threatening academic freedom, why propose this reform? Inside this proposal is an acknowledgment that disciplinary experts and professional societies in the social sciences and humanities have been excluded from the present IRB system. While such exclusion does not automatically threaten academic freedom, we should not be surprised when it does. For all her skepticism of the AAUP report, Rasmussen has presented her own suggestion that the current system is rotten at the core.

UMKC's Respectful Oral History Policy

The University of Missouri-Kansas City (UMKC) has posted a promising new policy: "Social Sciences IRB and Oral History."

The policy has a number of elements that set it apart from the typical university policy, which seeks to cram oral history into a system designed for medical experimentation. Instead, it adapts only those elements of the medical IRB system that encourage historians to follow their own discipline's ethics and best practices.

I suggest that readers of this blog read the whole policy, but here are some highlights:

1. Respect for Critical Inquiry

As I have written repeatedly on this blog, historians do not take the Hippocratic Oath, and should not promise not to harm the people they interview. Any IRB that imposes the Belmont Report on historians is asking them to forswear their own ethics.

UMKC understands this. Its policy notes that

akin to a journalist or lawyer, an historian is also responsible to a wider public to recover a shared past “as it really happened.” In keeping with the public role of an historian in a democratic society, these responsibilities, especially when conducting narrative interviews, can necessitate a confrontational style of critical inquiry. So while historians do not set out to hurt their interviewees, oral historians are expected to ask tough questions in their interrogation of the past.

2. Respect for Peer Review

The UMKC neither subjects oral historians to the whims of board members unfamiliar with their field, nor does it leave them on their own. Instead, it offers scholars a number of relevant readings, including publications of the Oral History Association, and then encourages them to talk to colleagues knowledgeable about interviewing:

After reviewing these resources on their own, the researcher is strongly encouraged to discuss their research protocol with peers before implementing their research protocol. In some cases, peer review by members of one’s own department would be most useful; in other cases, a researcher might be better served by seeking review from a colleague in a different department.

To foster these kinds of conversations among the faculty, the Social Sciences IRB Subcommittee for Oral History will hold two meetings per semester . . . to discuss “Best Practices” in oral history. Faculty experts in oral history will guide these conversations . . . These meetings are designed to meet the needs of researchers seeking advice and peer review for their research protocols. They are also designed to meet the needs of Chairs and/or designees interested in learning how to advise researchers in their departments to make responsible decisions regarding oral history.

3. Respect for OHRP's Pledge

UMKC takes seriously the carefully negotiated 2003 agreement between the American Historical Association and the Oral History Association and OHRP, even posting a copy on its website. The university elaborates on that agreement:

At UMKC, we draw a distinction between idiographic research that uses oral histories to describe the unique story of some particular social group or individual, which does not constitute “human subjects research”; and nomothetic research that employs oral histories in the hopes of contributing to a general theoretical or comparative debate about the human nature or behavior, which does fall under the category of “human subjects research."

While I confess that the terms idiographic and nomothetic are not in my working vocabulary, I believe they do express a real difference between the ethics of oral historians and those of other scholars. If one is interested in a general theoretical or comparative debate about the human nature or behavior--as many social scientists seem to be--then it makes less sense to single out individuals for potential honor or calumny. Writing about unique individuals or groups changes one's responsibility toward the individuals interviewed.

4. Respect for Researchers

Policies like UCLA's infantilize researchers, making them submit every judgment to an administrator. By contrast, UMKC trusts its scholars:

The bottom line is that the researcher makes these determinations in careful consultation with the Chair of the department or another official designee appropriate to the kind of study being planned. Together this determination is based on shared understanding of all relevant guidelines and their shared expertise in their specialized field of scholarship.

5. Respect for the IRB

Even as it empowers historians, the UMKC policy keeps the IRB involved, making it a resource, rather than an obstacle. Researchers still have to learn something about human subjects regulations, and they must complete a form explaining why they have determined that their policy does not fall under federal regulations.

(The form's demand for an explanation of "no more than 1500 characters" sounds suspiciously bureaucratic, but it's a good length for the presentation of a single idea--about the same as the 150-word limit for a New York Times letter to the editor.)

More importantly, the frequent meetings of the Social Sciences IRB Subcommittee on Oral History suggest that some scholars at UMKC have devoted their time to helping colleagues deal with the real ethical challenges of oral history.

The website explaining the policy notes that it was developed by "a group of faculty and administrators involved with the Social Science Institutional Review Board (SSIRB) . . . with input from members of the SSIRB, the College of Arts & Sciences, and the Faculty Senate at UMKC." I congratulate all the scholars and administrators who developed this innovative system, and I hope it works as well in practice as it reads on the screen.

With this policy, UMKC joins Amherst College, Columbia University, the University of Michigan-Ann Arbor, and the University of Nebraska-Lincoln Policy on a small but growing list of schools that have adopted OHRP's 2003 position removing most oral history research from IRB jurisdiction. Five schools not very many, but it's five more than the AHA could find in February 2006. Who will be number six?

Reform or Revolution? Comment on Bosk, "The New Bureaucracies of Virtue"

Following the introduction, the first substantive piece in the PoLAR symposium is Charles L. Bosk, "The New Bureaucracies of Virtue or When Form Fails to Follow Function."

In the past, Bosk has advocated living with IRB review. As he wrote in 2004

Prospective review strikes me as generally one more inane bureaucratic requirement in one more bureaucratic set of procedures, ill-suited to accomplish the goals that it is intended to serve. Prospective review, flawed a process as it is, does not strike me as one social scientists should resist. After all, we agree with its general goals: that our informants should not be subject to untold risks, that they be treated with decency, that their confidentiality and anonymity be safeguarded, when feasible. Given this, we should not waste our energies resisting a process that has an enormous amount of both bureaucratic momentum and social consensus behind it. Instead, we should focus our energies on reforming and revising procedures; we should fix the system where it is broken.

[Charles Bosk, "The Ethnographer and the IRB: Comment on Kevin D. Haggerty,'"Ethics Creep: Governing Social Science Research in the Name of Ethics'," Qualitative Sociology 27 (December 2004), 417.]

But at some point in 2005 or 2006, an IRB seems to have really pissed him off. In this essay, he writes that "having now been on the receiving end of IRB objections that I find incomprehensible, I appreciate my colleagues' multiple frustrations." (204) He now seems to think the system is not merely broken, but so defectively designed that it cannot be repaired:

The presumption of prospective review—that our subjects are in need of protection— has embedded within it an insulting distrust of our integrity and motives. The insult inherent in a regulatory regime based on distrust deepens when the barriers the review system places between us and the doing of our research appear to protect powerful institutions from close scrutiny more than they guarantee the well-being of our research subjects. For me, the most serious defect of the current regulatory system is that the requirements of policy reduce and trivialize the domain of research ethics. In the process, our ability to conceptualize, discuss, and make sense of the ethical problems of ethnographic work is dulled. As we do our work, we face ethical dilemmas aplenty, almost none of which have to do with the dual mandate of prospective research review—the adequacy of the consent process, which is invariably reduced to concern about a "formal document" or potential risks to subjects. (194)

Bosk's essay is rich in ideas--too many, really, for an essay of this length. I will do my best to unpack them.

IRBs "Jerk Around" Education Research

Debra Viadero reports on the recent conference of the American Educational Research Association: "Security Checks of U.S. Education Contractors to Change," Education Week, 2 April 2008.

She includes the following description of one session:

Under federal human-protections laws, studies that involve human subjects—a category that includes most research in education—first have to be approved by institutional review boards, or IRBs, based at researchers’ home universities or research organizations.

But that process can sometimes be fraught with frustration and distrust . . . .

“While I was behind the curtain, it seemed to me that our board was quite reasonable,” said Frederick D. Erickson, a professor of anthropology in education at the University of California, Los Angeles. He has served on three institutional review boards over the course of his career. “Now, I’ve got a project of my own in expedited institutional review,” he added, “and it’s being jerked around in ways that make my blood boil.”

Some problems with the process, said Melissa S. Anderson, an associate professor of higher education at the University of Minnesota- Twin Cities who has studied IRBs, is that researchers often disagree with the boards’ judgments or may be skeptical of their authority.

“It’s, ‘What right do they have to tell me whether or not I can do research?’ ” she said. “The issue of peer review when peers aren’t seen as peers is … a sticking point.”

“This can lead to IRB shopping,” Ms. Anderson added, which is what occurs when researchers working on a study involving multiple universities try to figure out which one’s board is most likely to approve their project. “That’s becoming increasingly common and problematic.”

In a national survey of scientists that Ms. Anderson and her colleagues conducted last year, 5 percent of respondents admitted to having ignored or circumvented human-research requirements sometime in the previous three years. When medical researchers were removed from the sample, that percentage rose to 8 percent.

Maureen Fitzgerald's Ethics Project

Anthropologist and George Mason University alumna Maureen Fitzgerald, now affiliated with the University of Sydney, is the director of "An Analysis of Research Ethics and the Ethical Review Process as Culture and Cultural Process," an ongoing investigation more succinctly named "The Ethics Project."

Between 2001 and 2005 she and her associates observed the workings of 29 ethics committees in Australia, Canada, New Zealand, the United Kingdom, and the United States--not finding significant differences across national boundaries. They have published several papers based on this work, most of which are available on the project's website. Since the themes of many of these publications overlap, I will focus my comments on three articles that I found particularly helpful.

AAHRPP Calls for Research on IRBs and Behavioral and Social Science

Rob Townsend kindly alerted me to the Winter 2008 issue of AAHRPP Advance, published by the Association for the Accreditation of Human Research Protection Programs, which features the lead article, "IRBs and Behavioral and Social Science Research: Finding the Middle Ground." The article concedes that "many behavioral and social scientists feel constrained by a system that seems tilted toward biomedical research and, therefore, neither understands nor reflects their concerns." And it reports the interest of Drs. J. Michael Oakes, "an author and frequent lecturer on IRB review of behavioral and social science research," and Howard Silver, Executive Director of the Consortium of Social Science Associations, in addressing some of those concerns.

Oakes and Silver "encourage researchers to investigate scientifically IRB oversight of behavioral and social science research. Such research could determine whether IRBs are consistent in applying the federal regulations, whether IRBs are taking advantage of the flexibility that’s built into the regulations, and whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research. The resulting data could shed light on ways to relieve tensions between these two groups."

That sounds good, but it's a bit disappointing that the article does not acknowledge the considerable research already completed on this topic, much of it already cited on this blog. Nor does it remark on the curiosity that IRB oversight has continued for four decades without anyone knowing if it does any good.

Moreover, the article states as fact some beliefs that should be investigated with just the sort of research it calls for. I hope Oakes, Silver, and AAHRPP will allow data to challenge some of their own presuppositions.

Here are some questions that could be answered by further research.

1. When did IRB review of social science go bad, and why?

AAHRPP thinks it already knows the answer to this one. The article claims that "Tensions began building in the late 1990s in response to increased government scrutiny of research involving human participants," and that "the regulations have not changed. What’s new is their enforcement and, in many instances, that enforcement is overdue." I am at work on a history of IRB review of social science and humanities research, and I think more research can challenge this view.

The first part of the AAHRPP claim is doubtful; social scientists have protested IRB regulations since 1966, and tensions have waxed and waned since then. If we take this longer view, then the assertion about the immutability of the regulations is wrong; the regulations--first promulgated in 1974--changed twice, in 1981 and 1991. And the 1991 revisions greatly expanded the reach of IRBs. The 1981 regulations exempted survey, interview, and observational research unless it “deals with sensitive aspects of the subject’s own behavior, such as illegal conduct, drug use, sexual behavior, or survey or interview procedures is use of alcohol" and if "the subject’s responses, if they became known outside the research, could reasonably place the subject at risk of criminal or civil liability or be damaging to the subject’s financial standing or employability. The 1991 regulations, in contrast, eliminated the "sensitive aspects" clause and added potential harms to reputation to the list of triggers for IRB review. These changes were made over the objection of social scientists. And they set the stage for the conflict of the 1990s and today.

The claim that "what's new is [the regulations'] enforcement," is only half-true. Also new is the guidance issued by OPRR/OHRP since 1995 that reversed previous policies.

Finally, the article claims that "enforcement is overdue." Really? What errors did social scientists commit in the 1980s--a decade of relatively light regulation?

2. Why do IRBs sometimes delay or prohibit social science research?

Dr. Oakes states, “IRB members are not those folks who are looking to thwart your study. They are peer researchers who have a job to do.” But clearly some IRB members are looking to thwart studies, or else studies wouldn't get thwarted as often as they do. The question is how many IRB members do this, and why.

One part of this question concerns membership. Oakes's claim that IRB members "are peer researchers" depends on an odd definition of peers. In the NIH peer review process, for example, proposals are reviewed by study sections whose members are chosen for their expertise. The NIH's Center for Scientific Review requires, among other things, that

"* Expertise is the paramount consideration when developing/updating a study section roster.

"* Each scientific area reviewed by the study section needs appropriate expert representation."

IRBs theoretically must include experts on each type of research reviewed, but Oakes knows as well as I that this requirement is often ignored. Additional research might indicate how often a researcher faces an IRB with no expertise in the methods under review.

Then, of course, some IRB members are not researchers at all, but the "one member whose primary concerns are in nonscientific areas" required by the regulations. As Laura Stark's dissertation suggests, these members can be particularly undisciplined in their meddling.

3. What types of research now fall subject to IRB review?

Like PRIM&R, AAHRPP thinks that IRBs review only two kinds of scholarship: biomedical research, and something called "behavioral and social science research." The article states, "AAHRPP’s Founding Members, Board of Directors, Council on Accreditation, and Supporting Members all include representatives of organizations engaged in behavioral and social science research."

This statement suggests the fallacy of the undistributed middle term:

* Ehnographers are represented by organizations engaged in behavioral and social science research.
* Organizations engaged in behavioral and social science research have a voice in AAHRPP.
* Therefore, ethnographers are represented by organizations that have a voice in AAHRPP.

The second premise is "undistributed," since it is not true that all organizations engaged in behavioral and social science research have a voice in AAHRPP.

Here's a counter example:

* Countries in South America, Africa, and South Asia are not in North America or Europe.
* Countries from parts of the world other than North America and Europe are permanent members of the UN security council and the G-8.
* Therefore, countries in South America, Africa, and South Asia are permanent members of the UN Security Council and the G-8.

In committing this fallacy, AAHRPP lumps together a dozen or more scholarly disciplines--each with its own history, methods, and ethics--into a single category: "behavioral and social science research." The AAHRPP website does not list the disciplinary affiliations of members of its Board of Directors, Council on Accreditation, or list of site visitors, but if there's a journalist, historian, or folklorist in the lot, I'll be surprised.

To take the example I know best, oral historians do not expect psychologists, social workers, or education researchers to understand or represent their interests. AAHRPP (like PRIM&R) should find out how many disciplines are now subject to review, and include representatives from all of them.

4. What models of ethical review exist, and what models might we imagine?

The article asks "whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research." But that is only one of several alternative systems in place on various campuses. For example, Macquarie University delegates ethical review to a number of subcommittees with special expertise in certain fields. And the University of Pennsylvania allows researchers using some social science methods to forego "a fixed research protocol." And we can imagine even more models, some of which would require redrafting present regulations, others of which might not.

I appreciate AAHRPP's call for research, and I hope it agrees that research is most valuable when the answers are not predetermined.

Why IRBs Are Not Peer Review: A Reply to E. Taylor Atkins

In his recent article on “Oral History and IRBs,” E. Taylor Atkins applauds the argument of J. Michael Oakes “that IRBs should be viewed as—and should behave as—peer review systems rather than as unaccountable ‘leviathans’ exercising authority from on high.” But Oakes and Atkins overlook profound differences between IRB review and peer review. The two systems are not merely different; they are antithetical.
The most obvious difference between peer review and IRB review is that peer review means review by peers—fellow scholars in one’s field. This was also the system envisioned in 1966 when the Public Health Service first mandated “prior review by [a researcher’s] institutional associates” before that researcher could received PHS funds, and to some degree it is the way IRB review works in biomedical research, where teams of physicians and other biomedical researchers review protocols by others in their field. But fundamental to the complaints of oral historians and others in the humanities and social sciences is the fact that IRBs are never (to my knowledge) composed mostly of scholars who conduct qualitative interviews. Indeed, most IRBs probably lack a single such researcher.
Even when IRBs do include such scholars, they operate in a web of laws, regulations, and guidelines written by biomedical researchers and bioethicists without input from historians. Consider the involvement of historians in shaping the present system:

• Number of historians invited to testify before Congress as it was shaping the National Research Act: zero
• Number of historians on the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, creator of the Belmont Report: zero
• Number of historians among the advisors and consultants to the Institutional Review Board Guidebook : zero
• Number of historians on the board of Public Responsibility in Medicine and Research (PRIM&R): zero
• Number of historians among the authors of the CITI Program: zero
• Number of historians on the Secretary's Advisory Committee on Human Research Protections : zero

Although historians have had no chance to shape the framework within which IRBs operate, by law and by their responsibility to their institutions, IRBs cannot ignore this framework. So it is not peer review at all that they impose, but values foreign to the historical profession.
One result of this peerless system is that IRBs are rarely able to offer the kind of expert advice we seek in good peer review. I have reviewed many manuscripts myself, but only when editors sought me out among a national or international pool of scholars. As a result, every manuscript I have read concerned some aspect of urban planning, infrastructure, or transportation—areas to which I have devoted years of research—or was designed as a textbook for use in the kinds of courses I regularly teach. Yet, as I note in my posting, “In Search of Expertise,” IRBs rely not on such a broad pool, but on whomever happens to be on campus and willing to serve on the local IRB.
Atkins also ignores the power of researchers to shape the peer review of their work. Certainly a university researcher who never submitted anything for peer review would be unlikely to win promotion, but Atkins forgets the choice researchers enjoy in deciding which peer-reviewed journal or press will receive their manuscripts, and which will be alternatives should the first choice reject the proposal. Comparable choosiness in ethical review is derided as “IRB shopping,” and university IRBs retain their monopolies (See Jeffrey Brainard, “Federal Agency Decides Not to Regulate 'IRB Shopping,'” Chronicle of Higher Education, 18 January 2006). Moreover, scholars and editors are free to ignore the advice of peer reviewers in preparing a final publication; how fondly I recall my first editor’s telling me not to worry about the difficult theory one reviewer wanted me to incorporate! Most importantly, scholars are always free to publish in non-peer-reviewed forums, such as this blog. In contrast, IRBs insist on the power to decide when a project requires their review.
Atkins recently attended a PRIM&R conference and “was astounded by the persistent ignorance among IRB administrators and board members in attendance about the special needs of SBER and oral history.” He responded by blaming the victim: “If we refuse to teach IRBs, how can they learn what we need them to know?” What he refuses to acknowledge is that IRBs’ power frees them of any incentive to learn. A publisher whose peer-reviewers offered consistently bad advice would soon lack for material. An IRB can mistreat researchers yet suffer no loss of power; its decisions are final.
While peer review is a persuasive effort by volunteers, IRB review is a coercive practice by agents of the state. Leviathans or not, IRBs do little but exercise authority from on high, empowered by federal regulations stating that “an IRB shall review and have authority to approve, require modifications in (to secure approval), or disapprove all research activities covered by [federal] policy,” and that no university official can overturn such decisions. Institutions that disobey this mandate risk losing millions of dollars in funds; individual researchers risk denial of degrees, promotion, or even the right to continue any research.
Take away this coercive power, and perhaps historians will learn to respect IRBs. Until then, expect them to resist.

In Search of Expertise

When, in the 1960s, the federal government began requiring ethical review of proposals seeking funding for medical experiments, regulators expected that the review would be conducted by experts in the field. Thus, the U.S. Public Health Service's PPO #129, "Clinical Investigations Using Human Subjects" (8 February 1966), required "prior review of the judgment of the principal investigator or program director by a committee of his institutional associates," presumably other doctors or biomedical researchers. The 1974 version of 45 CFR 46 [PDF] required that each IRB be composed of "not less than five persons with varying backgrounds to assure complete and adequate review of activities commonly conducted by the organization. The committee must be sufficiently qualified through the maturity, experience, and expertise of its members and diversity of its membership to insure respect for its advice and counsel for safeguarding the rights and welfare of human subjects."

In response to concerns that researchers could not be trusted to watch other researchers, in 1981 the Department of Health and Human Services added the requirement that "each IRB shall include at least one member whose primary concerns are in nonscientific areas; for example: lawyers, ethicists, members of the clergy." [PDF] The language of the provision shows that the department did not imagine applying these rules to nonscientific researchers (such as lawyers, ethicists, or members of the clergy), but that's not the point here. The point is that the regulations' drafters expected IRBs to be dominated by experts "possessing the professional competence necessary to review specific research activities," with one or two lay members added to represent the conscience of the community.

Recently, some have called for a greater role for non-expert members. In 2001, for example, the National Bioethics Advisory Commission recommended that "members who represent the perspectives of participants, members who are unaffiliated with the institution, and members whose primary concerns are in nonscientific areas . . . should collectively represent at least 25 percent of the Institutional Review Board membership," rather than the single member now required.

But a bigger problem may be not a lack of lay representation, but a lack of expertise.