Advice on Scholarly Blogging

Alice Dreger is soliciting advice “for a young scholar thinking of starting a blog.” My wife and I have each maintained blogs for several years, and here’s what we’ve come up with.

What Can One University Do?

A few weeks ago, a correspondent asked me what reforms individual universities can implement while awaiting systemic, regulatory reform. It's an excellent question, so here's a roundup from material previously covered on the blog.

No university has adopted all of these measures, and at least one of these measures has not been adopted by any. But most of them are in place already, and there's no reason they can't spread.

Health Researchers Learn that Ethics Review Really Is Creepy

Three Canadian scholars with backgrounds in various approaches to health research and bioethics find that, contrary to their expectations, many REB members and staffers believe that "ethics creep" is real. The authors have collected interesting thoughts from REB staff and members, but the authors' analysis and their own proposals for reform are unclear.

[Adrian Guta, Stephanie A. Nixon, and Mike G. Wilson, "Resisting the Seduction of 'Ethics Creep': Using Foucault to Surface Complexity and Contradiction in Research Ethics Review," Social Science & Medicine available online 25 September 2012, DOI: 10.1016/j.socscimed.2012.09.019.]

IRBs Impeded Harvard Dissertation on Addiction and Incarceration

Kimberly Sue, a medical anthropologist in Harvard's MD/PhD program, reports that IRB review can seem "a hassle, a nuisance or a stumbling block, as we seek to enact a more relevant and engaged era of anthropology."

[Kimberly Sue "Are IRBs a Stumbling Block for an Engaged Anthropology?" Somatosphere, 9 August 2012, http://somatosphere.net/2012/08/are-irbs-a-stumbling-block-for-an-engaged-anthropology.html . h/t Michelle Meyer]

Emmerich Reviews Behind Closed Doors and Ethical Imperialism

Nathan Emmerich of Queen's University, Belfast, finds that Laura Stark's book and my own "together . . . illustrate the nature of ethics as an aspect of research governance fundamentally contributing to our understanding of the phenomena in a manner that goes beyond the relatively limited or restricted consideration offered by applied ethical analysis."

[Nathan Emmerich, Review of Behind Closed Doors: IRBs and the Making of Ethical Research and Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965-2009, Sociological Research Online 17, Issue 2 (May 2012).]

IRBs Misunderstand Researchers' Complaints

A study of IRB chairs, members, and staff finds that they understand that researchers regard them as powerful actors, "Yet IRBs may misperceive PI complaints."

[Robert Klitzman, "The Ethics Police?: IRBs’ Views Concerning Their Power," PLoS ONE 6 (2011): e28773. doi:10.1371/journal.pone.0028773. h/t Roberto Veloso]

SBS White Paper Calls for Drastic Reform

Weighing in at 72 pages, the "Social and Behavioral Science White Paper" is the most detailed response to the ANPRM from scholars in the social sciences. The paper presents a grim picture of the state of IRB review and is generally supportive of the ANPRM's goal of reform. But it offers detailed, helpful warnings about the potential effects of the proposed "excused" category and the adoption of HIPAA as a model for confidentiality requirements. Though it shies away from the toughest questions about the IRB system, it is a good expression of the frustrations felt by so many researchers in the social sciences and humanities.

[American Educational Research Association et al., "Social and Behavioral Science White Paper on Advanced Notice for Proposed Rulemaking (ANPRM), Federal Register 44512-531 (July 26, 2011); ID Docket HHS-OPHS-2011-0005," 26 October 2011.]

Would ANPRM's Audits Require Massive Record-Keeping?

Part of the ANPRM's proposal for a new category of Excused research is an audit regime for projects that researchers declare eligible for Excusal. An observer asks whether this requirement would impose inappropriate burdens on researchers. I am hopeful that it would not, but commenters on the ANPRM should be aware of this possibility as they answer question 21.

ANPRM's Problem Statement: Helpful but Incomplete

One of the many remarkable sections of the July 26 advance notice of proposed rulemaking (ANPRM) is its admission that the Common Rule is flawed.

(Note: I have added a link to the ANPRM at the top of the link list in the sidebar.)

Since the 1970s, IRB apologists have claimed that federal regulations are flexible enough, and that local IRBs are to blame for any problems. In 2007, for example, Jerry Menikoff quoted with approval Jeffrey Cohen's 2006 claim that "the regulations provide sufficient flexibility for the efficient and appropriate review of minimal risk research. IRB review of such research does not have to be burdensome or unreasonable if IRBs appropriately utilize the flexibility in the regulations." Menikoff reiterated his claim of "flexibility within the system" in his 2009 speech, “The Legal Assault on the Common Rule."

After thirty years of such claims, it is wonderfully refreshing that the ANPRM takes so seriously many of the critiques leveled at the federal regulations themselves. And the ANPRM helpfully organizes those critiques into seven general categories.

On the other hand, ANPRM's problem statement (pages 44513-44514 in the Federal Register version) overlooks some major critiques. Fortunately, some of those critiques are implicitly recognized by some of the ANPRM's proposals.

Law Professor Sees Broad Role for IRBs in Virtual Worlds

Professor Joshua A. T. Fairfield of the Washington & Lee University School of Law explores the law of research in virtual worlds.

[Joshua Fairfield, "Avatar Experimentation: Human Subjects Research in Virtual Worlds." (November 29, 2010). U.C. Irvine Law Review, Symposium Issue, 2011; Washington & Lee Legal Studies Paper No. 2010-14. Available at SSRN: http://ssrn.com/abstract=1717057.]

Fairfield makes some important points about how researchers interested in these environments might apply ethical guidelines developed for older forms of research. But the article pays insufficient attention to the limits of ethical guidance developed to govern medical experimentation, misstates some of the provisions of current regulations, and downplays the troubles Internet researchers have faced with IRBs.

Texas's All-Star IRB Report

In February 2008, the University of Texas System formed an IRB Task Force to examine ways to improve IRB operations throughout the UT System. In April 2009, that task force issued its report: "IRB TASK FORCE REPORT: Trust, Integrity, and Responsibility in the Conduct of Human Subjects Research."

I recently obtained a copy of that report. While it has not previously been posted on the web, a University of Texas official assured me that a final policy report like this is public information under Texas law. So to make this important public document freely available, I have posted a copy on my website. (See link above.)

The report offers an exceptionally thorough and thoughtful consideration of how IRBs should work at great research universities. While some of its recommendations may be inapplicable to univerisities that are not part of larger systems or do not operate a medical campus, many of the task force's procedures and recommendations offer a model for others. I salute all those who were involved in the report's preparation.

In particular, I commend the following elements of the UT task force and its report.

Procedures

The UT system carefully set up the task force in a way that would build respect for its findings. In particular, it managed to:

1. Represent Multiple Disciplines

UT included task force members "representing a variety of academic disciplines," among them SACRHP member Lisa Leiden. (3, 24) It also sought help from a range of consultants, including such "national experts" Tina Gunsalus; John Heldens, Moira Keene, Dan Nelson, Ivor Pritchard, and Marjorie Speers. (8)

2. Include Stakeholders

At each UT campus, the task force solicited comments from university officials, IRB chairs, investigators, and other interested parties. (8)

3. Allow Adequate Time

The UT task force completed its deliberations over the course of a year, from February 2008 to February 2009, then released its report in April 2009. A report with so broad a scope cannot be rushed.

Recommendations

Careful investigation led to thoughtful recommendations. The report suggests that the UT system:

1. Employ Faculty Expertise

The UT report recognizes that researchers are often expert in a particular area of human subjects research, and it recommends that faculty experts be identified to prepare standards for specific types of research (e.g., research involving subjects with impaired capacity, internet research) and to consult on individual projects as needed. (10)

2. Utilize Flexibility and Empirical Evidence

The report recommends that the university encourage IRB staff to provide "an efficient level of regulatory review compatible with adequate protection of human subjects," rather than the most stringent level of review. In particular, it suggests that IRB staff and members employ empirical evidence when determining risk, by consulting experts or scholarly literature. (11) The report also suggests that IRBs rely strongly on experts when determining the scientific soundness of a proposal. (14)

3. Uncheck the Box

The UT report notes that, according to Speers, fewer than 50 percent of AAHRP accredited institutions check all the box on their federal-wide assurances, and that it is "primarily the major research universities that are considering unchecking the box," thus maximizing their flexibility in handling projects not directly funded by a Common Rule agency. (12) (More on this in an upcoming post.)

4. Diversify the IRB

The UT report recommends that "Institutional Officials should ensure that the institution's disciplines are well represented in the IRB." (18)

5. Provide IRB Oversight

The UT report recommends that institutions "consider the implementation of a research ombudsperson to increase the opportunity for rapid resolution of issues involving human subjects research." (21) It also suggests an ongoing "IRB Advisory Group" to implement recommendations and assist with human subject policy issues. (23)

6. Define Key Terms

The UT Report includes definitions of key terms. Among other things, these definitions make clear that information-gathering interviews, service surveys, and classroom activities may not meet the definition of human subjects research, and that biography and oral history interviews do not meet that definition. (27) As I mentioned earlier, this recommendation has already led to the deregulation of oral history at the University of Texas at Austin.

7. Explore Alternatives

The report include an appendix on "alternative IRB models," based on the November 2006 "National Conference on Alternative IRB Models." While it is promising that the task force is willing to consider such alternatives, I would have liked it to pick up on that conference's call for "further exploration" of models for social and behavioral research. The University of Pennsylvania policy on research in the sociobehavioral sciences might be such a model.

Goal

Most importantly, the UT task force understood the IRB problem as something larger than an administrative challenge. While it explored ways to increase IRB "effectiveness, efficiency, and productivity," it went beyond such managerial concerns to probe big questions about "IRB authority, mission, and functions," the "unnecessary obstruction of research and lapses of effective human subject protection," and best practices from the available literature. (8)

The task force hoped to foster a "culture of conscience" rather than a "culture of compliance," and it understood that conscience cannot be dictated from above. (20) If other universities also seek to promote a culture of conscience, they must give a voice to all those involved with human subjects research.

AAHRPP Urges Extra Care When Talking to Pregnant Women

As I mentioned in October, Marjorie Speers, president of the Association for the Accreditation of Human Research Protection Programs (AAHRPP), has stated that her organization permits institutions it accredits to submit Federalwide Assurances (FWAs) that do not pledge to impose federal regulations on all research not directly funded by the federal government. In otherwords, institutions may "uncheck the box" without losing accreditation.

At the same time, Speers insisted that "if the boxes are unchecked, we hold the organization to have equivalent protections in place for all research." In my earlier post, I asked what an equivalent exemption might look like.

AAHRPP takes a stab at answering this in the Winter 2010 issue of AAHRPP Advance.

"To Check or Uncheck the FWA Boxes" (p. 6) notes that "The regulations that come into play when the FWA boxes are checked were written primarily with clinical research in mind. Thus, if an organization checks Subpart B of the FWA, for example, it will not be allowed to conduct SBER studies that involve pregnant women, because any study involving pregnant women is required to advance biomedical knowledge."

To give institutions more flexibility, the article claims,

AAHRPP has designed its standards to apply as much to protecting participants in SBER as they apply to protecting those in biomedical research. When organizations have unchecked the boxes, AAHRPP allows them to provide protections that are appropriate to the level and nature of the risk involved in the study and meaningful to the type of research. Under the accreditation standards, an organization could require that the results of a SBER study that includes pregnant women, for example, must contribute to general knowledge or knowledge that is beneficial to society, rather than to biomedical knowledge.

While any concession is welcome, this one is unimpressive. The regulation in question, 45 CFR 46.204 states that "pregnant women or fetuses may be involved in research" only if the research holds out the prospect of direct benefit to the pregnant women or fetus or if "risk to the fetus is not greater than minimal and the purpose of the research is the development of important biomedical knowledge that cannot be obtained by any other means."

Changing "biomedical" to "general" here yields a requirement that researchers meet a higher standard if they want to interview, survey, or observe pregnant women than if they want to interview, survey, or observe non-pregnant women, prospective fathers, or other autonomous adults. Such a requirement violates the rights of both researchers and women.

While I appreciate AAHRPP's understanding that much of the Common Rule is inappropriate when applied to non-biomedical research, the example it has chosen suggests that as an organization, AAHRPP still lacks a basic grasp of some of the issues surrounding IRB review of social science research. What is needed is not mere tinkering with the Common Rule, but wholesale reconsideration of the rights and responsibilities of social scientists.

I thank Rob Townsend for bringing this to my attention.

AAHRPP and the Unchecked Box

Regular readers of this blog likely know that most United States universities submit "federalwide assurances" (FWAs) pledging to abide by the Common Rule for research funded directly by federal agencies that have adopted that rule.

Section 4 of the standard assurance includes an optional pledge that "This Institution elects to apply . . . to all of its human subjects research regardless of the source of support, except for research that is covered by a separate assurance" either the Common Rule or the Common Rule and subparts B, C, and D of 45 CFR 46. Institutions that check this box--as seems to have been common in the past--with one stroke of the pen eliminate one of the major concessions made by federal regulators in 1981, when they promised that non-federally-funded research would not be regulated.

Recently, however, at least 164 universities have "unchecked the box," declining to promise to apply the regulations to all research. The American Association of University Professors has strongly recommended that universities uncheck the box as a first step toward devising procedures less burdensome than those specified in the regulations.

Malcolm Feeley has noted that unchecking the box could also yield important empirical data:

If there are few reports of negative consequences . . . they might encourage national officials to rethink the need for such an expansive regulatory system . . . On the other hand, if opt-out results in increased problems, the findings might help convince Katz, Dingwall, me, and still others of the value of IRBs.

Nor are such comments confined to outsiders. At the July 16, 2008, meeting of the Secretary's Advisory Committee on Human Research Protections, committee member Lisa Leiden of the University of Texas system spoke of her own interest in freeing nonfunded research from direct federal regulation:

We have talked about limiting the federal wide assurances, unchecking the box, and I believe the position that we're going to be taking is to advocate in a gentle way thinking about doing that. We have heard both sides of the story or maybe just a few sides, but we think that there are certainly some advantages. And one of the advantages might be . . . what can we do with the expedited review level. It seems that there is a lot of flexibility in that, and we might be able to increase some of that by unchecking the boxes and adding different categories for that.

Unchecking the box is therefore one of the most promising incremental reforms now on the table. This is why I was disappointed to see that the AAHRPP's proposed revised standards, described in my previous post, seem to preclude this option.

A correspondent questioned this assertion, noting that AAHRPP president Majorie Speers had mentioned unchecking the box in her presentation, "Finding Flexibility in the Regulations." But there's nothing in the slides to suggest that AAHRPP or Speers approves of such a practice, and a July 2008 memo from the University of California states that AAHRPP site visitors have told university administrators "that in order for a human research protection program to be accredited, it must apply the Common Rule and its subparts to all human research at the institution, irrespective of funding."

Either AAHRPP forbids accredited organizations from unchecking the box, or its policies are so unclear that its site visitors are giving out bad information. Either way, I suggest that the revised standards permit unchecking the box as a means of reform.

Deregulation "Is Not Going to Happen"

Linda Shopes kindly alerts me to the April 20 issue of COSSA Washington Update, the newsletter of the Consortium of Social Science Organizations, which reports on an April 1 meeting of the National Academies’ Board on Behavioral, Cognitive, and Sensory Sciences, at which IRBs were discussed.

Here's the key passage:

Philip Rubin, CEO of Haskins Laboratories in New Haven, CT, and former director of the National Science Foundation’s (NSF) Division of Behavioral and Cognitive Sciences, chairs the Board. He began the session with a review highlighting the difficulties social/behavioral researchers have had with the current system under the Common Rule regulation and its interpretation by campus Institutional Review Boards (IRBs). Complaints have been loud, but mostly anecdotal . . . Once again the bottom line is that despite efforts by Joan Sieber and the Journal of Empirical Research on Human Ethics, which she edits, there are still large gaps in our empirical knowledge of how the system works for social and behavioral scientists.

Rubin was followed by Jerry Menikoff, new head of the U.S. government’s Office of Human Research Protections (OHRP). Menikoff announced that he was all for “flexibility” in the system and that “changes can be made.” He also endorsed conducting more research. He rejected the arguments of the American Association of University Professors and Philip Hamburger of Northwestern University Law School that IRBs violate researchers’ first amendment rights. He acknowledged the importance of expedited review, but stated quite clearly that “removing minimal risk research from the system is not going to happen.”

I don't want to make too much of these comments; an OHRP spokesperson tells me that they were an extemporaneous response to Rubin, and not prepared remarks. Still, I am disappointed. Menikoff's comments suggest a retreat from his earlier concession that "flexibility" often can be code for arbitrary power. And it's a pity for a public official to insist that a given policy "is not going to happen" even as he endorses more research. Wise governance depends on making policies after finding facts, not before.

Less Flexibility, More Freedom

Defenders of the present IRB system often boast of the "flexibility" offered by current regulations. (See, for example, Dr. Jeffrey Cohen's report from the November PRIM&R meeting.)

But flexibility--when combined with the possibility of punishment--can actually empower censorship. Here is how Human Rights Watch describes an analogous system, China's censorship of the Internet:

The display of politically objectionable content can result in reprimands to company management and employees from the MII, the State Council Information Office, the Communist Party's Propaganda Department, and/or various state security organs, accompanied by warnings that insufficient controls will result in revocation of the company's license. In order to minimize reprimands and keep their licenses in good standing, BBS and blog hosting services maintain lists of words and phrases that either cannot be posted or which cause monitoring software to "flag" the content for manual removal by employees.

Search engines likewise maintain lists of thousands of words, phrases and web addresses to be filtered out of search results so that links to politically objectionable websites do not even appear on the search engine's results pages, even when those websites may be blocked at the backbone or ISP level . . . Such lists are not given directly to Internet companies by the Chinese government; rather, the government leaves the exact specifics and methods of censorship up to companies themselves. Companies generate their "block-lists" based on educated guesswork plus trial-and-error: what they know to be politically sensitive, what they are told in meetings with Chinese officials, and complaints they may receive from Chinese authorities in response to the appearance of politically objectionable search results.

But the complicity of companies is even more direct: they actually run diagnostic tests to see which words, phrases, and web addresses are blocked by the Chinese authorities at the router level, and then add them to their lists, without waiting to be asked by the authorities to add them. And because they seek to stay out of trouble and avoid complaints from the authorities, many businesspeople who run [Internet Content Providers] in China confess that they are inclined to err on the side of caution and over-block content which does not clearly violate any specific law or regulation, but which their instincts tell them will displease the authorities who control their license. In all these ways, companies are doing the government's work for it and stifling access to information. Instead of being censored, they have taken on the role of censor.

In other words, by keeping secret the exact terms that will trigger a license revokation, the Chinese government achieves more censorship than it could by publishing a list of forbidden terms, and it makes Google, Yahoo!, and other U.S. companies complicit in the censorship. Similarly, OHRP's vagueness about what will trigger a shutdown fails to assure universities that they can safely deregulate research, so universities restrict research that should be exempt from review.

Fortunately, OHRP's new director, Jerry Menikoff, understands this. In the Winter 2009 issue of AAHRPP Advance he writes,

We often hear that it’s better not to provide specific guidance—that the absence of guidance allows people greater flexibility in interpreting the regulations. In my experience, the opposite can be true. Guidance can empower individuals and advance both research and research protections. In the absence of guidance, people tend to be reluctant to take certain actions out of fear that they are violating the rules. In some instances, important research is not even attempted, all because of a misunderstanding. Guidance could eliminate the misconception and clear the way for research.

I'm delighted that Dr. Menikoff takes this approach, and I look forward to more specific guidance from OHRP that would clear the way for research.

[Thanks to Rob Townsend for altering me to Menikoff's comments.]

Comments Oppose New Regulations on Training

As reported by the Chronicle of Higher Education, OHRP recently released the replies it had received in response to its July call for comments on education and training requirements. I thank Chronicle reporter David Glenn and OHRP associate director Michael Carome for supplying me with copies of the comments.

As I see it, the comments pose three main questions.

Can We Patch This Flat Tire?

The fourth article in the PS symposium is Felice J. Levine and Paula R. Skedsvold, “Where the Rubber Meets the Road: Aligning IRBs and Research Practice.” Both authors been involved in IRB debates for several years, and this article reflects their sophisticated understanding of some of the issues involved. But for an article published in a political science journal, it is disappointingly insensitive to the power dynamics that govern IRB-researcher relationships.

Unlike symposium participants Tony Porter, Dvora Yanow and Peregrine Schwartz-Shea, Levine and Skedsvold do not question the premise that IRBs help promote ethical research. Instead, they assert that there is no fundamental conflict between IRBs and social science researchers: "federal regulations, professional ethics codes, and research practice may have shared goals but tend to speak with different languages—creating frustration and skepticism in a system that could potentially work quite well if transformations are made." (502) Based on that assertion, they suggest four such transformations, ranging from the bold to the timid.

Michael Rowe on Situational Ethics

Bill Hart kindly referred me to Michael Rowe, "Tripping Over Molehills: Ethics and the Ethnography of Police Work," International Journal of Social Research Methodology 10 (February 2007): 37-48. Rowe, a criminologist at Victoria University, New Zealand, explains the ethical challenges he faced as a participant observer with British police in 2004.

Rowe comes across as an unusually conscientious researcher. He writes, "while designing the project, and discussing it with academic colleagues and gatekeepers within the police service, I read many of the methodological texts on ethnography and accounts by previous researchers who had used similar methods with the police." (38) Some of this preparation was overkill, in that Rowe did not himself face some of the greatest challenges of previous researchers, like witnessing excessive force, false charges, or corruption. But it did sensitize him to some important issues about police work, and the result seems to have been a careful study, respectful at once of the police, the citizens with whom they interacted, and the scholarly pursuit of truth.

Rowe concludes that generic ethical advice and rigid rules are poor guides to researchers doing fieldwork:

It is the nature of ethnographic research that the principles contained in methodological textbooks or professional codes of conduct will be stretched and perhaps distorted as they are applied in dynamic situations. Since policing is unpredictable, the ethical dilemmas police researchers might face cannot be easily anticipated . . . If an absolute code of ethics is not feasible, researchers must be prepared to be reflexive in terms of ethical dilemmas and the methodological difficulties experienced in securing informed consent and meaningful access to research subjects. (48)

Though Rowe does not mention ethics committees in this article, it is striking how much his preparation diverged from the typical requirements of IRBs, at least in the United States. Rowe's experience points to the benefit of reading as specifically as possible in preparation for fieldwork. But the standardized training programs required by most IRBs, such as the CITI Program, present highly generic information about such topics as informed consent, and nothing about topics as specific as police ethnography. And while Rowe emphasizes the researcher's need to remain flexible, IRBs focus on protocol review. By making researchers pledge in advance what they will and will not do, protocol review reduces, rather than enhances, researchers' flexibility to adapt to unexpected situations. In other words, the IRB system is structured to hamper the kind of ethical preparation that Rowe recommends.

As I've mentioned before, the University of Pennsylvania's policy on evolving research promises to relieve the second part of this problem, since some researchers, at least, are spared the need to file a "fixed research protocol."

But Penn still requires its researchers to "have documented discipline-appropriate education regarding human subject protection, in accordance with certification standards defined by the Vice Provost for Research." While Penn staff have refused me permission to see the approved training modules, from corresponding with people at Penn, I get the sense that they are pretty generic. Rowe's article suggests that such programs are not helpful, and what is really needed is for each researcher to prepare an ethical bibliography, based on problems faced by researchers who have conducted similar work. That way, each researcher would be equipped with the ethical guidance most relevant to her particular case. And in assembling the bibliography she would exercise the very independent judgment she will need in the field.

(Thanks to Rebecca Tushnet for discussing with me the idea of an ethical bibliography.)

AAHRPP Calls for Research on IRBs and Behavioral and Social Science

Rob Townsend kindly alerted me to the Winter 2008 issue of AAHRPP Advance, published by the Association for the Accreditation of Human Research Protection Programs, which features the lead article, "IRBs and Behavioral and Social Science Research: Finding the Middle Ground." The article concedes that "many behavioral and social scientists feel constrained by a system that seems tilted toward biomedical research and, therefore, neither understands nor reflects their concerns." And it reports the interest of Drs. J. Michael Oakes, "an author and frequent lecturer on IRB review of behavioral and social science research," and Howard Silver, Executive Director of the Consortium of Social Science Associations, in addressing some of those concerns.

Oakes and Silver "encourage researchers to investigate scientifically IRB oversight of behavioral and social science research. Such research could determine whether IRBs are consistent in applying the federal regulations, whether IRBs are taking advantage of the flexibility that’s built into the regulations, and whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research. The resulting data could shed light on ways to relieve tensions between these two groups."

That sounds good, but it's a bit disappointing that the article does not acknowledge the considerable research already completed on this topic, much of it already cited on this blog. Nor does it remark on the curiosity that IRB oversight has continued for four decades without anyone knowing if it does any good.

Moreover, the article states as fact some beliefs that should be investigated with just the sort of research it calls for. I hope Oakes, Silver, and AAHRPP will allow data to challenge some of their own presuppositions.

Here are some questions that could be answered by further research.

1. When did IRB review of social science go bad, and why?

AAHRPP thinks it already knows the answer to this one. The article claims that "Tensions began building in the late 1990s in response to increased government scrutiny of research involving human participants," and that "the regulations have not changed. What’s new is their enforcement and, in many instances, that enforcement is overdue." I am at work on a history of IRB review of social science and humanities research, and I think more research can challenge this view.

The first part of the AAHRPP claim is doubtful; social scientists have protested IRB regulations since 1966, and tensions have waxed and waned since then. If we take this longer view, then the assertion about the immutability of the regulations is wrong; the regulations--first promulgated in 1974--changed twice, in 1981 and 1991. And the 1991 revisions greatly expanded the reach of IRBs. The 1981 regulations exempted survey, interview, and observational research unless it “deals with sensitive aspects of the subject’s own behavior, such as illegal conduct, drug use, sexual behavior, or survey or interview procedures is use of alcohol" and if "the subject’s responses, if they became known outside the research, could reasonably place the subject at risk of criminal or civil liability or be damaging to the subject’s financial standing or employability. The 1991 regulations, in contrast, eliminated the "sensitive aspects" clause and added potential harms to reputation to the list of triggers for IRB review. These changes were made over the objection of social scientists. And they set the stage for the conflict of the 1990s and today.

The claim that "what's new is [the regulations'] enforcement," is only half-true. Also new is the guidance issued by OPRR/OHRP since 1995 that reversed previous policies.

Finally, the article claims that "enforcement is overdue." Really? What errors did social scientists commit in the 1980s--a decade of relatively light regulation?

2. Why do IRBs sometimes delay or prohibit social science research?

Dr. Oakes states, “IRB members are not those folks who are looking to thwart your study. They are peer researchers who have a job to do.” But clearly some IRB members are looking to thwart studies, or else studies wouldn't get thwarted as often as they do. The question is how many IRB members do this, and why.

One part of this question concerns membership. Oakes's claim that IRB members "are peer researchers" depends on an odd definition of peers. In the NIH peer review process, for example, proposals are reviewed by study sections whose members are chosen for their expertise. The NIH's Center for Scientific Review requires, among other things, that

"* Expertise is the paramount consideration when developing/updating a study section roster.

"* Each scientific area reviewed by the study section needs appropriate expert representation."

IRBs theoretically must include experts on each type of research reviewed, but Oakes knows as well as I that this requirement is often ignored. Additional research might indicate how often a researcher faces an IRB with no expertise in the methods under review.

Then, of course, some IRB members are not researchers at all, but the "one member whose primary concerns are in nonscientific areas" required by the regulations. As Laura Stark's dissertation suggests, these members can be particularly undisciplined in their meddling.

3. What types of research now fall subject to IRB review?

Like PRIM&R, AAHRPP thinks that IRBs review only two kinds of scholarship: biomedical research, and something called "behavioral and social science research." The article states, "AAHRPP’s Founding Members, Board of Directors, Council on Accreditation, and Supporting Members all include representatives of organizations engaged in behavioral and social science research."

This statement suggests the fallacy of the undistributed middle term:

* Ehnographers are represented by organizations engaged in behavioral and social science research.
* Organizations engaged in behavioral and social science research have a voice in AAHRPP.
* Therefore, ethnographers are represented by organizations that have a voice in AAHRPP.

The second premise is "undistributed," since it is not true that all organizations engaged in behavioral and social science research have a voice in AAHRPP.

Here's a counter example:

* Countries in South America, Africa, and South Asia are not in North America or Europe.
* Countries from parts of the world other than North America and Europe are permanent members of the UN security council and the G-8.
* Therefore, countries in South America, Africa, and South Asia are permanent members of the UN Security Council and the G-8.

In committing this fallacy, AAHRPP lumps together a dozen or more scholarly disciplines--each with its own history, methods, and ethics--into a single category: "behavioral and social science research." The AAHRPP website does not list the disciplinary affiliations of members of its Board of Directors, Council on Accreditation, or list of site visitors, but if there's a journalist, historian, or folklorist in the lot, I'll be surprised.

To take the example I know best, oral historians do not expect psychologists, social workers, or education researchers to understand or represent their interests. AAHRPP (like PRIM&R) should find out how many disciplines are now subject to review, and include representatives from all of them.

4. What models of ethical review exist, and what models might we imagine?

The article asks "whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research." But that is only one of several alternative systems in place on various campuses. For example, Macquarie University delegates ethical review to a number of subcommittees with special expertise in certain fields. And the University of Pennsylvania allows researchers using some social science methods to forego "a fixed research protocol." And we can imagine even more models, some of which would require redrafting present regulations, others of which might not.

I appreciate AAHRPP's call for research, and I hope it agrees that research is most valuable when the answers are not predetermined.

Guidance Creep

I am often frustrated by the argument that federal regulations provide, in Jeffrey Cohen’s words, “sufficient flexibility for the efficient and appropriate review of minimal risk research.” While individual IRBs have much to answer for, federal regulators have, over the years, stripped them of a great deal of flexibility.

I recently came across a striking example of this. In 1983, Richard Louttit of the National Science Foundation, who had helped craft the list of exemptions encoded in 45 CFR 46.101, explained them as follows:

Much research of minimal risk was exempted from IRB review in order to reduce the IRB workload so that research involving ethical questions could get more than cursory review. But some institutions have decided that the IRB, or its chairperson, must review proposals to decide if they are exempt from review. If this seems contradictory, it is. And this was not envisioned by the staff group which worked out the exemptions.

[Richard T. Louttit, "Government Regulations: Do They Facilitate or Hinder Social and Behavioral Research?," in Joan E. Sieber, ed., NIH Readings on the Protection of Human Subjects in Behavioral and Social Science Research: Conference Proceedings and Background Papers (Frederick, Md: University Publications of America, 1984), 179.]

Yet in 1995, the Office for Protection from Research Risks adopted just that contradictory position:

Institutions should have a clear policy in place on who shall determine what research is exempt under .46.101(b). Those persons who have authority to make a determination of what research is exempt are expected to be well-acquainted with interpretation of the regulations and the exemptions. In addition, the institution should be prepared to reinforce and review, as necessary, the method of determining what is exempt. OPRR advises that investigators should not have the authority to make an independent determination that research involving human subjects is exempt and should be cautioned to check with the IRB or other designated authorities concerning the status of proposed research or changes in ongoing research.
(OPRR Reports, 95-02) [21 July 2014 updated link.]

The regime in place today is far more intrusive than the one worked out in 1981. Changes in the regulations themselves are part of the problem, but so are radical reinterpretations like the one above.