IRBs Unfamiliar With Internet Research Ethics

Two experts in Internet research ethics find "that IRBs generally do not know what . . . protections apply strictly to online research, and such boards often ignore the complexities of such research and thereby risk harming subjects while also violating federal regulations, or, they apply such restrictive models that inhibit researchers from pursuing important online endeavors."

[Elizabeth A. Buchanan and Charles M. Ess, "Internet Research Ethics and the Institutional Review Board: Current Practices and Issues," ACM SIGCAS Computers and Society 39 (December 2009): 43-49.]

In 2006, the authors sent surveys to more than 700 US-based IRBs, of which they received responses from 334. The learned that

Few boards were aware of extant guidelines such as the Association of Internet Researchers Ethical Decision Making document, and 74% did not provide specific training around Internet research issues, and that less than half (42%) felt the Office [for] Human Research Protections or other regulatory documents were useful in Internet research reviews. These data suggest that ethics boards may not be fully informed when reviewing such research. We must consider on what bases are review boards making decisions around [Internet research ethics]? Of particular concern, our qualitative data provided indications that many boards were "unsure of who to ask," "we don't even know what questions to ask of the researcher," and, "we rely on the IT department to advise us on such IT related issues."

This is typical of an IRB system that rewards members and, especially, staffers who can recite the Common Rule from memory but imposes no real incentive to read the ethics literature of the disciplines over which they claim power.

See also: The Dormant Right to Expertise
and In Search of Expertise.

Research Ethics--Now with Academic Freedom?

Canada's Panel on Research Ethics has released a new (December 2009) draft of the Second Editon of the Tri-Council Policy Statement (TCPS). It is accepting comments until March 1. Below are mine.

AAHRPP Urges Extra Care When Talking to Pregnant Women

As I mentioned in October, Marjorie Speers, president of the Association for the Accreditation of Human Research Protection Programs (AAHRPP), has stated that her organization permits institutions it accredits to submit Federalwide Assurances (FWAs) that do not pledge to impose federal regulations on all research not directly funded by the federal government. In otherwords, institutions may "uncheck the box" without losing accreditation.

At the same time, Speers insisted that "if the boxes are unchecked, we hold the organization to have equivalent protections in place for all research." In my earlier post, I asked what an equivalent exemption might look like.

AAHRPP takes a stab at answering this in the Winter 2010 issue of AAHRPP Advance.

"To Check or Uncheck the FWA Boxes" (p. 6) notes that "The regulations that come into play when the FWA boxes are checked were written primarily with clinical research in mind. Thus, if an organization checks Subpart B of the FWA, for example, it will not be allowed to conduct SBER studies that involve pregnant women, because any study involving pregnant women is required to advance biomedical knowledge."

To give institutions more flexibility, the article claims,

AAHRPP has designed its standards to apply as much to protecting participants in SBER as they apply to protecting those in biomedical research. When organizations have unchecked the boxes, AAHRPP allows them to provide protections that are appropriate to the level and nature of the risk involved in the study and meaningful to the type of research. Under the accreditation standards, an organization could require that the results of a SBER study that includes pregnant women, for example, must contribute to general knowledge or knowledge that is beneficial to society, rather than to biomedical knowledge.

While any concession is welcome, this one is unimpressive. The regulation in question, 45 CFR 46.204 states that "pregnant women or fetuses may be involved in research" only if the research holds out the prospect of direct benefit to the pregnant women or fetus or if "risk to the fetus is not greater than minimal and the purpose of the research is the development of important biomedical knowledge that cannot be obtained by any other means."

Changing "biomedical" to "general" here yields a requirement that researchers meet a higher standard if they want to interview, survey, or observe pregnant women than if they want to interview, survey, or observe non-pregnant women, prospective fathers, or other autonomous adults. Such a requirement violates the rights of both researchers and women.

While I appreciate AAHRPP's understanding that much of the Common Rule is inappropriate when applied to non-biomedical research, the example it has chosen suggests that as an organization, AAHRPP still lacks a basic grasp of some of the issues surrounding IRB review of social science research. What is needed is not mere tinkering with the Common Rule, but wholesale reconsideration of the rights and responsibilities of social scientists.

I thank Rob Townsend for bringing this to my attention.

Canadian Historians Ponder Exclusion from Ethics Board Review

Google rather belatedly alerted me to the Canadian Historical Association's comments on the December 2008 draft of the Tri-Council Policy Statement; I am told the comments were posted on the association's website in late June 2009. Since late is better than never, I mention them now.

The historians are "supportive of the changes that have been made in the second edition and consider it a very good policy paper." They particularly appreciate the various passages sprinkled throughout the statement noting that not all research fits into a standard form, and that research ethics boards need to maintain flexibilility.

But, taking a broader view, the historians are dismayed that these passages appear as mere exceptions to general rules designed for quantitative, especially medical, research. As they put it:

While there is no question that the ethical issues arising from clinical or quantitative research must be addressed, the effect of this emphasis is to marginalize qualitative research in the humanities and some social sciences. Indeed, the TCPS-2 casts all qualitative research as the exception; something best exemplified by the inclusion of Chapter 10, "Qualitative Research." There is no parallel explanation of quantitative research; perhaps because it is considered the "normal" research practice everyone is familiar with.

Casting qualitative research as exceptional puts individuals undertaking such research – like historians – in the position of asking for exemptions from REBs. REBs, like all administrative tribunals, are likely to look on requests for an exemption from the guidelines with suspicion, making the bar higher for those undertaking qualitative research higher – simply because of the kind of research they are doing, not its quality. Not only will historians have to demonstrate the soundness of their particular research designs, but they will also have to establish that the norms of their professional practice are legitimate. In our view, this places an undue burden on our profession and on all those engaged in qualitative research.

This passage neatly summarizes most of what has gone wrong in the regulation of research in the social sciences and humanities over four decades in countries around the world. Medical regulators have written rules based in clinical medical practice, then imposed them on other fields in a way that defines those fields as abnormal, and therefore suspect. Seeing this flaw as intrinsic to the ethics review system, the Canadian Historical Association understandably "urge[s] the Advisory Panel to consider the position of the Oral History Association (US) which since 2003 has argued that oral history should be excluded from institutional review boards."

United States Handicapper General

Writing in the Journal of Broadcasting & Electronic Media, Ruthann Weaver Lariscy of the University of Georgia argues that "Communication journal editors and concerned participants in the system should implement a policy that assures non-U.S. generated communication scholarship report procedures for assuring the protection of human subjects, the use of informed consent, and compliance with comparable IRB protocols."

[Ruthann Weaver Lariscy, "IRBs and Communication Research: Some Perplexing Issues," Journal of Broadcasting & Electronic Media 53 (October 2009): 668 - 671.]

Lariscy notes that IRB review degrades research. She recalls helping to plan a study of eighth-graders in Georgia, only to find that "the human subjects protection protocol required for this study made the research more difficult and depressed response rates," so that she and her colleagues got a participation rate of only 30 percent. In contrast, she refereed a study--apparently done abroad without IRB intervention--which got a participation rate of more than 85 percent. "I am left to conclude that IRB controls may have a considerable impact on our data collection, and the claims we make based on the data," she writes.

She does not claim that such IRB restrictions protect participants in communication research. To the contrary, she applauds the Illinois White Paper's position that "not all methodologies require the same rigorous overview and approval by IRBs, and that procedural changes should be made that acknowledge such methodological and content area differences."

If IRB review is degrading research without protecting anyone, why does Lariscy want journals to impose it on scholars who are currently free of such review?

First, she argues, "Failure to have uniform reporting requirements tarnishes the blind review process. Once a reviewer can identify that a piece of work originated either domestically or internationally, there is potential for bias." Well, yes. But there are dozens of factors that can flag a manuscript as originating outside of the U.S., including the spellings of words, the places studied, the literature cited, and even the proportions of the document. (While I am doing less work on paper, I still notice when a manuscript is formatted for A4.) This is a pretty thin benefit for a significant cost.

And a cost-free alternative exists. Journals could ask that U.S. authors omit descriptions of human protections procedures from their submissions, just as they now omit their own names. This would retain blind review without destroying data.

Bad as her first argument is, Lariscy follows it with an even worse one: "A non-level playing field exists for those of us conducting human subjects studies in the United States compared to the relative ease of conducting similar studies elsewhere." In other words, IRBs are wrecking our research, so let's be sure they wreck everyone's else's.

If Lariscy wants to make the case that journals should insist on IRB review of non-U.S. research in order to protect children, then let her do it. But it is unworthy of a scholar to advocate interference with research for the sake of interference itself.

Two Years' Inaction at OHRP

On October 26, 2007, OHRP formally requested "written comments on a proposed amendment to item 5 of the categories of research that may be reviewed by the institutional review board (IRB) through an expedited review procedure, last published in the Federal Register on November 9, 1998 (63 FR 60364)."

By the December 26, 2007, deadline, 65 people and institutions submitted comments, two-thirds of which concerned oral history or folklore.

That was two years ago. And as far as I can tell, OHRP has taken no action on these comments.

Meanwhile, OHRP's new guidance on what constitutes research subject to regulation, which Bernard Schwetz promised before the end of 2007, is now two years overdue.

After Human Terrain, Will AAA Debate IRBs?

Earlier this month, the American Anthropological Association's Commission on the Engagement of Anthropology with the US Security and Intelligence Communities (CEAUSSIC) issued its Final Report on The Army's Human Terrain System Proof of Concept Program.

The report argues that the Human Terrain System (HTS) combines scholarly research and military information-gathering in a way that muddles ethical issues:

HTS ethnographers attempt to juggle dual loyalties both to civilian populations and to their military units, under conditions which almost inevitably lead to conflicting demands. Potentially conflicting demands (between serving occupied, studied populations, and serving the needs of the military with whom [Human Terrain Teams] embed) almost necessitates that HTS social scientists choose between multiple interests in ways that stand to undermine basic ethical principles that govern research with human subjects among anthropologists and among government researchers. (52)

Significantly, the report more or less recognizes that the choice of interests could go either way. One possibility would be to bring HTS wholly into the realm of scholarly research, with all of its ethical codes and legal regulations, including IRBs:

If HTS carries out a research function as advertised, and if it encourages its social scientists to use ethical research practices, then it should comply with 32CFR219, regulations issued by the Office of the Secretary of Defense (OSD) that address human subjects protection. (47)

Alternatively, the report hints that the real problem is merely a poor choice of words. "We should consider the work of HTTs to be sharply different, in its goals, from conventional disciplinary ethnographic pursuits and not to be 'ethnography' in any credible sense." (54) If HTS were re-branded to avoid the terms "anthropology," "ethnography," and "social science," and instead present itself as a counterinsurgency program pure and simple, then--it seems--CEAUSSIC would not expect it to follow either the AAA ethics codes or the Common Rule. All of this points to the need for clear definitions when discussing ethical and legal obligations.

For the purposes of this blog, a more interesting document is the October 13 blog post, "Why not Mandate Ethics Education for Professional Training of Anthropologists?" by CEAUSSIC member Carolyn Fluehr-Lobban.

Fluehr-Lobban calls for "ethics education as a mandatory part  of anthropology curricula." As she describes it,

A future standard ethics curriculum would minimally include a history of the discipline and ethics– this would help to correct misconstruing history, as has been the case in security engagement polemics where a standard of "voluntary informed consent" is often cited as 'traditional' or normative when, in fact, language on informed consent appears for the first time in the 1998 AAA code.  It would also include case studies representing a realistic spectrum of scenarios and dilemmas where mixed outcomes are the likely norm, and clear positive or negative outcomes are likely exceptions.

But while Fluehr-Lobban seems open to questioning such standards as "informed consent" and to exploring the nuances of real-world research, she is dismissive of comparable discussion of the legitimacy of IRBs:

There is still a tradition of resistance to the annoyance of having to go before an IRB. Part of this history rests with anthropology as the study of "the other," of "subjects," using "informants," whereby the anthropologist is ideally unfettered with unlimited freedom to conduct research. But, clearly, this is not the world we live in. As standard practice, all anthropological research is, or should be, subject to external review.

In other words, Fluehr-Lobban suggests that anyone who doesn't like IRBs wants unlimited freedom to study "the other." This is an insult to the many thoughtful critics who, over the decades, have shown that IRBs and their attendant apparatus can be a barrier to true ethical reflection. It is also an indicator of how entrenched the belief in IRBs has become within the AAA leadership. But has the organization ever really debated whether IRBs are the best way to promote its ethical standards? If not, CEAUSSIC should seize this opportunity for such a discussion within the profession.

Grad Student Needed 80 IRB Approvals

In an account apparently posted in July 2008, Jennifer M. Purcell describes what she went through to get approval for her dissertation research in education at the University of South Florida.

Purcell was investigating the apparent disparity between the knowledge and skills needed by college faculty, and the knowledge and skills taught in doctoral programs. She wanted to ask college professors what they thought faculty and students should know and who should teach it. A typical question asked how important these professors considered the ability to "appreciate the history and purposes of higher education." (Jennifer M. Purcell, "Perceptions of Senior Faculty Concerning Doctoral Student Preparation For Faculty Roles," Ph.D. diss., University of South Florida, 2007.)

Is Documentary Film Human Subjects Research?

Kimberlianne Podlas, a lawyer and an assistant professor of media studies at the University of North Carolina, Greensboro, argues that "virtually all journalistic inquiry and nonfiction filmmaking . . . are not subject to IRB jurisdiction." ("This Film Has Been Rated 'Approved': Are Documentary Films Subject To Institutional Review Board Approval and Federal Human Subjects Research' Rules?")

To reach this result, Podlas argues that documentary films fail one or more of five tests necessary to trigger IRB jurisdiction:

First, the general type of undertaking must be one that is directly regulated by a federal agency. Second, the activity must be "human subjects research"; This requires the undertaking to conform to the regulatory definition of "research." Third, that research must collect information from or about living individuals. Fourth, that information must be either "data" or "private information." And finally, the "human subjects research" must be either biomedical or behavioral.

Let's take these in order.

Hooah!

Google Alerts uncovered an April 2008 memo from the Army's chief of military history, explaining that while Army historians are obliged to obtain the informed consent of anyone they interview, the U.S. Army's Historical Program does not consider oral history to be under IRB purview:

Given that oral history is the collection of personal and unique insights on events, it does not fit the definition of scientific research as outlined in 45 Code of Federal Regulations 46 that is at the center of the Department of Health and Human Services regulations of the issue. Oral histories are not a "systematic" attempt to gather data from "human subjects" that can be used in any way to contribute to "generalizable" knowledge. They are therefore exempt from HRPP oversight and IRBs.

Happy Birthday, Institutional Review Blog!