Librarians Ponder IRB Resolution

On June 29, at the American Library Association's Annual Conference, Melora Ranney Norman proposed a "Resolution on Institutional Review Boards and Intellectual Freedom."

Norman, a former chair of the ALA's Intellectual Freedom Round Table, noted that

Despite the fact that walking down the street is more dangerous than any conversation could ever be, on some college and university campuses, assertions of liability or vague, unproven risk are allowed to trump any actual proof of risk or danger, to the detriment of the preservation of knowledge and the human record.

Libraries are all about preserving and providing access to the human record with all its pimples, bumps, and bruises. Many of us have heard a quote attributed to Jo Godwin asserting that "A truly great library contains something in it to offend everyone." If the human record is not created to begin, how can we collect, preserve, and provide access to it?

She then called for the ALA to "[support] the American Historical Association in its position on oral history Institutional Review Board exemption, and [join] with the American Association of University Professors in recommending that 'research on autonomous adults whose methodology consists entirely in collecting data by surveys [or] conducting interviews . . . be exempt from the requirement of IRB review—straightforwardly exempt, with no provisos, and no requirement of IRB approval of the exemption.'"

The ALA Council voted to refer the resolution to the Intellectual Freedom Committee, the Library Research Roundtable, the Library History Roundtable, and the Committee on Professional Ethics. The American Historical Association hopes "they will reconsider the decision and support our efforts after further review."

Norman has also posted a Q & A, IRBs and Intellectual Freedom.

Hat tip: Rob Townsend.

A Plea for "Networked Learning"

Alexander Halavais reports on a recent workshop sponsored by the Digital Media and Learning Research Hub, which brought together experts to discuss the challenges that IRBs pose to research in digital media.

Having agreed that IRB review sometimes produces unnecessary delays, particularly when multiple IRBs must sign off on a collaborative project, the workshop participants

found that while there might be some fairly intractable issues, as there are for any established institution, some of the difficulties that IRBs and investigators encountered were a result of reinventing the wheel locally, and a general lack of transparency in the process of approving human subjects research. The elements required to make good decisions on planned research tend to be obscure and unevenly distributed across IRBs. From shared vocabularies between IRBs and investigators, to knowledge of social computing contexts, to a clear understanding of the regulations and empirical evidence of risk, many of the elements that delay the approval of protocols and frustrate researchers and IRBs could be addressed if the information necessary was more widely accessible and easily discoverable.

Rather than encouraging the creation of national or other centralized IRBs, more awareness and transparency would allow local solutions to be shared widely. Essentially, this is a problem of networked learning: how is it that investigators, IRB members, and administrators can come quickly to terms with the best practices in DML research? Not surprisingly, we think digital media in some form can be helpful in that process of learning.

That is not an implausible idea. Plans for IRBs to share problems and solutions date back to the early 1970s, and they resulted in such institutions as PRIM&R and the journals, IRB: Ethics & Human Research and, more recently, the Journal of Empirical Research on Human Research Ethics. But these are fairly low-bandwidth channels: infrequent conferences and journal issues, with a few dozen sessions or articles per year, and devoted primarily to biomedical research. Hardly enough to generate a sustained discussion of an issue like social computing.

Alternatively, there exist online exchanges, like the IRB Forum. But these may lack the rigor of the journals. Rather than offering "empirical evidence of risk," as Halavais wants, they can amplify unrealistic fears. As Norman Bradburn testified before the National Bioethics Advisory Commission in 2000:

What is bothersome to me is that -- and the trend that I see in IRB's -- is that they are becoming more and more conservative, that is there is a kind of network at least in the ones that -- there is a kind of -- I do ont know what you call it -- ListServ kind of network that administrators of IRB's communicate with one another and they sort of say here is a new problem, how do you handle that, and then everybody sort of responds. And what happens is the most conservative view wins out because people see, oh, gee, they interpret it that way so maybe we better do it too. So over time I have seen things getting more and more restrictive . . .

May I suggest, then, that without proper supervision, digital media can be a liability rather than an asset. The challenge for Halavais and his colleagues is to build a conversation that combines immediacy and scholarly care.

NYU IRB Withdraws Demand that Field Notes Be Shared

Professor Deborah Padgett of the Silver School of Social Work, New York University, posted a query on H-MedAnthro concerning her IRB's demand that she allow her informants to review her ethnographic field notes.

Readers of that forum encouraged her to resist the demand. Simon Lee, a medical anthropologist and an IRB member, was particularly emphatic:

Field notes are raw data, much like a lab notebook in bench science. Raw data is private and not appropriate to share, precisely because lay people draw conclusions from looking at fieldnotes that pre-empt the anthropological analysis. Sharing raw field notes during an active (in process) study is really only appropriate for a safety/audit process: that is, the IRB can request to review your field notes for example if there was a concern about confidentiality and you needed to explain what precautions you were taking. But generally field notes should not be available for review by informants on a required or routine basis. The whole point of protecting raw field notes is so that ONLY the anthropological team sees the raw data and no one makes assumptions about their observations: not informant's colleagues, not informant's supervisors etc. To make them available creates both a chilling effect on your ability to collect data, and can in fact promote misunderstanding because it is raw data.

Others pointed out that sharing the notes would allow informants to see what their coworkers had said, perhaps in confidence, thus adding rather than decreasing risk.

Having read these replies, Padgett

crafted a response to the IRB asserting that this was not appropriate and a troublesome precedent for other ethnographers. The Chair (a psychologist) replied graciously and said that the committee was in error as they had assumed field notes were the equivalent of a video (which the 'subjects' are allowed to view and amend/erase).

While this story has a happy ending, it is still troubling that an IRB at a leading research university would make such an inappropriate, ill-informed demand. A less self-assured, perhaps less senior, researcher might well have bowed to the requirement, at the expense of both the research and the welfare of the people being studied.

It Can Happen to Anyone

On Saturday, my Journal of Policy History article, "How Talking Became Human Subjects Research: The Federal Regulation of the Social Sciences, 1965–1991," was honored with that journal's Ellis Hawley Prize. In presenting the prize, Professor Hawley--one of the leaders of my profession--mentioned that he himself used to ask his students to interview someone who had lived through the Great Depression. On being told that he would have to submit to IRB review, he abandoned the assignment. If IRBs can deter Ellis Hawley from learning more about the 1930s, we have a problem.

APA Launches Committee on Human Research

John Mueller kindly alerted me to the formation of the American Psychological Association's Committee on Human Research, which met for the first time in March. The committee expects to work for the next 3-5 years on various issues, including, at the top of the list, "interpreting federal regulations for psychological research."

Among the committee members is Miriam F. Kelty, who, as an NIH psychologist, served on the staff of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research in the 1970s.

Researchers Deceive Thousands of Professors

Professor Andrew Gelman reports that he was sent a deceptive e-mail as part of a research project by two business professors, Katherine Milkman and Modupe Akinola. Milkman and Akinola wished to see if "students from underrepresented groups" (presumably racial and ethnic minorities, and perhaps women) would be less likely to gain the interest of doctoral faculty than "other students" (i.e., white guys). So they sent e-mails to Gelman and about 6300 other professors in PhD-granting departments at American universities. The messages, purportedly from a student planning to apply to PhD programs and wishing for a brief meeting, varied by the name of the student and by the time of the proposed meeting. When a professor answered, they replied to cancel the meeting. The idea was to see if students with white-guy names received more or fewer invitations to meet than others.

After receiving a debriefing e-mail explaining the sham, Gelman replied to the authors that "My helpful impulses toward inquiring students are being abused by this sort of study, which I think belongs in the trash heap of ill-advised research projects along with Frank Flynn's notorious survey from a few years ago when he tried to get free meals out of NYC restaurants by falsely claiming food poisoning." He later elaborated on his blog, "What bothers me is that we were involuntary participants in the study. The researchers took advantage of our time and our good nature (that we were willing to meet with an unfamiliar student). Not cool."

Texas's All-Star IRB Report

In February 2008, the University of Texas System formed an IRB Task Force to examine ways to improve IRB operations throughout the UT System. In April 2009, that task force issued its report: "IRB TASK FORCE REPORT: Trust, Integrity, and Responsibility in the Conduct of Human Subjects Research."

I recently obtained a copy of that report. While it has not previously been posted on the web, a University of Texas official assured me that a final policy report like this is public information under Texas law. So to make this important public document freely available, I have posted a copy on my website. (See link above.)

The report offers an exceptionally thorough and thoughtful consideration of how IRBs should work at great research universities. While some of its recommendations may be inapplicable to univerisities that are not part of larger systems or do not operate a medical campus, many of the task force's procedures and recommendations offer a model for others. I salute all those who were involved in the report's preparation.

In particular, I commend the following elements of the UT task force and its report.

Procedures

The UT system carefully set up the task force in a way that would build respect for its findings. In particular, it managed to:

1. Represent Multiple Disciplines

UT included task force members "representing a variety of academic disciplines," among them SACRHP member Lisa Leiden. (3, 24) It also sought help from a range of consultants, including such "national experts" Tina Gunsalus; John Heldens, Moira Keene, Dan Nelson, Ivor Pritchard, and Marjorie Speers. (8)

2. Include Stakeholders

At each UT campus, the task force solicited comments from university officials, IRB chairs, investigators, and other interested parties. (8)

3. Allow Adequate Time

The UT task force completed its deliberations over the course of a year, from February 2008 to February 2009, then released its report in April 2009. A report with so broad a scope cannot be rushed.

Recommendations

Careful investigation led to thoughtful recommendations. The report suggests that the UT system:

1. Employ Faculty Expertise

The UT report recognizes that researchers are often expert in a particular area of human subjects research, and it recommends that faculty experts be identified to prepare standards for specific types of research (e.g., research involving subjects with impaired capacity, internet research) and to consult on individual projects as needed. (10)

2. Utilize Flexibility and Empirical Evidence

The report recommends that the university encourage IRB staff to provide "an efficient level of regulatory review compatible with adequate protection of human subjects," rather than the most stringent level of review. In particular, it suggests that IRB staff and members employ empirical evidence when determining risk, by consulting experts or scholarly literature. (11) The report also suggests that IRBs rely strongly on experts when determining the scientific soundness of a proposal. (14)

3. Uncheck the Box

The UT report notes that, according to Speers, fewer than 50 percent of AAHRP accredited institutions check all the box on their federal-wide assurances, and that it is "primarily the major research universities that are considering unchecking the box," thus maximizing their flexibility in handling projects not directly funded by a Common Rule agency. (12) (More on this in an upcoming post.)

4. Diversify the IRB

The UT report recommends that "Institutional Officials should ensure that the institution's disciplines are well represented in the IRB." (18)

5. Provide IRB Oversight

The UT report recommends that institutions "consider the implementation of a research ombudsperson to increase the opportunity for rapid resolution of issues involving human subjects research." (21) It also suggests an ongoing "IRB Advisory Group" to implement recommendations and assist with human subject policy issues. (23)

6. Define Key Terms

The UT Report includes definitions of key terms. Among other things, these definitions make clear that information-gathering interviews, service surveys, and classroom activities may not meet the definition of human subjects research, and that biography and oral history interviews do not meet that definition. (27) As I mentioned earlier, this recommendation has already led to the deregulation of oral history at the University of Texas at Austin.

7. Explore Alternatives

The report include an appendix on "alternative IRB models," based on the November 2006 "National Conference on Alternative IRB Models." While it is promising that the task force is willing to consider such alternatives, I would have liked it to pick up on that conference's call for "further exploration" of models for social and behavioral research. The University of Pennsylvania policy on research in the sociobehavioral sciences might be such a model.

Goal

Most importantly, the UT task force understood the IRB problem as something larger than an administrative challenge. While it explored ways to increase IRB "effectiveness, efficiency, and productivity," it went beyond such managerial concerns to probe big questions about "IRB authority, mission, and functions," the "unnecessary obstruction of research and lapses of effective human subject protection," and best practices from the available literature. (8)

The task force hoped to foster a "culture of conscience" rather than a "culture of compliance," and it understood that conscience cannot be dictated from above. (20) If other universities also seek to promote a culture of conscience, they must give a voice to all those involved with human subjects research.

I See Dead People

A history professor at Central Connecticut State University and one of his graduate students, a retired corporate lawyer, spent months persuading the state Freedom of Information Commission to release medical records from hospitalized Civil War soldiers and veterans.

[Thomas B. Scheffey, "A Legal Skirmish Over Civil War Records," Connecticut Law Tribune, 26 April 2010. Thanks to Josh Gerstein for the reference.]

The case is a reminder that ethics rules can interfere with the scholarly study of not only the living, but also the dead. For an excellent introduction to this issue, see Susan C. Lawrence, "Access Anxiety: HIPAA and Historical Research," Journal of the History of Medicine and Allied Sciences 62 (2007): 422-460.

Chronicle Readers Vent IRB Complaints

The Chronicle of Higher Education's "Chronicle Forums" features a discussion of the question, "Do IRB's Go Overboard?" Unsurprisingly, several off the discussants answer yes. Among the angriest:

1. "I supervised an MA student a few years ago whose ethics proposal was sent back seven times. By the end they were asking questions like "what if you become really famous from this research and then the police decided to subpeona your records and the confidentiality of your sources was comprised?". Sadly, that's not even a made up question (and this was for an MA in English lit, btw). Obviously, the student was freaked and the process took seven months and almost scuttled her plans. We were drawing up a Plan B for her thesis to become about the review process, since she wouldn't have time to do the actual research. In the end she got what she wanted, did an excellent thesis and kissed academia good-bye rather than pursue a PhD since she thought the whole process of doing research is clearly deranged."

2. "IRBs are better some places than others, and it depends on the discipline. At my current institution, humanities scholars are subject to an IRB that only makes sense for scientists collecting blood and doing life-threatening experiments on small children. Yet, most of the major ethical concerns that are well known and of the greatest importance and concern in my discipline, those aspects of tangible risk resulting from research, are outside the purview of the IRB forms/processes. The policies change every year without notice, meaning it's very hard to teach our grad students how to successfully navigate the IRB process. It's become an arbitrary affair."

3. "ARGH YES. A network ("snowball") sample (so I don't know exactly who or how many yet!), semi-structured exploratory interview protocol that will evolve during the course of data collection--it was just too much for them. In principle, I'm OK with them watching out for ethical stuff, of course, but when they start questioning my research methods, they've gone too far. Back off. You're a chemist. The advice to be as vague as possible--yes. What I really want is exempt status when I'm talking to professionals about their professional roles--I've heard of it at other universities."

4. "My subjects are often powerful people doing bad things. It is quite reasonable to suppose that as a result of my research they could lose their jobs (it hasn't happened so far, but it could). And I wouldn't necessarily think that was a bad thing. From my own perspective, research ethics means making sure interviewees understand the implications of being interviewed, know the potential uses to which the data could be put, and that any deals I make with them about anonymization, or not writing about certain topics, or whatever, are kept. If they don't want to be interviewed, sometimes I use other ways to find out about what they are up to, and write about them anyways. Which often makes them prefer to be interviewed, because at least then they can put their view across.

"It seems to me that in an environment where the standard assumption is that no harm can come to research subjects as a result of the research, this kind of research is right out. Which pretty much means that I can't come back to the USA. Which is ok, since its not like there are any jobs there anyways."

5. "A new pernicious IRB trend is to need to get targeted organizational sign off on research about an organization, even if you interview individuals who work for it in their homes or in a coffee shop off campus. Clearly, this is to protect one's own campus from lawsuits, but I'm not sympathetic. (Key is the more important and latent function of IRBs-- protecting one's organization, not the subjects.)"

The discussion also features some defenses of IRBs, but they are vaguer and less eloquent. In particular, none tells a story of an IRB review that proved necessary.

More Universities Deregulate Oral History

The University of Texas at Austin has ruled that "Biography or oral history research involving a living individual is not generalizable beyond that individual. Therefore, it does not meet the definition of research and does not require IRB review and approval." This is based on the recommendations of a University of Texas System task force report whose public release I am trying to secure. It is a stronger than Texas's earlier statement that oral history "in general" did not require review.

Also deregulating oral history are Brigham Young University and Princeton University. Princeton's policy is particularly clear:

Proposed research including journalistic interviews, oral histories, biographical profiles, or other forms of nonfiction narratives, normally does not fall within the jurisdiction of the IRB. In these cases, the individuals being interviewed understand that they are being quoted, and have every expectation that their views will be made known. The interviewees are advised of their right to remain anonymous, to have their remarks printed without attribution, or kept 'off the record'. If the interviewee is directly quoted, they are allowed to read or hear the quotations attributed to them. The interviewee will also be advised of any publication plans for the project. Most projects from Humanities meet the above criteria; therefore they do not qualify for the IRB review, and do not need to submit their project to the IRB for approval.

Other colleges and universities that have cleared oral history include Amherst College, Columbia University, University of Missouri-Kansas City, the University of Michigan, and the University of Nebraska-Lincoln.

This is still not a long list, but the field is shifting from early 2006, when the American Historical Association struggled to find such unambiguous statements.

---

Update, 4 May 2010: I have posted a copy of the Texas report.