Monday, May 24, 2010

APA Launches Committee on Human Research

John Mueller kindly alerted me to the formation of the American Psychological Association's Committee on Human Research, which met for the first time in March. The committee expects to work for the next 3-5 years on various issues, including, at the top of the list, "interpreting federal regulations for psychological research."

Among the committee members is Miriam F. Kelty, who, as an NIH psychologist, served on the staff of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research in the 1970s.

Saturday, May 8, 2010

Researchers Deceive Thousands of Professors

Professor Andrew Gelman reports that he was sent a deceptive e-mail as part of a research project by two business professors, Katherine Milkman and Modupe Akinola. Milkman and Akinola wished to see if "students from underrepresented groups" (presumably racial and ethnic minorities, and perhaps women) would be less likely to gain the interest of doctoral faculty than "other students" (i.e., white guys). So they sent e-mails to Gelman and about 6300 other professors in PhD-granting departments at American universities. The messages, purportedly from a student planning to apply to PhD programs and wishing for a brief meeting, varied by the name of the student and by the time of the proposed meeting. When a professor answered, they replied to cancel the meeting. The idea was to see if students with white-guy names received more or fewer invitations to meet than others.

After receiving a debriefing e-mail explaining the sham, Gelman replied to the authors that "My helpful impulses toward inquiring students are being abused by this sort of study, which I think belongs in the trash heap of ill-advised research projects along with Frank Flynn's notorious survey from a few years ago when he tried to get free meals out of NYC restaurants by falsely claiming food poisoning." He later elaborated on his blog, "What bothers me is that we were involuntary participants in the study. The researchers took advantage of our time and our good nature (that we were willing to meet with an unfamiliar student). Not cool."

Gelman's post has spurred several comments. Some feel the study should not have been conducted at all: "I really want . . . a formal apology from both institutions, and an acknowledgment that this project should never, ever have gotten IRB approval." And "Since the researchers took the subjects' time without asking, I think they're guilty of stealing something of this magnitude. People often get put in jail for less."

Others think Gelman is being too sensitive: "Everyone is so precious about this. . . . The fact everyone here wrote, read or commented on this blog post suggests that a few minutes here and there don't cost anyone much." Another cites the work of Peter Riach and Judith Rich, who found that the "minimal inconvenience" imposed on the unwitting subjects of such field experiments can be justified by the "degree of accuracy and transparency which is not available from any other procedure." [Peter A. Riach and Judith Rich, "Deceptive Field Experiments of Discrimination: Are They Ethical?" Kyklos 57 (2004): 457-70.]

Gelman himself is in the middle; he thinks that deceptive surveys are OK, he just wants the researchers to send $10 in compensation to everyone who received the e-mail. "Then, after they send us the study results, if we think the findings are interesting, we can each individually decide whether to send the $10 back to them."

The lesson here is that just because a study is approved by an IRB (two, in this case), doesn't mean it won't leave some participants feeling abused. Nor should IRBs strive for such innocuity; they need to weigh the values of free choice and honesty against the values of knowledge and freedom.

I would be curious to see the original protocols and to learn what reactions Milkman, Akinola, and the two IRBs expected. If they did in fact anticipate that a number of recipients would feel as Gelman does--misused, but only to the extent of $10 in damage--the idea of a compensation fund for the outraged isn't bad. (Though perhaps it could require participants to ask for the money rather than having payment as the default.) If the researchers and the IRBs didn't expect this reaction, there's an opportunity here for some good empirical research. Or, as one commenter put it, "Maybe it's a double dummy study design where they wanted to see how easy it is to annoy professors with email."

Monday, May 3, 2010

Texas's All-Star IRB Report

In February 2008, the University of Texas System formed an IRB Task Force to examine ways to improve IRB operations throughout the UT System. In April 2009, that task force issued its report: "IRB TASK FORCE REPORT: Trust, Integrity, and Responsibility in the Conduct of Human Subjects Research."

I recently obtained a copy of that report. While it has not previously been posted on the web, a University of Texas official assured me that a final policy report like this is public information under Texas law. So to make this important public document freely available, I have posted a copy on my website. (See link above.)

The report offers an exceptionally thorough and thoughtful consideration of how IRBs should work at great research universities. While some of its recommendations may be inapplicable to univerisities that are not part of larger systems or do not operate a medical campus, many of the task force's procedures and recommendations offer a model for others. I salute all those who were involved in the report's preparation.

In particular, I commend the following elements of the UT task force and its report.


The UT system carefully set up the task force in a way that would build respect for its findings. In particular, it managed to:

1. Represent Multiple Disciplines

UT included task force members "representing a variety of academic disciplines," among them SACRHP member Lisa Leiden. (3, 24) It also sought help from a range of consultants, including such "national experts" Tina Gunsalus; John Heldens, Moira Keene, Dan Nelson, Ivor Pritchard, and Marjorie Speers. (8)

2. Include Stakeholders

At each UT campus, the task force solicited comments from university officials, IRB chairs, investigators, and other interested parties. (8)

3. Allow Adequate Time

The UT task force completed its deliberations over the course of a year, from February 2008 to February 2009, then released its report in April 2009. A report with so broad a scope cannot be rushed.


Careful investigation led to thoughtful recommendations. The report suggests that the UT system:

1. Employ Faculty Expertise

The UT report recognizes that researchers are often expert in a particular area of human subjects research, and it recommends that faculty experts be identified to prepare standards for specific types of research (e.g., research involving subjects with impaired capacity, internet research) and to consult on individual projects as needed. (10)

2. Utilize Flexibility and Empirical Evidence

The report recommends that the university encourage IRB staff to provide "an efficient level of regulatory review compatible with adequate protection of human subjects," rather than the most stringent level of review. In particular, it suggests that IRB staff and members employ empirical evidence when determining risk, by consulting experts or scholarly literature. (11) The report also suggests that IRBs rely strongly on experts when determining the scientific soundness of a proposal. (14)

3. Uncheck the Box

The UT report notes that, according to Speers, fewer than 50 percent of AAHRP accredited institutions check all the box on their federal-wide assurances, and that it is "primarily the major research universities that are considering unchecking the box," thus maximizing their flexibility in handling projects not directly funded by a Common Rule agency. (12) (More on this in an upcoming post.)

4. Diversify the IRB

The UT report recommends that "Institutional Officials should ensure that the institution's disciplines are well represented in the IRB." (18)

5. Provide IRB Oversight

The UT report recommends that institutions "consider the implementation of a research ombudsperson to increase the opportunity for rapid resolution of issues involving human subjects research." (21) It also suggests an ongoing "IRB Advisory Group" to implement recommendations and assist with human subject policy issues. (23)

6. Define Key Terms

The UT Report includes definitions of key terms. Among other things, these definitions make clear that information-gathering interviews, service surveys, and classroom activities may not meet the definition of human subjects research, and that biography and oral history interviews do not meet that definition. (27) As I mentioned earlier, this recommendation has already led to the deregulation of oral history at the University of Texas at Austin.

7. Explore Alternatives

The report include an appendix on "alternative IRB models," based on the November 2006 "National Conference on Alternative IRB Models." While it is promising that the task force is willing to consider such alternatives, I would have liked it to pick up on that conference's call for "further exploration" of models for social and behavioral research. The University of Pennsylvania policy on research in the sociobehavioral sciences might be such a model.


Most importantly, the UT task force understood the IRB problem as something larger than an administrative challenge. While it explored ways to increase IRB "effectiveness, efficiency, and productivity," it went beyond such managerial concerns to probe big questions about "IRB authority, mission, and functions," the "unnecessary obstruction of research and lapses of effective human subject protection," and best practices from the available literature. (8)

The task force hoped to foster a "culture of conscience" rather than a "culture of compliance," and it understood that conscience cannot be dictated from above. (20) If other universities also seek to promote a culture of conscience, they must give a voice to all those involved with human subjects research.