Tuesday, November 30, 2010

Friday, November 26, 2010

Survey: One-Third of UConn Researchers Dislike CITI Program

A 2007 survey of researchers at the University of Connecticut found that more than one third were dissatisfied with the Collaborative Institutional Training Initiative (CITI) program in human subjects research.

The UConn IRB and Office of Research Compliance offered the survey to about 350 researchers, of whom 114 (33 percent) returned it. Part of the survey asked respondents about the CITI Program:

7 Questions asked respondents to rate different aspects of the CITI course on a scale of 1-7 (1=least, 7=most). 4 out of these 7 questions asked if the CITI course increased understanding of risks and protections for human subjects in research. There were no statistical differences in the answers received on this group of 4 questions.

53% rated this group 5 or above
16% rated this group 4, moderate
31% rated this group 3 or below

Similar rates were received for overall satisfaction with the CITI course:

54% rated it 5 or above
9% rated it 4, moderate
37% rated it 3 or below

The course did appear to have an impact on the respondent's understanding of the Federal
Regulations. On this criteria,

72% rated it 5 or above
4% rated it 4, moderate
24% rated it 3 or below

The course had a negative impact on the respondents' willingness to join an IRB:

29% rated it 5 or above
13% rated it 4, moderate
58% rated it 3 or below

These figures suggest wider dissatisfaction with CITI than one of its founders, Paul Braunschweiger, admitted in a 2006 presentation. That presentation (slide 60) reported that principal investigators gave the program an average of about 7.8 on a 10 point scale on overall satisfaction. Though the presentation did not show the distribution of researchers' responses, it would be difficult to get so high a mean if 37 percent of researchers offered negative assessments. We need more data.

The UConn survey also offered researchers the chance to write open-ended comments. The most common suggestions were that the training should be shorter, and that the course content "should be limited to a researcher's area of research." Researchers were happy with the online form of the course, with 74 asking for no change, and only 12 choosing the next most popular option: video instruction.

All of these results suggest the potential for online courses that are shorter than CITI and targetted to a specific research discipline, such as Macquarie University's Human Research Ethics for the Social Sciences and Humanities.

UConn also surveyed researchers on their views of the UConn IRB. But the university has only reported the mean ratings, not the distribution of responses, so it is impossible to say if the IRB earned as many unsatisfactory grades as did the CITI program.

Sunday, November 21, 2010

La Noue Reviews Ethical Imperialism

Political scientist George La Noue terms Ethical Imperialism "a powerful indictment of the IRB regime."

[George R. La Noue, Review of Zachary M. Schrag, Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965-2009, Law & Politics Book Review 20 No. 11 (October, 2010): 616-618.]

La Noue, who has himself written about the IRB controversy, notes that "Universities might seem to be a most unlikely place to welcome and implement a process that is in effect a form of prior censorship. Reconciling the IRB process with legal or professorial concepts of academic freedom is extremely difficult." He finds that "Schrag provides a carefully researched and well written historical perspective providing all members of the academy with essential information to reconsider the role of IRBs."

La Noue calls for more study of the constitutionality of current IRB regulations and practices, a subject I would prefer to leave to the law professors. He also concludes that

What is missing is comparative empirical research about the standards and procedures of a variety of IRBs in different settings. While it seems intuitively unlikely that the process is always fair, objective and consistent, from IRB committee to committee, campus to campus, beyond anecdotes what proof exists? Without an appropriate factual basis, courts would struggle with both the compelling interest and narrow tailoring prongs that constitute the strict scrutiny test that should apply to censorship. Schrag’s book provides a necessary and very carefully researched historical context for the debate about IRBs, but the next step needs to be taken by professional associations and social scientists to study their actual practice to see if the current system can be improved.

I concur, though I would suggest that we in fact need two separate branches of such emprical study. One would continue the work of Maureen Fitzgerald and Laura Stark, both of whom have observed committees in action without finding huge variation from campus to campus, or even--in Fitzgerald's case--country to country.

A second branch would look at the development of human research protections policies. Read through this blog, and you will find enormous variation in casts of characters involved in shaping university policies on human subjects, from research offices that feel free to make up whatever rules they want, to the participation of university-level faculty committees, to the involvement of departments most affected by a given policy.. I don't know of any scholarship that has examined this variation in depth.

I hope that other scholars heed Professor La Noue's call.

Thursday, November 18, 2010

Is Facebook Data Mining Human Subjects Research?

Recent law-school graduate Lauren Solberg finds that "data mining on Facebook likely does not constitute research with human subjects, and therefore does not require IRB review, because a researcher who collects data from Facebook pages does not 'interact' with the individual users, and the information on Facebook that researchers mine from individual users' pages is not 'private information.'"

[Lauren Solberg, "Data Mining on Facebook: A Free Space for Researchers or an IRB Nightmare?" article under review, University of Illinois Journal of Law, Technology & Policy 2010 (2). The article has been accepted for publication, but the journal is still soliciting comments.]

Solberg challenges policies now in place at Indiana University and the University of Massachusetts Boston, where researchers must get Facebook's written permission or the written permission of every individual who is studied. These policies, she argues, impose unnecessary burdens on researchers and IRBs alike. (The two policies are identical, but it's not clear which university borrowed from the other.)

She argues that most data mining projects do not meet the regulatory definition of human subjects research. Reading existing profiles is not interaction with an individual. Nor is a Facebook profile that is open to strangers private information, i.e., "information which has been provided for specific purposes by an individual and which the individual can reasonably expect will not be made public (for example, a medical record)." If a college admissions officer or a potential employer can read your profile, you've lost little by having an anthropologist read it as well.

This analysis seems sound, but it's not clear to me that anyone disagrees. In particular, the third university Solberg mentions, Washington University in St. Louis, applies its policy only to "
Any activity meeting the definition of 'human subject research' which is designed to recruit participants or collect data via the Internet.
" It then lists several examples, most of which involve interaction with living individuals. Thus, I doubt Solberg's claim that "researchers at Washington University need only inform Facebook users that they are recording information that is posted on their pages." Rather, if the project does not meet the definition of human subject research, then Wash U. researchers need not do even that much.

Solberg's article skirts some interesting questions. One concerns the boundaries of a reasonable expectation of privacy. Thus, Michael Zimmer gives the example of a study by Harvard graduate students of the Facebook profiles of Harvard undergraduates. If an undergraduate had made some information visible only to other Harvard students (a choice Facebook's software allows), and a Harvard student-researcher sees it, does that change Solberg's analysis?

A second question concerns the authority of university research offices and IRBs to insist that researchers abide by website terms of service. Notably, the Indiana and UMASS policies do not cite federal human subjects regulations as their authority. Rather, they claim that Facebook and Myspace "explicitly state that their sites are not intended for research but for social networking only."

Solberg writes that evaluating such claims is "outside the scope of this article," but they are interesting in three ways. First, they may be factually false; I could find no such explicit statements in the Facebook or Myspace terms of service. Second, they are divorced from federal regulation. For example, the Facebook terms of service do not distinguish between living and dead Facebook members, whereas federal human subjects protections apply only to the living. Finally, they are internally inconsistent. If Facebook and Myspace did prohibit the use of their sites for research, would not researchers still be violating the terms of service even if they got signed consent from individual members, as allowed by the policies? Just who are these two universities trying to protect?

Solberg concludes that "Unfortunately, and somewhat surprisingly, the OHRP has issued no guidance pertaining to Internet research in general, let alone guidance specifically relating to the issue of data mining on the Internet." To give the feds some credit, in summer 2010 (after Solberg wrote her article), SACHRP did sponsor a panel on the Internet in Human Subjects Research. It can take a long time from a SACHRP presentation to OHRP guidance, but the wheels may be moving on this one.


Note, 19 November 2010: The original version of this post identified Ms. Solberg as a law student. She has in fact graduated. I have also changed the link about Michael Zimmer's work from his SACHRP presentation to his article, "'But the data is already public': on the ethics of research in Facebook," Ethics and Information Technology 12 (2010): 313-325.

Wednesday, November 10, 2010

Comments: FWA Forms Should Reflect Common Rule

On October 4, I reported that OHRP was inviting comments on drafts of new FWA form and FWA Terms of Assurance.

Prior to the October 25 deadline, OHRP received comments from only five individuals and two professional organizations, all of which are posted at regulations.gov.

Of these seven comments, three (including mine, of course) complained that the draft Terms of Assurance, like the existing ones, violate the Common Rule's pledge that an institution's statement of principles "may include an appropriate existing code, declaration, or statement of ethical principles, or a statement formulated by the institution itself."

No one made a case for retaining the discrepancy between the regulations and the forms.

Friday, November 5, 2010

IRBs and Procedural Due Process

A law student finds that "current IRB regulations fail to provide procedural due process as guaranteed by the Fifth and Fourteenth Amendments of the United States Constitution."

[Daniel G. Stoddard, "Falling Short of Fundamental Fairness: Why Institutional Review Board Regulations Fail To Provide Procedural Due Process," Creighton Law Review 43 (June 2010): 1275-1327]

Stoddard notes a number of measures that might protect researchers against capricious IRBs but which are not currently required:

Federal IRB regulations are silent . . . regarding a number of specific aspects of IRB function including public attendance of IRB functions, a researcher's opportunity to hear and cross-examine information opposing that researcher's research, and a researcher's right to privacy with regard to an IRB's media interaction. IRB regulations additionally fail to address whether an IRB should base its decision exclusively on evidence presented to it, whether a researcher should have a right to a hearing before the IRB suspends research, whether a researcher has a right to judicial review of an IRB decision, and whether a researcher has a right to an attorney. Federal IRB regulations also fail to include a researcher's right to have informal communications with an IRB, a researcher's right to present further evidence to an IRB following a rejection, a researcher's right to consult with personnel opposing that researcher's research in an effort to understand and prepare to challenge them, and an IRB's obligation to evaluate its own functioning procedures periodically. (1290)

All of these measures could be helpful, but the question for Stoddard is whether their absence violates procedural due process. To answer that question, he turns to Mathews v. Eldridge (424 US 319 - Supreme Court 1976), a 1976 Supreme Court case named for the same HEW secretary who was sued for violating IRB procedures in Crane v. Mathews, 417 F. Supp. 532 - Dist. Court, ND Georgia 1976.

Mathews states that

the specific dictates of due process generally requires consideration of three distinct factors: First, the private interest that will be affected by the official action; second, the risk of an erroneous deprivation of such interest through the procedures used, and the probable value, if any, of additional or substitute procedural safeguards; and finally, the Government's interest, including the function involved and the fiscal and administrative burdens that the additional or substitute procedural requirement would entail.

Stoddard, it turn, argues that "The inability to contest or appeal an IRB decision is a substantial procedural shortcoming when evaluated under the three prong Mathews [v. Eldridge] balancing test."

The problem with this argument is that Mathews does not require the right to appeal, nor do other key precedents. In particular, Stoddard would be more persuasive had he addressed head-on what I take to be the federal court decision that most directly addressed due process and IRBs: Halikas v. University of Minnesota. Though Stoddard cites the district court's denial of a preliminary injunction to the plaintiff in that case, an aggrieved researcher, he does not analyze the court's reasoning behind that denial: "An IRB proceeding is, simply, not a federal criminal prosecution. Such a proceeding is governed by contracts and federal regulations which do not require, or provide, the full panoply of criminal procedural rights . . . Dr. Halikas voluntarily entered into an employment contract and conducted his research under the aegis of the University and its research-regulatory regime. He received the process which is his due." [Halikas v. University of Minnesota, 856 F. Supp. 1331; 1994 U.S. Dist.]

Nor does Stoddard analyze the final judgment in that case, which was not published. [Case number 4-94-CV-448, Federal District Court, Fourth Division, District of Minnesota; filed 18 May 1994; Judgment entered 9 June 1996. I am very grateful to Dr. Dale Hammerschmidt, one of the named defendants in the Halikas suit, for providing me with a copy of this document. I have posted it on my website (see previous link) so it will be easier to find in the future.]

In that judgment, the court found that "as the Eighth Circuit Court of Appeals has determined in similar cases, the Constitution requires only that Dr. Halikas receive: (1) clear and actual notice of the charges against him; (2) notice of the names of those bringing the charges and the specific nature and factual basis for the charges; (3) a reasonable time and opportunity to respond; and (4) a hearing before an impartial board or tribunal." It did not include the right to appeal as a component of procedural due process under the Constitution.

The "similar cases" which Judge James Rosenbaum used to reach this result were two cases in which employees of public universities contested their firing: Riggins v. Board of Regents of Univ. of Neb., 790 F. 2d 707, 712 (8th Cir. 1985) and King v. University of Minn., 774 F. 2d 224, 228 (8th Cir. 1985), cert. denied, 475 U.S. 1095 (1986).

This comparison casts doubt on Dr. Hammerschmidt's claim that Judge Rosenbaum "formally recognized the concept that the opportunity to conduct research upon human subjects is a privilege, rather than a right." ["'There is no substantive due process right to conduct human-subject research': The Saga of the Minnesota Gamma Hydroxybutyrate Study," IRB: Ethics and Human Research. 19 (May - Aug., 1997): 13-15. This is amplified in Steven Peckman, ["A Shared Responsibility for Protecting Human Subjects," in Institutional Review Board: Management and Function, ed. Robert J. Amdur, Elizabeth A. Bankert (Jones & Bartlett Learning, 2006), 17.]

To the contrary, Riggins specifically states that "Public employees may have a property right in continued employment." And King involved the dismissal of a tenured professor. By invoking these precedents, the Halikas decision suggests that while research is not a substantive due process right, researchers may have procedural due processs rights comparable to those enjoyed by public employees and tenured professors. Halikas makes no mention of research as a "privilege."

Though the Halikas judgment was sufficient to decide the case before the court, it left unanswered many questions about the rights of researchers who face IRBs. Are the procedural protections set forth in King and Riggins adequate to protect the right to research, academic freedom, or a property right in continued employment? Do professors in their capacity as researchers deserve more, less, or equivalent protections as professors in their capacity as teachers? What rights, if any, might student-researchers claim? Would Halikas or King have been decided differently had the plaintiffs offered free-speech claims? Does the "human research" in the judgment refer to social research as well as the medical research that was the subject of the IRB proceedings against Halikas? (The final judgment describes the IRB as a "medical research review body.") Does a board or tribunal have to be competent as well as impartial?

Halikas leaves all these questions unanswered. A careful analysis of that case would be a good starting place for further legal scholarship on the due process implications of IRB policies.

Thursday, November 4, 2010

Hear Me Talk About My Book

Online Programming for All Libraries (OPAL) has posted recordings of my October 27 discussion of Ethical Imperialism as full streaming audio with text chat and a downloadable MP3 audio recording. The presentation lasts 64 minutes.