Big Data researchers call for IRB review, based on shaky premises

Jacob Metcalf of the Data & Society Research Institute and Kate Crawford of Microsoft Research, MIT Center for Civic Media, and New York University Information Law Institute (I think those are three different things) want to subject Big Data research to IRB review, at least in universities. Their argument rests on shaky premises.

[Jacob Metcalf and Kate Crawford, “Where Are Human Subjects in Big Data Research? The Emerging Ethics Divide,” Big Data & Society 3, no. 1 (January–June 2016): 1–14, doi:10.1177/2053951716650211.]

"Vulnerable" participants may have the most to gain from talking

Seven qualitative researchers forcefully argue that IRBs mislead research participants when they demand consent forms stating that interview research has “no known benefits.” In fact, people labeled “vulnerable” by IRBs often gain a great deal by participating in projects the IRBs deem “risky.”

[Tara Opsal, Jennifer Wolgemuth, Jennifer Cross, Tanya Kaanta, Ellyn Dickmann, Soria Colomer, and Zeynep Erdil-Moody, “‘There Are No Known Benefits …’ Considering the Risk/Benefit Ratio of Qualitative Research,” Qualitative Health Research 26, no. 8 (July 2016): 1137–50, doi:10.1177/1049732315580109.]

NSF Officer Misstates Belmont and Common Rule Standards

In the final contribution to the PS symposium, Lee Demetrius Walker, currently serving as program officer for the Political Science Program at the National Science Foundation, acknowledges the problems of applying a biomedical review system to social science. But he misstates the Belmont and Common Rule standards for assessing research.

[Lee Demetrius Walker, “National Science Foundation, Institutional Review Boards, and Political and Social Science,” PS: Political Science & Politics 49, no. 02 (April 2016): 309–12, doi:10.1017/S1049096516000263.]

Texas A&M IRB Imposed Review for Surveys of Public Officials

In their contribution to the PS symposium, Kenneth Meiera and Apolonia Calderona complain of IRB interference in work that is clearly exempt or not even human subjects research.

[Kenneth J. Meier and M. Apolonia Calderon, “Goal Displacement and the Protection of Human Subjects: The View from Public Administration,” PS: Political Science & Politics 49, no. 02 (April 2016): 294–98, doi:10.1017/S1049096516000238.]

U of Maryland Scapegoats IRB and Researcher for PR Foul Up

The University of Maryland has released a report on the problematic December 2015 press release, which included unsubstantiated claims about the benefits of a sports drink based on chocolate milk. While the press release was indeed a disaster, the university report fails to hold to account the people most responsible. Instead, it makes matters worse by accusing the researcher and the IRB of transgressions they did not commit, and by recommending drastic changes that are unnecessary and burdensome.

Protecting He-Man Subjects

After frustrating encounters with IRBs concerning two research projects, sociologists Liberty Walther Barnes and Christin L. Munsch argue that “IRBs are gendered institutions in which members base their decisions on culturally dominant, normative images of women and men.”

[Liberty Walther Barnes and Christin L. Munsch, “The Paradoxical Privilege of Men and Masculinity in Institutional Review Boards,” Feminist Studies 41, no. 3 (2015): 594–622, doi:10.15767/feministstudies.41.3.594.]

NPRM: Freedom for Historians, If They Can Keep It

The notice of proposed rulemaking (NPRM) promises long-sought relief for historians, journalists, and biographers. For these groups, the goal will be to ensure that the proposed rules are enacted as currently written.

[This post has been cross-posted to the Petrie-Flom Center's Bill of Health, which is conducting an online NPRM Symposium.]

11 September 2015: See update at the bottom of this post.

Rivera: Faculty Researchers Are Notoriously Poor Judges of Risks

Suzanne Rivera, Associate Vice President for Research at Case Western Reserve University and member of the Secretary's Advisory Committee on Human Research Protections, responds to the AAUP's IRB report by asserting that faculty are inept at making determinations of exemption. I question this claim.

[Rivera, Suzanne A. “Academic Freedom and Responsibility |.” Bill of Health. Accessed March 28, 2013. http://blogs.law.harvard.edu/billofhealth/2013/03/24/academic-freedom-and-responsibility/. h/t Michelle Meyer]

Bell and Salmon Warn of Dangerous Assumptions

Kirsten Bell and Amy Salmon, both of the University of British Columbia, warn that in trying to protect people they consider vulnerable, ethics committees ignore empirical evidence that some measures are counterproductive.

[Bell, Kirsten, and Amy Salmon. “Good Intentions and Dangerous Assumptions: Research Ethics Committees and Illicit Drug Use Research.” Research Ethics 8, no. 4 (December 2012): 191–199. doi:10.1177/1747016112461731.]

Study Finds IRBs Often Fail to Discuss Common Rule Criteria

The Common Rule [45 CFR 46.111(a)] requires IRBs to determine that each protocol they approve satisfies seven requirements: risks are minimized, risks are reasonable, selection of subject is equitable, informed consent is sought, informed consent is documented, safety is monitored when appropriate, and privacy and confidentiality are protected when appropriate. A team of researchers has found that "the IRB made clear determinations on all relevant criteria for 20 of the 104 (19%) protocols" they examined.

[Charles W. Lidz, Paul S. Appelbaum, Robert Arnold, Philip Candilis, William Gardner, Suzanne Myers, and Lorna Simon. "How Closely Do Institutional Review Boards Follow the Common Rule?" Academic Medicine 87 (July 2012), doi: 10.1097/ACM.0b013e3182575e2e, post author corrections version, 22 May 2012]

REC Forbids Dissertations on Lap Dancing

Robert Dingwall fears that "research ethics is co-opted to infantilize students who are legally adults but treated as if they should never be allowed to risk a bad experience."

[Robert Dingwall, "Better Drowned than Duffers…?," social science space, 19 February 2012.]

Emanuel: ANPRM Will End Reliance on Worthless Gut Reactions

I watched the webcast of this morning's appearance by Ezekiel Emanuel before the Presidential Commission for the Study of Bioethical Issues. There were no huge surprises in Emanuel's presentation; much of it repeated points he had made in the New England Journal of Medicine. But I found some of his remarks, and the Commission's response, noteworthy.

ANPRM's Problem Statement: Helpful but Incomplete

One of the many remarkable sections of the July 26 advance notice of proposed rulemaking (ANPRM) is its admission that the Common Rule is flawed.

(Note: I have added a link to the ANPRM at the top of the link list in the sidebar.)

Since the 1970s, IRB apologists have claimed that federal regulations are flexible enough, and that local IRBs are to blame for any problems. In 2007, for example, Jerry Menikoff quoted with approval Jeffrey Cohen's 2006 claim that "the regulations provide sufficient flexibility for the efficient and appropriate review of minimal risk research. IRB review of such research does not have to be burdensome or unreasonable if IRBs appropriately utilize the flexibility in the regulations." Menikoff reiterated his claim of "flexibility within the system" in his 2009 speech, “The Legal Assault on the Common Rule."

After thirty years of such claims, it is wonderfully refreshing that the ANPRM takes so seriously many of the critiques leveled at the federal regulations themselves. And the ANPRM helpfully organizes those critiques into seven general categories.

On the other hand, ANPRM's problem statement (pages 44513-44514 in the Federal Register version) overlooks some major critiques. Fortunately, some of those critiques are implicitly recognized by some of the ANPRM's proposals.

Survey: Most IRBs Lack Sociologists

The Western Massachusetts Institute for Social Research kindly alerted me to its survey of socioogists, conducted in the summer of 2009. Of the 98 respondents who have conducted research in the past five years, 90 reported that they had undergone IRB review.

The survey found that IRBs are more likely than sociologists to judge a study risky. Only 13 respondents "said that they believe that some harm could have come to respondents as a result of their involvement in the research," but 20 reported that a member of the IRB believed there was such a risk.

This is not surprising. The premise of IRB review is that committees are better able to flag potential harms than are individual researchers, so the higher levels of risk seen by the IRBs could indicate that they are working well, or that they are overestimating the risks of research.

To distinguish the two possibilities, it would help to know why the IRB members saw risk. In 1979, for example, Lauren Seiler and James Murtha showed that IRB chairs commonly insisted on modifications even though most had never heard of harm coming to a participant in sociology research. [Lauren H. Seiler and James M. Murtha, "Federal Regulation of Social Research," Freedom at Issue, Nov-Dec 1979.] Is that still the case?

Another finding of the Western Massachusetts survey is that a minority (44 percent) of respondents reported that the IRB that reviewed their research included a sociologist. Federal regulations require IRBs to include members "with varying backgrounds to promote complete and adequate review of research activities commonly conducted by the institution." This was one of the few protections offered to social scientists worried that their research would be subject to the whims of people outside their field. But it appears that many or most IRBs have failed to meet this standard.