Study Finds IRBs Often Fail to Discuss Common Rule Criteria

The Common Rule [45 CFR 46.111(a)] requires IRBs to determine that each protocol they approve satisfies seven requirements: risks are minimized, risks are reasonable, selection of subject is equitable, informed consent is sought, informed consent is documented, safety is monitored when appropriate, and privacy and confidentiality are protected when appropriate. A team of researchers has found that "the IRB made clear determinations on all relevant criteria for 20 of the 104 (19%) protocols" they examined.

[Charles W. Lidz, Paul S. Appelbaum, Robert Arnold, Philip Candilis, William Gardner, Suzanne Myers, and Lorna Simon. "How Closely Do Institutional Review Boards Follow the Common Rule?" Academic Medicine 87 (July 2012), doi: 10.1097/ACM.0b013e3182575e2e, post author corrections version, 22 May 2012]

The researchers observed and recorded the full-board deliberations of IRBs at ten academic medical centers as they discussed a total of 104 protocols. They compared their own judgment about which of the seven Common Rule criteria should have been discussed in the deliberations to their findings on whether the IRB did in fact discuss the relevant criteria.

The researchers found that IRBs were extremely likely to tinker with consent forms:

In contrast to the other criteria, which sometimes were not discussed, IRBs routinely discussed informed consent. They approved unchanged consent forms for 5 of the 104 (5%) protocols while recommending or requesting changes to 92 (88%) others. In 5 (5%) reviews, an IRB committee member criticized or suggested changes to the consent form, but no further action was taken. In the remaining 2 (2%) reviews, the IRB made no mention of informed consent.

No other criteria came close to that frequency. As the researchers explain,

We found that not all IRBs discussed all of the criteria mandated by the Common Rule. First, in over 20% of the protocol reviews of studies that posed greater than minimal risk, IRBs did not consider whether the studies' risks were appropriately addressed in the application or whether those risks could be minimized. Second, IRBs did not compare the risks versus benefits in 57% of the protocol reviews that had risks other than those to confidentiality. Third, the fact that IRBs did not consider equity in subject selection in 60% of the applications that excluded categories of subjects suggests that they are not routinely determining whether the benefits and burdens of research are distributed evenly among populations. Finally, IRBs in only about 40% of reviews of relevant protocols either approved or proposed revisions to plans to monitor data for potential safety issues, in 70% made determinations with regard to confidentiality, and in 20% did not address the protection of vulnerable populations. In summary, although we could not listen to the audio recordings of the meetings without appreciating the seriousness with which the reviewers took their responsibilities, we found that the IRBs frequently failed to discuss many of the human subjects protection criteria mandated by the Common Rule.

The authors recognize that this is not a surprising finding; pretty much everyone outside of PRIM&R understands that IRBs spend a lot of their time editing consent forms, often making them harder to read in the process.

But the study does offer useful evidence that the IRB system is not working as advertised. I suspect the IRBs in question simply lacked the time to consider thoroughly all seven criteria for each protocol they reviewed, so they dwelled on the easiest task—the consent form—rather than the more nebulous challenges. And in many cases, this may have been the correct ethical decision. In particular, IRBs only rarely (12 percent of relevant protocols) raised questions or requested changes concerning risk-benefit reasonability. For critics of IRB paternalism, this may be welcome news.

(Here, as elsewhere, I wish the team had also investigated IRBs at non-medical institutions. It would be fascinating to know if those IRBs were any more or less likely to deem a study's potential benefits too small.)

On the other hand, the IRBs may have failed in what many would consider their most important duty of minimizing risks in significantly risky protocols. The article notes that IRBs failed to discuss the risks of "a phase 1 oncology study of a new combination of approved chemotherapy agents and a placebo-controlled, phase 3 trial of a new treatment for a previously untreatable disorder, both of which posed significant risks."

Assuming we don't want to double the time and money already devoted to IRB review, one could imagine improving the system in two, complementary ways. Either reduce the number of criteria each IRB is expected to examine, or reduce the number of protocols reviewed by IRBs. The article claims that the ANPRM does not address "IRBs' failures to assess all required Common Rule criteria, that is, to perform a comprehensive protocol review." But the ANPRM's proposal to move the current mass of expedited protocols off the desks of individual IRB members might be a step toward giving them time to focus on the protocols that raise the greatest concern.