Could Retrospective Review End Nitpicking?

Robert Klitzman and Paul Appelbaum, both of Columbia University, suggest that shifting research oversight to the retrospective review of a subset of projects, rather than prospective review of proposals, could reduce the "variability and subjectivity across IRBs" that now characterizes ethics review.

[Robert Klitzman and Paul S. Appelbaum, "To Protect Human Subjects, Review What Was Done, Not Proposed," Science 335 no. 6076 (30 March 2012): 1576-1577,
DOI: 10.1126/science.1217225.]

The authors deprecate the current system. They note that "IRBs often face difficulties defining, interpreting, and applying critical concepts embodied in the regulations and central to human subjects protection (e.g., 'justice' and 'autonomy')," and therefore produce inconsistent judgments on identical proposals. Worse still, IRBs are unaware of how useless much of their effort is. "Many IRBs feel they play a vital role in rewording consent forms, because researchers may downplay risks and overemphasize potential benefits," they note. "But IRBs 'wordsmithing' the contents differently produces inconsistent results." Nor should we expect anything else from a system based almost entirely on guesswork: "the inherently speculative nature of prospective review exacerbates variability and subjectivity across IRBs."

As an alternative, the authors propose a system of retrospective review, comparable to "audits of government-funded health-care programs (Medicare and Medicaid), accreditation of hospitals, Data Safety Monitoring Boards' assessment of patient safety and treatment efficacy data during clinical trials, federal and state tax collection audits, and the tort system." (They might have added existing mechanisms to handle allegations of plagiarism, fabrication, and falsification.) Such a system, they suggest, "would allow determinations based on evidence of what actually occurred, rather than fears of what might happen."

Retrospective review would also facilitate the introduction of due process guarantees now absent in the system. "Due process is so fundamental a right in modern democracies," they write, "that there seems little reason to deny it to researchers."

Klitzman and Appelbaum are pragmatists, noting that "We cannot simply turn the clock back to 1966, when prospective review was first introduced, and cast it aside." Instead, they propose initially shifting to retrospective review only for "certain minimal risk research," as proposed by the ANPRM. "If initial moves away from a strictly prospective model are successful," they suggest, "it may be possible to expand them progressively to studies that pose higher levels of risk."

I don't think I've ever seen so much empirical evidence and careful thinking about IRBs packaged in so brief an essay. While I look forward to reading more from Dr. Klitzman, he has set himself a high standard.