Anthropologist and George Mason University alumna Maureen Fitzgerald, now affiliated with the University of Sydney, is the director of "An Analysis of Research Ethics and the Ethical Review Process as Culture and Cultural Process," an ongoing investigation more succinctly named "The Ethics Project."
Between 2001 and 2005 she and her associates observed the workings of 29 ethics committees in Australia, Canada, New Zealand, the United Kingdom, and the United States--not finding significant differences across national boundaries. They have published several papers based on this work, most of which are available on the project's website. Since the themes of many of these publications overlap, I will focus my comments on three articles that I found particularly helpful.
Wednesday, March 5, 2008
Monday, February 18, 2008
Peter Moskos on What to Tell Your IRB
Sociologist Peter Moskos is author of Cop in the Hood, a book about police work he wrote based on his fieldwork as a Baltimore police cadet and police officer. He maintains a blog concerning the issues raised in the book (a splendid marriage of old media and new), and on February 12 he commented about the ethical issues raised by Sudhir Venkatesh's Gang Leader for a Day. In that posting, "Outing the insiders," Moskos wrote,
I’ve never been a fan of the I.R.B. Few professor are. I don’t think that overt non-experimental academic researchers should need approval to observe and interact with most human subjects. We’re not giving out experimental drugs. We’re not running experiments. We’re watching and talking and living. I don’t even like the term “human subjects.” It’s dehumanizing. They're people, damnit! It’s condescending to think that adults aren’t smart enough to make their own decisions about what to say to whom. And if they’re not, well, such is life.
Nor am I convinced that research subjects who harm others deserve institutional protection. I believe academics should act under a code similar to journalists. But federal law disagrees with me. And the press has explicit constitutional protection that professors don’t.
Tuesday, February 12, 2008
AAHRPP Calls for Research on IRBs and Behavioral and Social Science
Rob Townsend kindly alerted me to the Winter 2008 issue of AAHRPP Advance, published by the Association for the Accreditation of Human Research Protection Programs, which features the lead article, "IRBs and Behavioral and Social Science Research: Finding the Middle Ground." The article concedes that "many behavioral and social scientists feel constrained by a system that seems tilted toward biomedical research and, therefore, neither understands nor reflects their concerns." And it reports the interest of Drs. J. Michael Oakes, "an author and frequent lecturer on IRB review of behavioral and social science research," and Howard Silver, Executive Director of the Consortium of Social Science Associations, in addressing some of those concerns.
Oakes and Silver "encourage researchers to investigate scientifically IRB oversight of behavioral and social science research. Such research could determine whether IRBs are consistent in applying the federal regulations, whether IRBs are taking advantage of the flexibility that’s built into the regulations, and whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research. The resulting data could shed light on ways to relieve tensions between these two groups."
That sounds good, but it's a bit disappointing that the article does not acknowledge the considerable research already completed on this topic, much of it already cited on this blog. Nor does it remark on the curiosity that IRB oversight has continued for four decades without anyone knowing if it does any good.
Moreover, the article states as fact some beliefs that should be investigated with just the sort of research it calls for. I hope Oakes, Silver, and AAHRPP will allow data to challenge some of their own presuppositions.
Here are some questions that could be answered by further research.
AAHRPP thinks it already knows the answer to this one. The article claims that "Tensions began building in the late 1990s in response to increased government scrutiny of research involving human participants," and that "the regulations have not changed. What’s new is their enforcement and, in many instances, that enforcement is overdue." I am at work on a history of IRB review of social science and humanities research, and I think more research can challenge this view.
The first part of the AAHRPP claim is doubtful; social scientists have protested IRB regulations since 1966, and tensions have waxed and waned since then. If we take this longer view, then the assertion about the immutability of the regulations is wrong; the regulations--first promulgated in 1974--changed twice, in 1981 and 1991. And the 1991 revisions greatly expanded the reach of IRBs. The 1981 regulations exempted survey, interview, and observational research unless it “deals with sensitive aspects of the subject’s own behavior, such as illegal conduct, drug use, sexual behavior, or survey or interview procedures is use of alcohol" and if "the subject’s responses, if they became known outside the research, could reasonably place the subject at risk of criminal or civil liability or be damaging to the subject’s financial standing or employability. The 1991 regulations, in contrast, eliminated the "sensitive aspects" clause and added potential harms to reputation to the list of triggers for IRB review. These changes were made over the objection of social scientists. And they set the stage for the conflict of the 1990s and today.
The claim that "what's new is [the regulations'] enforcement," is only half-true. Also new is the guidance issued by OPRR/OHRP since 1995 that reversed previous policies.
Finally, the article claims that "enforcement is overdue." Really? What errors did social scientists commit in the 1980s--a decade of relatively light regulation?
Dr. Oakes states, “IRB members are not those folks who are looking to thwart your study. They are peer researchers who have a job to do.” But clearly some IRB members are looking to thwart studies, or else studies wouldn't get thwarted as often as they do. The question is how many IRB members do this, and why.
One part of this question concerns membership. Oakes's claim that IRB members "are peer researchers" depends on an odd definition of peers. In the NIH peer review process, for example, proposals are reviewed by study sections whose members are chosen for their expertise. The NIH's Center for Scientific Review requires, among other things, that
"* Expertise is the paramount consideration when developing/updating a study section roster.
"* Each scientific area reviewed by the study section needs appropriate expert representation."
IRBs theoretically must include experts on each type of research reviewed, but Oakes knows as well as I that this requirement is often ignored. Additional research might indicate how often a researcher faces an IRB with no expertise in the methods under review.
Then, of course, some IRB members are not researchers at all, but the "one member whose primary concerns are in nonscientific areas" required by the regulations. As Laura Stark's dissertation suggests, these members can be particularly undisciplined in their meddling.
Like PRIM&R, AAHRPP thinks that IRBs review only two kinds of scholarship: biomedical research, and something called "behavioral and social science research." The article states, "AAHRPP’s Founding Members, Board of Directors, Council on Accreditation, and Supporting Members all include representatives of organizations engaged in behavioral and social science research."
This statement suggests the fallacy of the undistributed middle term:
* Ehnographers are represented by organizations engaged in behavioral and social science research.
* Organizations engaged in behavioral and social science research have a voice in AAHRPP.
* Therefore, ethnographers are represented by organizations that have a voice in AAHRPP.
The second premise is "undistributed," since it is not true that all organizations engaged in behavioral and social science research have a voice in AAHRPP.
Here's a counter example:
* Countries in South America, Africa, and South Asia are not in North America or Europe.
* Countries from parts of the world other than North America and Europe are permanent members of the UN security council and the G-8.
* Therefore, countries in South America, Africa, and South Asia are permanent members of the UN Security Council and the G-8.
In committing this fallacy, AAHRPP lumps together a dozen or more scholarly disciplines--each with its own history, methods, and ethics--into a single category: "behavioral and social science research." The AAHRPP website does not list the disciplinary affiliations of members of its Board of Directors, Council on Accreditation, or list of site visitors, but if there's a journalist, historian, or folklorist in the lot, I'll be surprised.
To take the example I know best, oral historians do not expect psychologists, social workers, or education researchers to understand or represent their interests. AAHRPP (like PRIM&R) should find out how many disciplines are now subject to review, and include representatives from all of them.
The article asks "whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research." But that is only one of several alternative systems in place on various campuses. For example, Macquarie University delegates ethical review to a number of subcommittees with special expertise in certain fields. And the University of Pennsylvania allows researchers using some social science methods to forego "a fixed research protocol." And we can imagine even more models, some of which would require redrafting present regulations, others of which might not.
I appreciate AAHRPP's call for research, and I hope it agrees that research is most valuable when the answers are not predetermined.
Oakes and Silver "encourage researchers to investigate scientifically IRB oversight of behavioral and social science research. Such research could determine whether IRBs are consistent in applying the federal regulations, whether IRBs are taking advantage of the flexibility that’s built into the regulations, and whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research. The resulting data could shed light on ways to relieve tensions between these two groups."
That sounds good, but it's a bit disappointing that the article does not acknowledge the considerable research already completed on this topic, much of it already cited on this blog. Nor does it remark on the curiosity that IRB oversight has continued for four decades without anyone knowing if it does any good.
Moreover, the article states as fact some beliefs that should be investigated with just the sort of research it calls for. I hope Oakes, Silver, and AAHRPP will allow data to challenge some of their own presuppositions.
Here are some questions that could be answered by further research.
1. When did IRB review of social science go bad, and why?
AAHRPP thinks it already knows the answer to this one. The article claims that "Tensions began building in the late 1990s in response to increased government scrutiny of research involving human participants," and that "the regulations have not changed. What’s new is their enforcement and, in many instances, that enforcement is overdue." I am at work on a history of IRB review of social science and humanities research, and I think more research can challenge this view.
The first part of the AAHRPP claim is doubtful; social scientists have protested IRB regulations since 1966, and tensions have waxed and waned since then. If we take this longer view, then the assertion about the immutability of the regulations is wrong; the regulations--first promulgated in 1974--changed twice, in 1981 and 1991. And the 1991 revisions greatly expanded the reach of IRBs. The 1981 regulations exempted survey, interview, and observational research unless it “deals with sensitive aspects of the subject’s own behavior, such as illegal conduct, drug use, sexual behavior, or survey or interview procedures is use of alcohol" and if "the subject’s responses, if they became known outside the research, could reasonably place the subject at risk of criminal or civil liability or be damaging to the subject’s financial standing or employability. The 1991 regulations, in contrast, eliminated the "sensitive aspects" clause and added potential harms to reputation to the list of triggers for IRB review. These changes were made over the objection of social scientists. And they set the stage for the conflict of the 1990s and today.
The claim that "what's new is [the regulations'] enforcement," is only half-true. Also new is the guidance issued by OPRR/OHRP since 1995 that reversed previous policies.
Finally, the article claims that "enforcement is overdue." Really? What errors did social scientists commit in the 1980s--a decade of relatively light regulation?
2. Why do IRBs sometimes delay or prohibit social science research?
Dr. Oakes states, “IRB members are not those folks who are looking to thwart your study. They are peer researchers who have a job to do.” But clearly some IRB members are looking to thwart studies, or else studies wouldn't get thwarted as often as they do. The question is how many IRB members do this, and why.
One part of this question concerns membership. Oakes's claim that IRB members "are peer researchers" depends on an odd definition of peers. In the NIH peer review process, for example, proposals are reviewed by study sections whose members are chosen for their expertise. The NIH's Center for Scientific Review requires, among other things, that
"* Expertise is the paramount consideration when developing/updating a study section roster.
"* Each scientific area reviewed by the study section needs appropriate expert representation."
IRBs theoretically must include experts on each type of research reviewed, but Oakes knows as well as I that this requirement is often ignored. Additional research might indicate how often a researcher faces an IRB with no expertise in the methods under review.
Then, of course, some IRB members are not researchers at all, but the "one member whose primary concerns are in nonscientific areas" required by the regulations. As Laura Stark's dissertation suggests, these members can be particularly undisciplined in their meddling.
3. What types of research now fall subject to IRB review?
Like PRIM&R, AAHRPP thinks that IRBs review only two kinds of scholarship: biomedical research, and something called "behavioral and social science research." The article states, "AAHRPP’s Founding Members, Board of Directors, Council on Accreditation, and Supporting Members all include representatives of organizations engaged in behavioral and social science research."
This statement suggests the fallacy of the undistributed middle term:
* Ehnographers are represented by organizations engaged in behavioral and social science research.
* Organizations engaged in behavioral and social science research have a voice in AAHRPP.
* Therefore, ethnographers are represented by organizations that have a voice in AAHRPP.
The second premise is "undistributed," since it is not true that all organizations engaged in behavioral and social science research have a voice in AAHRPP.
Here's a counter example:
* Countries in South America, Africa, and South Asia are not in North America or Europe.
* Countries from parts of the world other than North America and Europe are permanent members of the UN security council and the G-8.
* Therefore, countries in South America, Africa, and South Asia are permanent members of the UN Security Council and the G-8.
In committing this fallacy, AAHRPP lumps together a dozen or more scholarly disciplines--each with its own history, methods, and ethics--into a single category: "behavioral and social science research." The AAHRPP website does not list the disciplinary affiliations of members of its Board of Directors, Council on Accreditation, or list of site visitors, but if there's a journalist, historian, or folklorist in the lot, I'll be surprised.
To take the example I know best, oral historians do not expect psychologists, social workers, or education researchers to understand or represent their interests. AAHRPP (like PRIM&R) should find out how many disciplines are now subject to review, and include representatives from all of them.
4. What models of ethical review exist, and what models might we imagine?
The article asks "whether relationships between IRBs and social scientists are less strained on campuses that have separate IRBs to review behavioral and social science research." But that is only one of several alternative systems in place on various campuses. For example, Macquarie University delegates ethical review to a number of subcommittees with special expertise in certain fields. And the University of Pennsylvania allows researchers using some social science methods to forego "a fixed research protocol." And we can imagine even more models, some of which would require redrafting present regulations, others of which might not.
I appreciate AAHRPP's call for research, and I hope it agrees that research is most valuable when the answers are not predetermined.
Saturday, February 2, 2008
Historians Flood OHRP With Comments
In response to the October 2007 announcement in the Federal Register calling for comments about the existing guidance on expedited review, oral historians and their allies have flooded OHRP with complaints about IRB review of oral history and requests for an unambiguous exemption.
As the original announcement noted, comments sent in response to Federal Register notices are a public record. Mr. Glen Drew and Ms. Toni Goodwin of OHRP kindly sent me copies of all 65 comments on expedited review. Of these, 38 commented on oral history or folklore, with all but one of those seeking exclusion for such research.
The comments came from a wide range of scholars. University historians ranged in rank from graduate students to chaired professors. Non-university historians included those working for federal and state agencies and for private companies. Historians of science and medicine—among those most familiar with the medical research that led to the current regulatory scheme—were particularly vocal. The American Historical Association weighed in against IRB review—reversing its 1998 stance—as did the American Association for the History of Medicine, the American Folklore Society, and the Society of American Archivists. Scholars in sociology, English, psychology, medicine, and American studies also called for oral history's exclusion from IRB jurisdiction, as did one IRB chair.
These scholars' complaints about IRB review of oral history will be familiar to those who have followed this controversy. Many noted IRBs' demands that scholars submit questions to be approved in advance, a practice that outrages oral historians who pride themselves on their ability to improvise questions in response to their research and the stories they hear.
Of the 38 comments, only one did not condemn IRB jurisdiction over oral history. Yet even that comment, by Claytee White of the Oral History Research Center at UNLV, does not endorse IRB review as practiced at most universities. Rather, White notes that she spends only 15 minutes clearing each project, suggesting blanket approval for her work rather than the project-by-project review that is the focus of most complaints.
The comments acknowledge that "memory can be painful," but they also point out that "historians are professionally obliged to ask our interview partners probing questions independent of the benefit or harm for the interviewee." Because they do not expect IRBs to understand this, historians—and the IRB chair—suggest that other oral historians would be better equipped to judge oral history projects than IRBs with few or no historians as members.
The scholars do not think IRB members mean-hearted, just hopelessly unfamiliar with the practice of oral history. "University IRBs do not have the necessary background to appreciate that oral history research is different from other research involving human subjects," wrote one. Nor do they expect this to change. A historian of medicine notes that "as long as IRB members are active, overworked faculty volunteering their time, they will be unable to track the nuances of a style of research they see very rarely."
For this reason, the historians do not seek modification of the current system, but an unambiguous removal of oral history from IRB jurisdiction. At least 14 comments, including the American Historical Association's, endorsed the 2006 recommendation of the American Association of University Professors "that research on autonomous adults whose methodology consists entirely in collecting data by surveys, conducting interviews, or observing behavior in public places, be exempt from the requirement of IRB review—straightforwardly exempt, with no provisos, and no requirement of IRB approval of the exemption."
As I noted in my own comment, the present guidance was adopted in 1998 in response to six comments about oral history, one of which included the American Historical Association's endorsement of oral history's inclusion on the list of methods eligible for expedited review. Now that the AHA and 36 others have called for the wholesale exclusion of oral history from IRB review, I hope OHRP will be as responsive as its predecessor was a decade ago.
Here are excerpts from the comments on oral history in PDF form.
As the original announcement noted, comments sent in response to Federal Register notices are a public record. Mr. Glen Drew and Ms. Toni Goodwin of OHRP kindly sent me copies of all 65 comments on expedited review. Of these, 38 commented on oral history or folklore, with all but one of those seeking exclusion for such research.
The comments came from a wide range of scholars. University historians ranged in rank from graduate students to chaired professors. Non-university historians included those working for federal and state agencies and for private companies. Historians of science and medicine—among those most familiar with the medical research that led to the current regulatory scheme—were particularly vocal. The American Historical Association weighed in against IRB review—reversing its 1998 stance—as did the American Association for the History of Medicine, the American Folklore Society, and the Society of American Archivists. Scholars in sociology, English, psychology, medicine, and American studies also called for oral history's exclusion from IRB jurisdiction, as did one IRB chair.
These scholars' complaints about IRB review of oral history will be familiar to those who have followed this controversy. Many noted IRBs' demands that scholars submit questions to be approved in advance, a practice that outrages oral historians who pride themselves on their ability to improvise questions in response to their research and the stories they hear.
Of the 38 comments, only one did not condemn IRB jurisdiction over oral history. Yet even that comment, by Claytee White of the Oral History Research Center at UNLV, does not endorse IRB review as practiced at most universities. Rather, White notes that she spends only 15 minutes clearing each project, suggesting blanket approval for her work rather than the project-by-project review that is the focus of most complaints.
The comments acknowledge that "memory can be painful," but they also point out that "historians are professionally obliged to ask our interview partners probing questions independent of the benefit or harm for the interviewee." Because they do not expect IRBs to understand this, historians—and the IRB chair—suggest that other oral historians would be better equipped to judge oral history projects than IRBs with few or no historians as members.
The scholars do not think IRB members mean-hearted, just hopelessly unfamiliar with the practice of oral history. "University IRBs do not have the necessary background to appreciate that oral history research is different from other research involving human subjects," wrote one. Nor do they expect this to change. A historian of medicine notes that "as long as IRB members are active, overworked faculty volunteering their time, they will be unable to track the nuances of a style of research they see very rarely."
For this reason, the historians do not seek modification of the current system, but an unambiguous removal of oral history from IRB jurisdiction. At least 14 comments, including the American Historical Association's, endorsed the 2006 recommendation of the American Association of University Professors "that research on autonomous adults whose methodology consists entirely in collecting data by surveys, conducting interviews, or observing behavior in public places, be exempt from the requirement of IRB review—straightforwardly exempt, with no provisos, and no requirement of IRB approval of the exemption."
As I noted in my own comment, the present guidance was adopted in 1998 in response to six comments about oral history, one of which included the American Historical Association's endorsement of oral history's inclusion on the list of methods eligible for expedited review. Now that the AHA and 36 others have called for the wholesale exclusion of oral history from IRB review, I hope OHRP will be as responsive as its predecessor was a decade ago.
Here are excerpts from the comments on oral history in PDF form.
Ethics Yes, IRBs No
The blog "Law and Letters" features a posting from "Belle Lettre" entitled Venkatesh's Gang Leader For a Day and IRBs. The author asks whether the ethical problems raised by Sudhir Venkatesh's book form an argument for IRB review of ethnography.
As the comments make clear, however, Lettre has uncritically equated IRB-approved research and ethical research. IRB critics do care about research ethics, but we question whether IRB review is either necessary or sufficient to ensure adherence to scholarly ethics and the law.
As the comments make clear, however, Lettre has uncritically equated IRB-approved research and ethical research. IRB critics do care about research ethics, but we question whether IRB review is either necessary or sufficient to ensure adherence to scholarly ethics and the law.
Wednesday, January 30, 2008
Must Employees Consent?
Two recent items concerning IRBs and studies of job performance caught my eye.
The more prominent item was a December 30, 2007, New York Times op-ed, "A Lifesaving Checklist" by Dr. Atul Gawande. Gawande describes efforts by the Johns Hopkins University and the Michigan Health and Hospital Association to offer five-step checklists to intensive-care providers, to help them remember to wash their hands and take other steps to avoid infection. OHRP shut the project down, even though it had been ruled exempt by the Hopkins IRB. (So much for local control.)
Although I try to keep this blog focused on issues concerning the humanities and social sciences, and to avoid issues concerning strictly medical research, I was struck by OHRP's determination that "the subjects of the research were both the healthcare providers at the participating ICUs and their patients." Are healthcare providers really human subjects in a case like this?
A similar question arose in a non-medical setting a few weeks later. In her pseudonymous blog on Education Week, eduwonkette complained that New York City teachers were being studied without their consent and without the approval of an IRB. Since eduwonkette herself concedes that "teachers did not need to consent in this case, as they are government employees and their employers can collect whatever data they want," I don't understand why she thought an IRB should be involved. But it does suggest that the question of studying employee effectiveness is not limited to medical contexts.
Poking around in my research notes, I find that at the July 1977 meeting of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Dr. Karen Lebacqz raised just this issue as a hypothetical:
While Albert Jonsen initially suggested that the staff would constitute subjects, other commissioners disagreed:
[National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Transcript, Meeting #32, July 1977 (Bethesda: The Commission, 1977), 133-135.]
After this exchange, the commission moved onto another subject, suggesting a consensus that employees are not human subjects whose informed consent is required. In its recent determination letters on the checklist project, OHRP has ruled the opposite.
I don't expect the compliance officers at OHRP to be familiar with the commission's deliberations, and I fault the commission for not defining human subjects more carefully. But it's worth noting how far OHRP has strayed from the commission whose work nominally informs its decisions.
The more prominent item was a December 30, 2007, New York Times op-ed, "A Lifesaving Checklist" by Dr. Atul Gawande. Gawande describes efforts by the Johns Hopkins University and the Michigan Health and Hospital Association to offer five-step checklists to intensive-care providers, to help them remember to wash their hands and take other steps to avoid infection. OHRP shut the project down, even though it had been ruled exempt by the Hopkins IRB. (So much for local control.)
Although I try to keep this blog focused on issues concerning the humanities and social sciences, and to avoid issues concerning strictly medical research, I was struck by OHRP's determination that "the subjects of the research were both the healthcare providers at the participating ICUs and their patients." Are healthcare providers really human subjects in a case like this?
A similar question arose in a non-medical setting a few weeks later. In her pseudonymous blog on Education Week, eduwonkette complained that New York City teachers were being studied without their consent and without the approval of an IRB. Since eduwonkette herself concedes that "teachers did not need to consent in this case, as they are government employees and their employers can collect whatever data they want," I don't understand why she thought an IRB should be involved. But it does suggest that the question of studying employee effectiveness is not limited to medical contexts.
Poking around in my research notes, I find that at the July 1977 meeting of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Dr. Karen Lebacqz raised just this issue as a hypothetical:
"Suppose you want to do a study that involves changing the delivery of services in a mental institution ward. The patients in the institution are the ones that you want to study in terms of what the impact is on their recovery rates and so on. The staff who work on that ward will also be very directly impacted by the research. Do they constitute subjects, even though the design and the purpose of the research is not set up to study the impact on them of what you are going to do?"
While Albert Jonsen initially suggested that the staff would constitute subjects, other commissioners disagreed:
Lebacqz. “There is a very direct impact on the licensed staff.
Ryan. “Then the administrator can force it on them as a condition of employment.
Turtle. “Yes, I was going to say, that is the whole key, it is a condition of employment. Many things that, while I am a great libertarian here, lots of things I ask the people who work for me to do, I don’t ask them for their informed consent.”
[National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Transcript, Meeting #32, July 1977 (Bethesda: The Commission, 1977), 133-135.]
After this exchange, the commission moved onto another subject, suggesting a consensus that employees are not human subjects whose informed consent is required. In its recent determination letters on the checklist project, OHRP has ruled the opposite.
I don't expect the compliance officers at OHRP to be familiar with the commission's deliberations, and I fault the commission for not defining human subjects more carefully. But it's worth noting how far OHRP has strayed from the commission whose work nominally informs its decisions.
Monday, January 21, 2008
How IRBs Decide--Badly: A Comment on Laura Stark's "Morality in Science"
Laura Stark's recent essay in Law & Society Review led me to her 2006 Princeton University dissertation, "Morality in Science: How Research Is Evaluated in the Age of Human Subjects Regulation." The heart of the dissertation is her description of the workings of three university IRBs--one in a medical school and two at universities without medical schools--based on recordings of their meetings and her direct observation of the IRBs at work. It makes for fascinating reading, and I applaud Stark for her achievement even as I disagree with her conclusions.
Stark claims to be neutral about IRBs' ability to perform their stated task: protecting the subjects of research. She writes, "My goal is not to judge the 'fairness' and 'effectiveness' of IRBs myself." (7) And she correctly notes that the ethical acceptability of an IRB-approved project is a "social truth," not an empirical one. (244) But her tone is generally sympathetic to the IRBs. For example, she writes that IRBs' "forms of evaluation provide directed, tangible ways for board members to carrying out their review process, given the practical difficulty of applying unmediated, abstract ethics principles," making the IRB members sound like heroes who have achieved a workable system against the odds. (186)
Indeed, in some cases she reports, IRBs seem to be doing some good. For example, a physiologist and a nurse had a fruitful debate about the need for a quick medical screening of subjects in an exercise study (197-200). That's an example of an IRB with multiple experts on a single type of research--something I hope is reasonably common in much biomedical research. But most of Stark's observations are distressing in ways I don't think she appreciates. Here, then, are some of the actions she observed, along with my reasons for finding in them an indictment of the IRB system as presently run.
Stark claims to be neutral about IRBs' ability to perform their stated task: protecting the subjects of research. She writes, "My goal is not to judge the 'fairness' and 'effectiveness' of IRBs myself." (7) And she correctly notes that the ethical acceptability of an IRB-approved project is a "social truth," not an empirical one. (244) But her tone is generally sympathetic to the IRBs. For example, she writes that IRBs' "forms of evaluation provide directed, tangible ways for board members to carrying out their review process, given the practical difficulty of applying unmediated, abstract ethics principles," making the IRB members sound like heroes who have achieved a workable system against the odds. (186)
Indeed, in some cases she reports, IRBs seem to be doing some good. For example, a physiologist and a nurse had a fruitful debate about the need for a quick medical screening of subjects in an exercise study (197-200). That's an example of an IRB with multiple experts on a single type of research--something I hope is reasonably common in much biomedical research. But most of Stark's observations are distressing in ways I don't think she appreciates. Here, then, are some of the actions she observed, along with my reasons for finding in them an indictment of the IRB system as presently run.
Sunday, January 13, 2008
Johns Hopkins Dean: Generalizable Means "Hypothesis-Based"
In his comments on Inside Higher Ed's recent story on oral history and IRBs, Eaton Lattman, Dean of Research at Johns Hopkins Krieger School of Arts and Sciences, writes that "many oral history projects probably do not meet the OHRP definition of research, and are therefore free from the need to go to an IRB for approval. They are not hypothesis-based projects and they do not produce generalizable results."
This interpretation of generalizability is consistent, as best I can tell, with the intent of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, which first established generalizability as part of the definition of human subjects research. It also resembles Columbia University's recent statement on oral history.
Dean Lattman's comment contrasts with definitions put forward by Michael Carome of OHRP and consultant Jeffrey Cohen, who maintain an oral history project is generalizable research if it lacks a hypothesis but draws conclusions, informs policy, or creates an archive for use by future researchers.
Thus, Hopkins joins Columbia and Northwestern in explicitly freeing oral history--as practiced by oral historians--from IRB review. While such advances are welcome, more welcome still would be a statement along the same lines from OHRP itself.
This interpretation of generalizability is consistent, as best I can tell, with the intent of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, which first established generalizability as part of the definition of human subjects research. It also resembles Columbia University's recent statement on oral history.
Dean Lattman's comment contrasts with definitions put forward by Michael Carome of OHRP and consultant Jeffrey Cohen, who maintain an oral history project is generalizable research if it lacks a hypothesis but draws conclusions, informs policy, or creates an archive for use by future researchers.
Thus, Hopkins joins Columbia and Northwestern in explicitly freeing oral history--as practiced by oral historians--from IRB review. While such advances are welcome, more welcome still would be a statement along the same lines from OHRP itself.
Sunday, January 6, 2008
How State IRB Laws Threaten the Social Sciences: A Comment on Shamoo and Schwartz
Dr. Adil E. Shamoo of the University of Maryland School of Medicine kindly sent me a copy of his new article, "Universal and Uniform Protections of Human Subjects in Research," The American Journal of Bioethics 7 (December 2007): 7-9, co-authored with Jack Schwartz of the Maryland Attorney General’s Office.
The article calls for a federal law "to require that all human subject research in the United States, regardless of funding source or relationship to FDA marketing approval, be undertaken only after IRB review and with the informed consent of subjects." Barring that, it applauds state laws to that effect, such as the one passed in Maryland in 2002, which states that "a person may not conduct research using a human subject unless the person conducts the research in accordance with the federal regulations on the protection of human subjects . . . notwithstanding any provision in the federal regulations on the protection of human subjects that limits the applicability of the federal regulations to certain research . . . ." (13 Maryland Code §13-2002)
Having read the current New Yorker's story on "Guinea-Pigging," which states that 70 percent of drug trials now take place in the private sector, I can see the reason to regulate biomedical research through the state's police power, rather than through funding restrictions. But aside from a quick mention that research sponsored by the National Endowment for the Humanities is not subject to the Common Rule, the article does not mention the law's effect on the humanities and social sciences. This is a bit surprising, given Shamoo's earlier argument in the Chronicle of Higher Education that human subjects regulations "have handicapped researchers whose work poses no threat to humans." This being the case, why would he want to extend those same regulations to countless researchers not previously covered by them? And do the authors expect all journalists and oral historians in the state to be subject to IRB review, the way that many university-affiliated journalists and oral historians are now?
I wrote to both authors, and here is what I learned from them:
* Shamoo does not believe that the law requires Maryland journalists to seek IRB approval of their work.
* Schwartz does not believe that the law requires Maryland oral historians to seek IRB approval of their work, relying on a 22 November 2005 message from Michael Carome of OHRP.
* According to Schwartz, "The Maryland law, like every bill passed by the Maryland Legislature, was reviewed for constitutionality. This overall review did not address whether some hypothetical application of the law, under a specific set of facts, might raise First Amendment or other constitutional concerns."
* According to Schwartz, the Maryland attorney general's office has not brought any enforcement actions since the law's passage in 2002.
This last point is the key; a law that lies dormant for five years is unlikely to have any effect on anybody, so perhaps Shamoo and Schwartz have reason to think it will not handicap researchers more than they already are handicapped by existing federal regulation. Indeed, the legions of journalists, book authors, and market researchers who conduct interview and survey research outside of universities may have little to fear from a law that has not been enforced and might well fail constitutional scrutiny if it were.
But Shamoo and Schwartz ignore the potential effect of the law on social science and humanities researchers within universities, the very ones for whom Shamoo earlier expressed concern. These researchers have led a growing movement to get their universities to agree to review only research funded by the federal government. Such a move would free few, if any, university-affiliated biomedical researchers from oversight, since their expensive research is generally federally funded. Rather, it would free up the very kind of research that Shamoo considers too low-risk to merit review. But if the state imposes the same regulations regardless of the source of funding, university IRBs can close this door.
Thus, if the law has any effect at all, it will not be what Shamoo and Schwartz claim: "universal application of the ethical standards applicable to human subjects research." Instead, it will be the continued regulation of research by university-affiliated researchers while non-affiliated researchers conducting the same activities work in freedom. The result will not be uniformity, but entrenched disparity.
Shamoo's heart is in the right place. As he wrote in his Chronicle piece, "I have long advocated the creation of universal rules for all human-subject research, whether or not it receives federal funds. But an equally urgent reform is to exempt from present and future regulation any research that poses little or no risk to human subjects." Unfortunately, the state of Maryland has not treated exemption of social research as "equally urgent" to the universalization of rules. Thus, the effect of the law, if any, is to eliminate a path to the very type of exemption that Shamoo champions. Once again, regulators write rules in response to serious concerns about medical experimentation, with little or no attention to the social sciences and humanities.
The article calls for a federal law "to require that all human subject research in the United States, regardless of funding source or relationship to FDA marketing approval, be undertaken only after IRB review and with the informed consent of subjects." Barring that, it applauds state laws to that effect, such as the one passed in Maryland in 2002, which states that "a person may not conduct research using a human subject unless the person conducts the research in accordance with the federal regulations on the protection of human subjects . . . notwithstanding any provision in the federal regulations on the protection of human subjects that limits the applicability of the federal regulations to certain research . . . ." (13 Maryland Code §13-2002)
Having read the current New Yorker's story on "Guinea-Pigging," which states that 70 percent of drug trials now take place in the private sector, I can see the reason to regulate biomedical research through the state's police power, rather than through funding restrictions. But aside from a quick mention that research sponsored by the National Endowment for the Humanities is not subject to the Common Rule, the article does not mention the law's effect on the humanities and social sciences. This is a bit surprising, given Shamoo's earlier argument in the Chronicle of Higher Education that human subjects regulations "have handicapped researchers whose work poses no threat to humans." This being the case, why would he want to extend those same regulations to countless researchers not previously covered by them? And do the authors expect all journalists and oral historians in the state to be subject to IRB review, the way that many university-affiliated journalists and oral historians are now?
I wrote to both authors, and here is what I learned from them:
* Shamoo does not believe that the law requires Maryland journalists to seek IRB approval of their work.
* Schwartz does not believe that the law requires Maryland oral historians to seek IRB approval of their work, relying on a 22 November 2005 message from Michael Carome of OHRP.
* According to Schwartz, "The Maryland law, like every bill passed by the Maryland Legislature, was reviewed for constitutionality. This overall review did not address whether some hypothetical application of the law, under a specific set of facts, might raise First Amendment or other constitutional concerns."
* According to Schwartz, the Maryland attorney general's office has not brought any enforcement actions since the law's passage in 2002.
This last point is the key; a law that lies dormant for five years is unlikely to have any effect on anybody, so perhaps Shamoo and Schwartz have reason to think it will not handicap researchers more than they already are handicapped by existing federal regulation. Indeed, the legions of journalists, book authors, and market researchers who conduct interview and survey research outside of universities may have little to fear from a law that has not been enforced and might well fail constitutional scrutiny if it were.
But Shamoo and Schwartz ignore the potential effect of the law on social science and humanities researchers within universities, the very ones for whom Shamoo earlier expressed concern. These researchers have led a growing movement to get their universities to agree to review only research funded by the federal government. Such a move would free few, if any, university-affiliated biomedical researchers from oversight, since their expensive research is generally federally funded. Rather, it would free up the very kind of research that Shamoo considers too low-risk to merit review. But if the state imposes the same regulations regardless of the source of funding, university IRBs can close this door.
Thus, if the law has any effect at all, it will not be what Shamoo and Schwartz claim: "universal application of the ethical standards applicable to human subjects research." Instead, it will be the continued regulation of research by university-affiliated researchers while non-affiliated researchers conducting the same activities work in freedom. The result will not be uniformity, but entrenched disparity.
Shamoo's heart is in the right place. As he wrote in his Chronicle piece, "I have long advocated the creation of universal rules for all human-subject research, whether or not it receives federal funds. But an equally urgent reform is to exempt from present and future regulation any research that poses little or no risk to human subjects." Unfortunately, the state of Maryland has not treated exemption of social research as "equally urgent" to the universalization of rules. Thus, the effect of the law, if any, is to eliminate a path to the very type of exemption that Shamoo champions. Once again, regulators write rules in response to serious concerns about medical experimentation, with little or no attention to the social sciences and humanities.
Friday, January 4, 2008
Inside Higher Ed Reports on Comments to OHRP
Scott Jaschik's January 3 Inside Higher Ed story, "Threat Seen to Oral History," reports on the comments submitted to OHRP by the AHA and by me. The story is a helpful summary of key issues, and the comments following it indicate the passion this issue arouses in scholars in a range of fields.
Subscribe to:
Posts (Atom)
Posts
