Wednesday, September 10, 2008

Draft Comments on Training Requirements

Back in July I reported that OHRP was seeking comments on its requirements for human subjects training for investigators and IRB members. The deadline for comments is September 29.

Here is a draft of of my comments. I would appreciate comments on the comments prior to the deadline.


Dear Dr. Carome,

Thank you for the Request for Information and Comments on the Implementation of Human Subjects Protection Training and Education Programs, published in the Federal Register on July 1. I would like to offer some brief comments on this issue.

In tracing the debate over IRB review of the humanities and social sciences as it developed over the past forty years, I have yet to come across anyone who suggests that scholars should conduct research without first receiving training of some sort. The whole purpose of a university is to teach researchers to form their inquiries along lines that will produce the best results, in ethics as well as knowledge. When it comes to human subjects protections, the question is what form of training will produce those results. So far, two general models have been proposed, and I would like to offer a third.

One model demands that every university researcher, regardless of her scholarly discipline or her subject of study, complete a basic, online course in medical ethics and regulatory compliance. The CITI Program, founded by a medical researcher and a medical research administrator, exemplifies this approach. The CITI Program has the great virtue of administrative convenience. A university research office can, in a single memo, declare that all investigators must complete the program, and it can easily monitor that they have done so. But it is not clear that such mandates serve the cause of ethics, particularly when researchers are not conducting medical or psychological experiments. While the program makes efforts at including non-biomedical perspectives, the sections on such disciplines as oral history and anthropology are written by people with no expertise in those fields. The result is that much of the material in those sections is irrelevant, inaccurate, or highly dubious in its interpretations. Such programs also reduce complex ethical problems to simplistic statements to be chosen on a multiple-choice test. While I cannot offer published citations or hard data, I know anecdotally that the requirement to complete such training breeds contempt for the whole review process in many researchers.

In 2002, the Social and Behavioral Sciences Working Group on Human Research Protections pioneered an alternative approach. Rather than preparing the same curriculum for all fields, it devised reading lists specific to each discipline. For example, materials prepared for the American Sociological Association included that association's code of ethics and essays written by sociologists (see Scholars asked to complete such training are likely to take it far more seriously than a program whose medical origins cannot be disguised. On the other hand, devising a single training regime for an entire discipline will still subject some researchers to a great deal of irrelevant material. For example, an ethnographer may not need nuanced instructions on forming survey questions, nor a survey researcher instructions about participant observation.

Finally, I would like to propose a third model that goes beyond the Working Group's approach. When scholars describe the training that most influenced their ethical decisions, they are less likely to cite general codes and principles than the work of other researchers who faced very similar challenges. Criminologist Michael Rowe put this very well in his essay, "Tripping Over Molehills: Ethics and the Ethnography of Police Work," International Journal of Social Research Methodology 10 (February 2007): 37-48. Rowe wrote,

It is the nature of ethnographic research that the principles contained in methodological textbooks or professional codes of conduct will be stretched and perhaps distorted as they are applied in dynamic situations. Since policing is unpredictable, the ethical dilemmas police researchers might face cannot be easily anticipated . . . If an absolute code of ethics is not feasible, researchers must be prepared to be reflexive in terms of ethical dilemmas and the methodological difficulties experienced in securing informed consent and meaningful access to research subjects. (48)

The best preparation for observing police work, he explains, is reading other accounts of observing police work. I believe this emphasis on specificity would hold true for most qualitative research (and a great deal of quantitative work as well).

I would suggest, then, that rather than impose universal, top-down training like the CITI Program, or even more specific top-down training like the Working Group curricula, OHRP empower researchers to devise their own ethical reading lists of materials most relevant to their work, just as they choose their own methodological models. A researcher seeking IRB certification could present an annotated bibliography, showing that he had investigated the problems he was most likely to encounter and the ways that other scholars had dealt with those problems. Researchers should also be able to use courses and seminars they have completed as evidence of their prepraration.

I assume the goal of any training requirement would be to get researchers to think seriously about the ethical problems they will face. Asking them to research those problems themselves will be far more effective than any multiple-choice test.


Melissa Schlenker said...

Dear Mr. Schrag,

Your response to the OHRP request for public comment in regard to training requirements for investigators is well researched and eloquently drafted. I must ask if you have drafted your response from an academic perspective based on the review of the literature or if you speak from personal Human Research Protection Program (HRPP) training experience?

As an individual who has done both, I must share with you, that in my experience in a non-University affiliated community teaching hospital with seven active residency programs requiring scholarly activity, your proposed solution would never occur. Residents are limited to 80 duty hours and a 60 hour limit is under consideration. There is just not enough time left in their schedule to provide HRPP training. The residency program directors just don’t allow it. Another problem we experience is the first time principal investigator who is interested in research or mandated to conduct research that does not know where to begin but has a sponsor ready to send them a regulatory binder the next day. Another is that the IRB members find it difficult to review all the research for the meeting. How would they identify the current literature and then find time to actually read it?

In my opinion, your proposal grants the investigators and IRB members the right to self determine what is important and what is not. This lack of knowledge and understanding of the Federal regulations and OHRP guidance has caused the shut down of some prominent academic institutions.

My recommendations are that a certification process should exist for investigators and IRB members. Most study coordinators are (or should be) certified by professional societies such as ACRP and SoCRA. Why shouldn’t the investigators and IRB members have the same requirements? Research is all about determining gold standards and comparing novel ideas to them. A certification process would set the standard and provide the means of measuring the knowledge of the applicant. Recommended curricula could be provided in preparation of the certification and a requirement for continuing education would ensure that the individual would stay current with the regulations and literature.

These are just my thoughts on the topic. I find your proposal to be thoughtful but not practical in some environments. I do believe that this is a topic in which there will be no right answer for every situation.

Melissa Schlenker

Zachary M. Schrag said...

Thank you for your comment.

To answer your first question, I was required to complete online ethics courses at Columbia University and George Mason University, which uses the CITI Program. Out of curiosity, I also completed the course at UCLA, and looked at courses offered by other universities. While the UCLA course was better than the others, I found all of them heavy with information that was irrelevant to much social science work and, in some cases, inaccurate or misleading.

I am troubled by your description of ethical training at your hospital. As I understand you, first-time investigators cram their ethical study into a short time, perhaps the night before they receive a grant. And IRB members can't find the time to learn about the research whose ethical merit they are judging. If these researchers and reviewers are looking for a quick way to get their ticket stamped, then the CITI Program is just the thing. But you don't claim that such training leads to serious consideration of the ethical challenges posed by each project. Indeed, your proposals for a certification process, recommended curricula, and continuing education suggest that you would like to see both researchers and reviewers spend more time studying.

That leads to a second question--what should they learn? You write that "research is all about determining gold standards and comparing novel ideas to them." That's fine, so long as the emphasis is on the plural in standards. I believe that the gold standard for ethical training in genetic therapy is rather different from that in the ethnography of underground economies. And where are novel ideas to come from, if not the researchers who have invested the most in a given scholarly problem?

Zachary Schrag