Friday, August 5, 2011

ANPRM: It's Time to Redefine Research

In an earlier posting, I warned that the ANPRM's proposed "Excused" category could expand rather than diminish IRB interference with research in the social sciences and humanities.

Another option mentioned in the ANPRM--a redefinition of the scope of the Common Rule--could do more to achieve the ANPRM's goals of "better protect[ing] human subjects who are involved in research, while facilitating valuable research and reducing burden, delay, and ambiguity for investigators." It could even the regulations with the statutes they claim as their basis.

I see several options here.

Problems with the Current Definition


The current Common Rule "applies to all research involving human subjects conducted, supported or otherwise subject to regulation by any federal department or agency which takes appropriate administrative action to make the policy applicable to such research."

It defines "research" as

a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for other purposes. For example, some demonstration and service programs may include research activities.

and "human subject" as

a living individual about whom an investigator (whether professional or student) conducting research obtains
(1) Data through intervention or interaction with the individual, or
(2) Identifiable private information.

The ANPRM offers the first chance in 30 years to rewrite these definitions. It does so with its Question 24 (about the application to "quality improvement, public health activities, and program evaluation studies") and the thrilling Question 25:

Are there certain fields of study whose usual methods of inquiry were not intended to or should not be covered by the Common Rule (such as classics, history, languages, literature, and journalism) because they do not create generalizable knowledge and may be more appropriately covered by ethical codes that differ from the ethical principles embodied in the Common Rule? If so, what are those fields, and how should those methods of inquiry be identified? Should the Common Rule be revised to explicitly state that those activities are not subject to its requirements?

How should scholars in the social sciences, humanities, and journalism respond? Let me offer several alternative definitions of research that could work better than the one now in place. I do so with the caveat that I think any defintion with need interpretation. The TCPS offers a model of what this would look like: each "article" is followed by a longer section explaining its "application." I hope that with the revision of the Common Rule will come some mechanism by which the federal government can offer informed interpretations as questions arise.

Option 1: Deregulate "Two People Talking"


A longstanding complaint of researchers has been that the current regulations require official permission for two adults to talk to each other, a basic human activity that should not be regulated in a free society. As the Illinois White Paper puts it:
Considerations of what is research, when inquiry becomes research, and who we are trying to protect bear especially on certain areas of academic effort. There are many cases where two people are talking with each other: in journalism, oral history, anecdotal research, and so on. When the participants are capable adults and understand that their words are being recorded, why should the special obligations of the researcher not end with honesty and truthfulness? Many such conversations cannot even be categorized as an interview in any direct or intentional sense—or it may not be clear who is primarily interrogating whom (maybe both parties intend to write about the encounter!).

In 1979, twelve scholarly and educational organizations suggested addressing such encounters with the following formula:

These regulations do not apply to research using legally competent subjects that involves neither deceit nor intrusion upon the subject’s person nor denial or withholding of accustomed or necessary resources.

In 2006, the AAUP offered similar proposal:
Research on autonomous adults whose methodology consists entirely in collecting data by surveys, conducting interviews, or observing behavior in public places, [shall] be exempt from the requirement of IRB review—straightforwardly exempt, with no provisos, and no requirement of IRB approval of the exemption.

Either one of these would achieve the ANPRM’s goal of “facilitating valuable research and reducing burden, delay, and ambiguity for investigators.” There should also be a provision for community research that might involve children or adults who are not legally competent. For example, an anthropologist living in a community should not trigger IRB review simply because he or she occasionally speaks to a child.

These proposals would not require rewriting the definitions section of the regulations. Rather, they would constitute a bifurcation of the ANPRM's prosed category of "Excused" research.

The ANPRM's proposes that Excused research would have to strip HIPAA identifiers, employ data security, and remain subject to audit.

An alternative would be to leave those requirements only for interventional research with competent adults--in ANPRM's terms, "methodologies which are very familiar to people in everyday life and in which verbal or similar responses would be the research data being collected. For example, a researcher might ask subjects to watch a video, or read a paragraph or solve puzzles, and then ask them some questions to elicit word associations or time performance of activities."

Mere interactions--especially conversations--would not require even registration. That is, the regulations could recognize these interactions as research, but determine that they are not subject to any review, before or after the research takes place.

One trick here would be to distinguish psychological inventories from other forms of survey research, but I think that could be done. For example, HIPAA refers to "health information," so one could require registration, expedited, or full review for studies that involve questions primarily designed to learn about participants' physical or mental health.

In this case, oral history, journalism, and the like could be categorized as human subjects research, but that categorization would not come with any regulatory restrictions.

Option 2: Define Research to Match the National Commission's Intentions


As the ANPRM notes, the current Common Rule is based on the work of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. So one possibility would be to rewrite the regulations to govern only those fields that the National Commission wanted to regulate.

It's hard to say just what those fields are, since the National Commission never listed the kinds of information gathering it did and did not want IRBs to review. But we can gather hints from its deliberations and publications, especially its 1978 Report and Recommendations: Institutional Review Boards.

At its June 1975 meeting, chairman Kenneth Ryan noted "the general Commission feeling that we are talking about biomedical, social, and behavioral research." And the 1978 IRB report makes glancing reference to both anthropology and survey research:

The requirement for documentation may place an undue burden on the research while adding little protection to the subjects. Such burdens might include a negative impact on the validity of a survey sample or introduction of an element that is incongruent with the social relationships involved in the research (e.g., in anthropological research). For research that would be burdened by a requirement of written documentation of consent, such documentation may be waived, provided that the research presents no more than minimal risk of harm to subjects and involves no procedures for which written consent is normally required outside of the research context. (For example, a physical intrusion into the body may generally require written consent, whether or not the intrusion is performed for purposes of research.) In many cases (e.g. , a survey using mailed questionnaires) it would be appropriate for the investigator to provide subjects with a written statement regarding the research, but not to request their signature. In other cases (e.g. , a telephone survey) an oral explanation might be sufficient,
because subjects can readily terminate their involvement in the research. (28-29)

and

Other situations in which informed consent might not be necessary arise in field research in the social sciences. Sometimes in such research, purely observational methods are supplemented by interaction with the persons being studied and therefore come within the Commission's definition of research involving human subjects. An IRB may waive the informed consent requirement in such research when it finds a number of factors to be present. The behavior to be studied must in some sense be public, e.g., responses of businesses or institutions to members of the public, or social behavior in public places. (30)

In its discussion of existing federal policies, the 1978 report notes that some agencies engaged in "survey research entailing no intervention in the lives or activities of the subjects. . . Although their activities fall within the Commission’s definition of research with human subjects, it should be noted that data gathering, in and of itself, has not universally been considered 'research with human subjects.'" (This is from p. 56196 of the Federal Register version of the report; the passage is cut off in the PDF of the original report.)

Finally, the Commission's IRB report complained that "when federal agencies conduct or support social experimentation, they may not consider it necessary to apply procedures for the protection of human subjects." (100) (That's pretty ironic, given that the Belmont Report claims that "Because the problems related to social experimentation may differ substantially from those of biomedical and behavioral research, the Commission specifically declines to make any policy determination regarding such research at this time.")

All of this is quite confusing, but I think we can say that the Commission hoped that regulations would cover:

  • "anthropological research"
  • "a survey using mailed questionnaires"
  • "a telephone survey"
  • "responses of businesses or institutions to members of the public, or social behavior in public places"

I think we can also say that the Commission did not wish to regulate journalism, though that's murkier still. (See Ethical Imperialism, page 62.)

And we have no evidence that the Commission sought to regulate history, folklore, or the humanities in general.

These latter fields--journalism and the humanities--were rarely if ever subject to IRB jurisdiction before 1994. And it is these fields have put up the greatest resistance to the expansion of IRB jurisdiction since then. By contrast, sociologists and anthropologists have proven more ambivalent about the process. They might prefer to have their work remain defined as human subjects research, though under an "Excused" category that would reduce the burdens of that label.

Thus, we can imagine drawing a line between the social sciences on the one hand and journalism and the humanities on the other. This would honor both the intentions of the National Commission and the general sentiments of researchers. I see at least two ways to do this.

Option 2a: Deregulate the Humanities and Journalism as Non-Generalizable


ANPRM's Question 25 is clearly targeted to historians, humanists, and journalists. (How did classics get in there?) It hints that regulators are considering letting these folks off the hook by declaring their work to be non-generalizable, and therefore not subject to regulation under the Common Rule.

The advantage of this approach is that it has something of a track record. The idea that "journalism, history, biography, philosophy" generally fall outside of the regulatory definition of research because they are not generalizable dates back at least to the 1999 report of the Human Subjects Research Subcommittee, Committee on Science, National Science and Technology Council, which argued that

a key aspect of research is that there be a systematic design in advance, generally utilizing a scientific approach or protocol, for the definite purpose of contributing to generalizable knowledge. Research can include a wide variety of activities including: experiments, observational studies, surveys, tests, and recordings designed to contribute to generalizable knowledge. It generally does not include such operational activities as: medical care, quality assurance, quality improvement, certain aspects of public health practice such as routine outbreak investigations and disease monitoring, program evaluation, fiscal or program audits, journalism, history, biography, philosophy, "fact-finding" inquiries such as criminal, civil and congressional investigations, intelligence gathering, and simple data collection or data collection for other purposes. However, some of these activities may include or constitute research in the specific circumstance where there is clear advance intent to contribute to generalizable knowledge with a formal scientific protocol.

In 2003, OHRP ruled that "most oral history interviewing projects are not subject to the requirements" of the regulation "primarily on the grounds that oral history interviews, in general, are not designed to contribute to 'generalizable knowledge.'" Most of the university policies excluding oral history from IRB jurisdiction do so on the grounds that it is not generalizable.

The ANPRM seems to suggest that the 1999 position might be "explicitly" written into the Common Rule, thus locking it in place and giving historians and narrators the freedom to converse without seeking official permission. For example, the Common Rule could be amended to read

For purposes of this policy, research means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for other purposes. For example, some demonstration and service programs may include research activities. This policy does not cover such operational activities as: medical care, quality assurance, quality improvement, certain aspects of public health practice such as routine outbreak investigations and disease monitoring, program evaluation, fiscal or program audits, journalism, history, biography, philosophy, "fact-finding" inquiries such as criminal, civil and congressional investigations, intelligence gathering, and simple data collection or data collection for other purposes.

If this were to happen, it could be a boon for scholars, research participants, and administrators alike. So the obvious response would be for historians and others to reply, "Why, yes, now that you mention it, we do not create generalizable knowledge!"

Yet this would leave problems.

For one thing, non-generalizability has proven an unreliable tool. OHRP has muddied the waters, apparently contending--for example--that "Open ended interviews are conducted with surviving Negro League Baseball players in order to create an archive for future research" would constitute generalizable research because "the creation of such an archive would constitute research under 45 CFR part 46 since the intent is to collect data for future research."

Well, you can't believe that and believe that "oral history interviews, in general, are not designed to contribute to 'generalizable knowledge." If generalizable means that some future researcher might conceivably use the information, then nothing is non-generalizable. Do not daily newspapers; criminal, civil and congressional investigations; and disease monitoring all create an archive for future research?

Moreover, as Rob Townsend noted on AHA Today,

The argument [that oral history is not generalizable] prompted some derision from outside the field, from academics who interpreted the phrase to say simply “history is not research.” (As a case in point, the vice president for research at my own university, after a fairly contentious meeting on the subject, wished me well on my “non-research dissertation.”)

We also received a number of complaints from within the discipline. Some historians argue that history does contribute generalizable knowledge, even if it bears little resemblance to the scientific definition of the word. And faculty members at history of medicine departments and in the social science side of history warned that this position undermined both their institutional standing and their ability to obtain grants. They made it clear that however finely worded, stating that history did not constitute research in even the most bureaucratic terms could have some real financial costs to the discipline.

More fundamentally, no one can be sure what generalizable means. It is left undefined in the Common Rule. The Belmont Report version is longer, but hardly more helpful:

The term ”research" designates an activity designed to test an hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge (expressed, for example, in theories, principles, and statements of relationships). Research is usually described in a formal protocol that sets forth an objective and a set of procedures designed to reach that objective.

This goes some way to distinguish research from diagnosis of an individual patient--the main goal of that section of the Belmont Report--but I am not even sure of that; I would hope that an MRI operator diagnosing a patient has an objective and a set of procedures designed to reach that objective.

Nor does it distinguish science from journalism, which regularly permits conclusions to be drawn and expresses statements of relationships. Thus, a debt-ceiling story in this week's Washington Post finds that "It was the new Washington that started this fight . . . Then, over three fast-moving days, the old Washington swooped in to save them." That's a conclusion, and a statement of a relationship.

Conversely, qualitative social scientists debate whether their work is generalizable. So "generalizable" covers research that the National Commission did not want covered and leaves uncovered research that the Commission did seek to regulate.

Serious observers have noted the problem. Tom Beauchamp recently complained that "generalizable knowledge," like other terms, "can be understood in several ways." Rena Lederman has found that "the regulatory definition did little to resolve the very ambiguities within medical practice for which it was designed. Heroic efforts of clarification can be found in works that interpret the Common Rule for IRBs. Nevertheless, to this day it continues to be a frequent topic of debate in IRB circles." And in 2008, David Strauss of SACHRP complained that "we shouldn't be reviewing research that we don't think needs to be reviewed because some folks 30 years ago, at the end of a long, hot day, decided to use the word 'generalizable' . . . We have to have language that makes sense to us."

So, can we find sensible language to draw a distinction between the social sciences and humanities?

Option 2b: Deregulate the Humanities and Journalism as Research about Individuals


Perhaps the answer lies in language used by Michael Carome in a 2003 message: "OHRP has noted that its position regarding oral history was not based upon the fact that oral history activities involve open-ended, qualitative interviews of a nonrandom sample of individuals." Could "sample" be the key?

Obviously, quantitative researchers use samples. And one could make the case that qualitative sociologists and anthropologists do as well. That is, when they write about people whose names they conceal, are they not presenting them as representatives of a broader group?

Oral historians, by contrast, research individuals as individuals. As the Columbia University policy explains, "the scholarly oral history interview is rooted in particular recollections about history based on the individual perspective of the narrator." The "individual perspective" works for journalism and biography as well.

So perhaps we could advocate this definition:

For purposes of this policy, research means a systematic investigation, including research development, testing and evaluation, that studies people as representatives of the species as a whole or of biological and social groups. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for other purposes. For example, some demonstration and service programs may include research activities. This policy does not cover research about individuals' unique experiences and perspectives, such as journalism, history, biography, philosophy, "fact-finding" inquiries such as criminal, civil and congressional investigations, and intelligence gathering.

Option 3: Define Research to Match Congress's Intentions


The National Research Act of 1974 (P.L. 93-348), established the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research and instructed it to

(i) conduct a comprehensive investigation and study to identify the basic ethical principles which should underlie the conduct of biomedical and behavioral research involving human subjects (ii) develop guidelines which should be followed in such research to assure that it is conducted in accordance with such principles, and (iii) make recommendations to the Secretary [of Health, Education, and Welfare] (I) for such administrative action as may be appropriate to apply such guidelines to biomedical and behavioral research conducted or supported under programs administered by the Secretary, and (II) concerning any other matter pertaining to the protection of human subjects of biomedical and behavioral research (emphasis added).

So when Kenneth Ryan observed "the general Commission feeling that we are talking about biomedical, social, and behavioral research," he could just as well have said, "the general Commission feeling is that we should exceed the authority granted us by Congress." For Congress had not empowered the Commission--or the Secretary of HEW--to study or regulate social research.

Today's Common Rule claims the National Research Act (encoded as 42 USC 289) as its statutory authority, but it does not limit itself to the scope of the law.

What would the Common Rule look like if it were grounded in the statute?

Option 3a: Limit the Regulations to the Concerns of Congress


One possibility was mooted in a formal interpretation by the Secretary of HEW in 1976:
The types of risk situations against which the regulations were designed to protect are suggested by the areas of concern which were addressed in the legislative hearings held in conjunction with the enactment of section 474 of the Public Health Service Act, 42 U.S.C. 289l-3 (added by Pub. L. No. 93-343), which forms part of the basis for the Departmental regulations at 45 CFR Part 46, and in the preambles to the proposed and final regulations at 45 CFR Part 46. The subjects addressed included the use of FDA-approved drugs for any unapproved purpose; psychosurgery and other techniques for behavior control currently being developed in research centers across the nation; use of experimental intrauterine devices; biomedical research in prison systems and the effect of that research on the prison social structure; the Tuskegee Syphilis Study: the development of special procedures for the use of incompetents or prisoners in biomedical research; and experimentation with fetuses, pregnant women, and human in vitro fertilization. The regulations were intended, and have been uniformly applied by the Department, to protect human subjects against the types of risks inherent in these types of activities.

In Ethical Imperialism I described this last sentence as "a poor description of DHEW’s previous actions." In an earlier draft, I called it a lie, and maybe I should have left that blunt language, since the department had not in fact limited its application to the areas of concern raised in the 1973 hearings.

But perhaps the 1976 interpretation, though not implemented at the time, would still be a wise course. This would be a radical break; while Congress did take testimony about behavior control, it did not investigate other behavioral research, like Milgram's experiments. So aligning the regulations to the concerns of Congress would mean deregulating a great deal of psychological experimentation. I am going to leave this to the psychologists to think about.

Option 3b: Limit the Regulations to the Language of the Statue


A less radical option would be to accept that HHS and other agencies have the authority to determine the scope of "behavioral research" under the statute, but that they must do so within the limits of ordinary understanding. Details can be found in my book, but suffice it to say that Ryan spoke for many when he broke research into three categories--"biomedical, social, and behavioral research"--only two of which appear in the statute. As the deputy general counsel of HEW noted in 1979, "If Congress had wished . . . to cover all human subjects’ research, rather than just biomedical and behavioral, it could have done so." (Ethical Imperialism, 100.)

The question is how to distinguish between behavioral and social research. I propose borrowing language about "health information" from 46 CFR 160.103 and inserting in the Common Rule:


For purposes of this policy, research means biomedical and behavioral research.

Biomedical and behavioral research means an investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge by systematically collecting health information. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for other purposes. For example, some demonstration and service programs may include research activities.

Human subject means a living individual about whom an investigator (whether professional or student) conducting research obtains

(1) Health information through intervention or interaction with the individual, or
(2) Identifiable private health information.

Health information means any information, whether oral or recorded in any form or medium relates to the past, present, or future physical or mental health or condition of an individual; the provision of health care to an individual; or the past, present, or future payment for the provision of health care to an individual. It does not include incidental information about an individual's health, such as the mention of a past illness in the course of an interview about a broad range of subjects.

Either of these options--3a or 3b--would remove from IRB jurisdiction not only history, journalism, and the humanities (including classics!) but also most anthropology, sociology, political science, and the like.

Medical anthropology, medical sociology, and related fields might still face some forms of review, so it would be important to get the Excused category right. And we would probably need some interpretation down the road to make sure those retired Negro Leaguers could talk about their sprained ankles without triggering IRB jurisdiction.

Moreover, this broad deregulation might dismay some social scientists, particularly what I take to be a large number of anthropologists who have grown accustomed, even fond, of IRB review, and see in the regulations the chance to police dodgy practices. To them I can only say, if the problem is so bad, why don't you persuade Congress to study it?

I find this last option--making the regulations conform to the law--the most appealing, since I don't believe executive agencies should regulate behavior that Congress has not found to be a problem through the regular process of hearings, reports, and legislation. I look forward to readers' reactions to these five options and others that they wish to propose.

No comments: