Blogger's note: On September 3, Christopher Leo posted a query to the H-Urban list, asking about the effect of ethics review on urban research . Roberta Gold's response hinted that she had thought hard about the issue, so I asked her to share her thoughts on this blog. She has graciously agreed.
Saturday, September 29, 2007
Roberta S. Gold, “None of Anybody’s Goddamned Business”?
Friday, September 21, 2007
Bledsoe et al., Regulating Creativity
The article, based largely on events at Northwestern itself, is particularly effective at challenging three myths of IRBs and the social sciences:
Myth #1: Reports of IRB interference with research are overblown, since few projects are rejected and few researchers disciplined.
An example of this myth is Jerry Menikoff's contribution to the same symposium, in which he claims, "social and behavioral scientists who maintain appropriate communication with their institution's IRBs need not be shaking in their boots, fearing some career-ending enforcement action is about to come down from Washington."
Unlike Menikoff, Bledsoe et al., talked to some researchers, asking their colleagues about experiences with Northwestern's IRB. They report,
As a number of our colleagues have emphasized . . . both in person and in their responses to our email query, they alter their course not because of any real risk they perceive to their subjects but simply to pass IRB muster. Trying to reduce their own professional risk, they divert their work, choosing topics or populations selectively, or adapting methods that will entail less demanding IRB review and lessen the probability that they will have to make substantial changes before proceeding. IRB procedures, that is, can snuff out ambition even before the project begins.
The disturbing point is that it is the mere anticipation of onerous IRB review that can result in some alteration of the proposed protocol. Because of the potential for delays and the IRB tendency to intrude into each step of the research process, many social science faculty report that they think twice about taking on research topics, methods, and populations that IRB frames in the mode of risk. One respondent described the impact thus:
"The IRB has become a nightmare over the years that I have been a researcher. I'm sure most of this pressure is coming from the federal government, but the rigidity of the model (based on the medical sciences) and the number of hurdles/ forms, and the scrutiny (to the point of turning back projects for mispagination or other pithy errors, as has happened for some of my students) is just terrible. It is very discouraging, and I find myself thinking of EVERY new research project as it relates to the possibility of IRB approval."
Two respondents indicated that faculty had moved toward non-field projects in large part because of IRB. One faculty member even pointed specifically to concerns about IRB in a decision to make a career shift away from field-project themes and methods that might jeopardize the researcher's career:
"Since last year, my research became more theoretical in large part because of IRB requirements. I simply try not to do any research which would involve Panel E [the social science review panel at Northwestern]. . . . I no longer interview people during my trips abroad and try to limit the data gathering to passive observation or newspaper clippings."
An IRB that approves all social science projects submitted to it (and many, no doubt, do) may still crush research by making it so burdensome that researchers give up submitting proposals.
Myth #2: Medical IRBs are the problem, so an IRB devoted only to non-medical research is the solution.
This suggestion gets thrown out from time to time; for example, it appears as one of Dale Carpenter's admittedly "modest proposals for reform" in his own Northwestern Law Review piece. But Bledsoe et al. report that Northwestern already has a separate non-medical panel, and it doesn't sound pretty:
even a separate social science IRB enterprise suffers from internal tensions between the need for standardization, whether imposed by OHRP rules or by our own desires to ensure equity, and the need to allow the very stuff of novelty that studies are supposed to produce. We have observed that social scientists who confront their review assignments can be no less critical of their fellows' studies than a biomedical panel might be. Indeed, IRB staff have sometimes had to step in diplomatically to rescue a project from a zealous social science faculty panelist threatening to dismember it altogether. In this regard, we have observed a typical life cycle for social science panel members. The typical panel member begins his or her tenure by making it known that a great deal of harmless social science research is delayed without any reasonable cause, and that henceforth the reckless invasiveness of the IRB must be tempered. Yet this same panel member, when given projects to review, is often the most critical.
This pattern reflects a broader impulse among social scientists. We think of ourselves first and foremost as academics. Our business is to read research proposals, journal articles, student papers, and to find fault. Turning to IRB protocols, we become fastidious reviewers. When we read consent forms, it is hard for us to refrain from editing them. When we read with an eye toward possible risk, whether large or small, our expertise itself will unmask it. As social science panel members, we will inevitably find problems with social science IRB submissions; we cannot help ourselves. Importing our own disciplines' ethical dilemmas, the concerns that we raise often go far beyond those imagined by the federal legislators. They also hand the IRB, seeing our plight, both our fears and our language of expressing them to incorporate into its already overburdened repertoire. Over time, such impulses are tempered, and we learn to see the big picture again. In the meantime, however, the damage to the research enterprise is done.
In retrospect, giving the social sciences a separate review channel and letting them into the review process was helpful in that the social sciences gained mediators who could explain studies to their panel colleagues and attempt to buffer the power of the medical model. At the same time, our social science panel's own efforts to help both added to the layers of regulatory stratigraphy and intensified the regulatory flux. All this has undoubtedly provided further grounds for investigators to conclude that the IRB was capricious and inconsistent.
The authors are wrong, however, to suggest that Northwestern has a "social science" panel. According to "Schools, Departments and Programs Served by Panel E of the Institutional Review Board, " Panel E has jurisdiction over "research projects involving human subjects that use social and behavioral science methodologies." The same document claims,
Federal guidance defines social and behavioral science methodologies as those that include research on individual or group characteristics or behavior (including, but not limited to, research on perception, cognition, motivation, identity, language, communication, cultural beliefs or practices, and social behavior) or research employing survey, interview, oral history, focus group, program evaluation, human factors evaluation, or quality assurance methodologies.
The range of methods included in this list means that far from letting ethnographers review ethnographers and experimental psychologists review experimental psychologists, Northwestern has locked all its non-medical researchers in a room and told them to fight it out. Such an arrangement makes no allowance for the wide variation of methods and ethics within non-medical research. (See "My Problem with Anthropologists.")
Moreover, the claim that "federal guidance defines social and behavioral science methodologies" is incorrect. The list of methodologies is taken from OPRR's 1998 "Protection of Human Subjects: Categories of Research That May Be Reviewed by the Institutional Review Board (IRB) Through an Expedited Review Procedure." That document does just what its title suggests: it lists categories of research eligible for expedited review. It does not define social and behavioral science methodologies, nor, to my knowledge, has the federal human subjects apparatus ever defined social or behavioral science.
In reality, therefore, Northwestern's Panel E exists solely to provide full IRB review for projects that even the federal government admits do not require full IRB review. No wonder it doesn't work well.
Myth #3: If social scientists were to join IRBs and learn about their workings, they wouldn't complain so much.
Take this statement by J. Michael Oakes, "Risks and Wrongs in Social Science Research: An Evaluator's Guide to the IRB," Evaluation Review 26 (October 2002) 443-479:
"Investigators well versed in the Belmont Report and more technical IRB procedures rarely need to dispute decisions, and when they do it concerns how known rules are interpreted or what is best for the subjects. It follows that a great deal of frustration may be eliminated by careful study of basic IRB regulations and issues. Education seems to modify frustration in the researcher-IRB-subject chain."
Nonsense. Bledsoe herself chaired a subcommittee of the Northwestern University IRB Advisory Committee, and several of her coauthors served on, chaired, or staffed IRBs at Northwestern or elsewhere, as well as having dealt with IRBs as applicants. They are about as educated and experienced in these issues as one could hope for, and they as frustrated as anyone by the current system.
Beyond busting myths, the article seeks to document the changes in IRB work since the 1990s. Based on their personal experience, Bledsoe and her co-authors describe the expansion of both OHRP and IRB jurisdiction:
The university's Office for the Protection of Research Subjects spiraled from two professionals to what is now a staff of 26, of whom 21 support the IRB operation. Review panels went from one to six—four were created simultaneously in September 2000, with one for the social sciences created a year later, and another medical panel added subsequently— and appointing their membership became the duty of the university's vice president for research. The length of the basic protocol template for new projects went from two pages to its present length of twelve for the social sciences, and fifteen for biomedical research. In addition, the number of supplementary forms and documents required for each submission went from one or two to far more than that, depending on the nature of the study. Many protocols are now better measured in inches of thickness than in number of pages. The level of bureaucratic redundancy, inconvenience and aggravation increased dramatically: Unreturned phone calls, dropped correspondence, and administrative errors on forms became routine.
They also report some good news:
For several years after the IRB ramp-up began, our IRB panel expected detailed interview protocols from everyone. Now, an ethnographer who intends to employ participant observation does not need to provide a detailed specification of what is to be said to participants, and is not asked for it. Without such collusion, ethnographic studies would not survive under the IRB system. As much as social scientists complain about the ill fit their projects pose in IRB review, their own protocols are now spared this level of scrutiny.
As I reported earlier, Northwestern has exempted oral history from review, though Bledsoe et al. do not explain when or why that happened.
The authors conclude that "one could scarcely imagine a better example of a bureaucracy of the kind that so fascinated and infuriated Weber than the contemporary IRB system." It is indeed crucial to look at the systematic pressures on members and administrators, for that can explain why the same IRB abuses show up in such diverse institutions spread around the country.
But while Weber can explain some long-term trends, analyzing bureaucracies, rather than people, obscures the role of individual decisions. In this lengthy account of events at Northwestern, the authors decline to blame, credit, or even name a single individual administrator, researcher, IRB member, consultant, or federal official. Typical is this passage:
When the ratcheting up of the IRB bureaucracy at Northwestern was occurring, administrators were working in an environment in which suspension of federal funding to other institutions had produced considerable anxiety. It was no secret that the Northwestern IRB director was under pressure to bring the university into full compliance as quickly as possible.
Who was ratcheting? Who felt considerable anxiety and why? Who communicated with the federal government? Who was the Northwestern IRB director? Who pressured him or her? Who knew the secret? And, on the other end, who ruled that interviewers did not have to submit detailed protocols?
Because the authors decline to ask such questions, they can hold no one to account for sudden and important decisions. They instead conclude, "the odd history of IRB and its effects have been no one's fault; no one's intention. No convenient villains or victims emerge anywhere we look." But there is nothing to indicate that they looked terribly hard.
Friday, September 14, 2007
Study Finds IRBs Exaggerate Risks of Survey Questions
Perhaps because they are punished for being too lax but never for being too strict, IRBs tend to err on the side of what they consider caution, exaggerating the risks of proposed research. It's easy to do so when, as these authors put it, "board members often rely on their 'gut' feeling in determining the potential for survey questions to effect adverse reactions."
To replace that gut feeling with some evidence, Fendrich, Lippert, and Johnson asked survey respondents who had been asked about illegal drug use whether they had felt felt threatened or embarrassed by the questions. Not much: the average score was less than 2 on a 7-point scale. But when asked if other people would feel threatened by those questions, the numbers shot above 5. Thus, survey respondents are as bad as IRBs at guessing how other people will feel about being questioned.
The authors conclude:
Consent documents often summarize potential adverse subject reactions to questions. For example, in the current study, the University of Illinois at Chicago’s REC [research ethics committee] approved consent document contained the following two sentences under the heading: “What are the potential risks and discomforts?”
There is a risk that you may feel anxious, uncomfortable or embarrassed as a result of being asked abut drug use and drug testing experience. However, you are free not to answer any question, and you are free to withdraw from the study at any time.
If our findings can be generalized to other studies asking questions about drug use, the first sentence may inappropriately convey an exaggerated sense of a drug survey’s risk. Even though voluntary participation is a non-contingent right, the second sentence seems to link the right of refusal and the voluntary nature of participation to this exaggerated risk.
The first author’s experience as a member and Chair of a behavioral science REC leads him to conclude that paragraphs like those cited above are common in survey consent documents. Researchers may pair statements about rights with statements about risk in order to appease REC concerns about study interventions to address risk. In the absence of empirical data, RECs should be cautious about recommending and approving consent documents that include clauses suggesting that questions about drug use cause emotional discomfort. Furthermore, RECs should recommend that consent documents decouple important reminders about subject rights from statements about potential risk (whether or not those risks are valid). While it may be important to reinforce rights in a consent document, we believe it is contrary to best practice to even imply that voluntary participation (and the right to withdraw or refuse to answer questions) should be contingent on adverse reactions. The type of text described above, however, would be obviated if RECs adopted a more realistic view of subject perceptions regarding drug use surveys.
Laud Humphreys Remembered
Friday, August 24, 2007
Macquarie's Respect for Expertise
Particularly striking is Downey's description of who reviews applications:
A kind of departmental review does take place within the university-wide committee at Macquarie, as members of the committee are clustered so that color-coded sub-groups do the preliminary and most serious review of applications for which they have special expertise. If an application has to go to the whole committee (for example, research with children, medical procedures, Aboriginal Australian groups, or ethically challenging research tends to), we usually turn to the members of our committee who are best versed in the area of study. If we have a particularly difficult ones, we’ll consult with a faculty member outside the committee who has special experience.
The key words here are "special expertise," "best versed in the area of study," and "special experience." Someone at Macquarie has decided that having a nutritionist review oral history, or an oral historian review nutrition experiments, is not in the best interest of researcher or subject, but that having knowledgeable people review applications might make everyone happy. I am particularly impressed that the experts do the preliminary review, which under the American regulatory framework (I'm not sure about Australia) would mean giving them the power to provide exemptions or expedited approval.
As I have noted in comments on his postings, Downey has yet to persuade me that even this level of expert review is necessary for projects by trained researchers that only involve survey, interview, and observation research, and I remain attracted to the University of Pennsylvania system, under which researchers are certified and then, largely, left alone. But I thank Downey for introducing me to a university that is thinking hard and creatively about ethical review.
Tuesday, August 21, 2007
Northwestern IRB: Unsystematic Interviews Are Not Subject to Review
Today's New York Times features a story, "Criticism of a Gender Theory, and a Scientist Under Siege," about the case of J. Michael Bailey, Professor of Psychology, Northwestern University. Bailey's controversial book about identity. The book provoked several complaints, including the charge by "four of the transgender women who spoke to Dr. Bailey during his reporting for the book . . . that they had been used as research subjects without having given, or been asked to sign, written consent."
As reported by the Times, the case was investigated by Alice Domurat Dreger, Associate Professor of Clinical Medical Humanities & Bioethics at Northwestern, who has posted a draft article on the subject, "The Controversy Surrounding The Man Who Would Be Queen: A Case History of the Politics of Science, Identity, and Sex in the Internet Age," [PDF]
Dreger finds that Bailey did not commit serious ethical violations, nor did he violate the requirements for IRB review:
the kind of research that is subject to IRB oversight is significantly more limited than the regulatory definition of “human subject” implies. What is critical to understand here is that, in the federal regulations regarding human subjects research, research is defined very specifically as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge” (United States Department of Health and Human Services, 2005, sect. 46.102, def. “b”). In other words, only research that is truly scientific in nature—that which is systematic and generalizable—is meant to be overseen by IRBs. Thus, a person might fit the U.S. federal definition of “human subject” in being a person from whom a researcher gains knowledge through interpersonal interaction, but if the way that the the knowledge she or he intends to gain is unlikely to be generalizable in the scientific sense, the research does not fall under the purview of the researcher’s IRB.
It is worth noting here, for purposes of illustration of what does and doesn’t count as IRB-qualified work, that I consulted with the Northwestern IRB to confirm that the interviews I have conducted for this particular project do not fall under the purview of Northwestern’s IRB. Although I have intentionally obtained data through interpersonal interaction, the interview work I have conducted for this historical project has been neither scientifically systematic nor generalizable. That is, I have not asked each subject a list of standardized questions—indeed, I typically enjoyed highly interactive conversations during interviews; I have not interviewed all of my subjects in the same way; I have negotiated with some of them to what extent I would protect their identities. This is a scholarly study, but not a systematic one in the scientific sense. Nor will the knowledge produced from this scholarly history be generalizable in the scientific sense. No one will be able to use this work to reasonably make any broad claims about transsexual women, sex researchers, or any other group.
When I put my methodology to the Northwestern IRB, the IRB agreed with me that my work on this project is not IRB-qualified, i.e., that, although I have obtained data from living persons via interactions with them, what I am doing here is neither systematic nor generalizable in the scientific sense.
Clearly Bailey's work hurt the feelings of some people he wrote about, but, as Dreger notes, "scholarship (like journalism) would come to a screeching halt if scholars were only ever able to write about people exactly according to how they wish to be portrayed." Indeed, that's what social scientists have been arguing for three decades.
Monday, August 20, 2007
Guidance Creep
I recently came across a striking example of this. In 1983, Richard Louttit of the National Science Foundation, who had helped craft the list of exemptions encoded in 45 CFR 46.101, explained them as follows:
Much research of minimal risk was exempted from IRB review in order to reduce the IRB workload so that research involving ethical questions could get more than cursory review. But some institutions have decided that the IRB, or its chairperson, must review proposals to decide if they are exempt from review. If this seems contradictory, it is. And this was not envisioned by the staff group which worked out the exemptions.
[Richard T. Louttit, "Government Regulations: Do They Facilitate or Hinder Social and Behavioral Research?," in Joan E. Sieber, ed., NIH Readings on the Protection of Human Subjects in Behavioral and Social Science Research: Conference Proceedings and Background Papers (Frederick, Md: University Publications of America, 1984), 179.]
Yet in 1995, the Office for Protection from Research Risks adopted just that contradictory position:
Institutions should have a clear policy in place on who shall determine what research is exempt under .46.101(b). Those persons who have authority to make a determination of what research is exempt are expected to be well-acquainted with interpretation of the regulations and the exemptions. In addition, the institution should be prepared to reinforce and review, as necessary, the method of determining what is exempt. OPRR advises that investigators should not have the authority to make an independent determination that research involving human subjects is exempt and should be cautioned to check with the IRB or other designated authorities concerning the status of proposed research or changes in ongoing research.
(OPRR Reports, 95-02) [21 July 2014 updated link.]
The regime in place today is far more intrusive than the one worked out in 1981. Changes in the regulations themselves are part of the problem, but so are radical reinterpretations like the one above.
Thursday, August 16, 2007
Study Finds IRBs Impose Inappropriate Forms and Guidelines
Sarah Flicker et al., "Ethical Dilemmas in Community-Based Participatory Research: Recommendations for Institutional Review Boards," Journal of Urban Health 84 (July 2007); 478-493.
I do not have access to the full article, but here's the abstract:
National and international codes of research conduct have been established in most industrialized nations to ensure greater adherence to ethical research practices. Despite these safeguards, however, traditional research approaches often continue to stigmatize marginalized and vulnerable communities. Community-based participatory research (CBPR) has evolved as an effective new research paradigm that attempts to make research a more inclusive and democratic process by fostering the development of partnerships between communities and academics to address community-relevant research priorities. As such, it attempts to redress ethical concerns that have emerged out of more traditional paradigms. Nevertheless, new and emerging ethical dilemmas are commonly associated with CBPR and are rarely addressed in traditional ethical reviews. We conducted a content analysis of forms and guidelines commonly used by institutional review boards (IRBs) in the USA and research ethics boards (REBs) in Canada. Our intent was to see if the forms used by boards reflected common CBPR experience. We drew our sample from affiliated members of the US-based Association of Schools of Public Health and from Canadian universities that offered graduate public health training. This convenience sample (n = 30) was garnered from programs where application forms were available online for download between July and August, 2004. Results show that ethical review forms and guidelines overwhelmingly operate within a biomedical framework that rarely takes into account common CBPR experience. They are primarily focused on the principle of assessing risk to individuals and not to communities and continue to perpetuate the notion that the domain of “knowledge production” is the sole right of academic researchers. Consequently, IRBs and REBs may be unintentionally placing communities at risk by continuing to use procedures inappropriate or unsuitable for CBPR. IRB/REB procedures require a new framework more suitable for CBPR, and we propose alternative questions and procedures that may be utilized when assessing the ethical appropriateness of CBPR.
Tuesday, August 14, 2007
Incestuous Gay Monkey Sex
Mary L. Gray, an anthropologist at Indiana University at Bloomington, described her work in graduate school, which raised all kinds of red flags with her IRB at the time: She wanted to study the way gay, lesbian, bisexual and transgender youth develop their identities in the rural Southeast, and she wanted to base her research on interviews with such youth, under the age of 18, without their parents’ knowledge. Her project, she said, “had every imaginable red flag.”
With some regrets, she won IRB support by appealing to prejudice many have of the rural South. Although she had no evidence to make this claim, she argued that the situation in the rural South is “so awful” for the young people she was studying that she couldn’t possibly approach their parents for consent. (Actually Gray believes that the situation for gay youth is more subtle and less uniform than she suggested, but she guessed it would work with the IRB, and it did.)
Because the IRB was — like most IRB’s — oriented around medical research, not social science, the focus was on potential harm that Gray could cause her research subjects in person. Gray reported that she received relatively little questioning or guidance from her IRB on one of her major areas of research: what the young people she studied wrote about themselves online. Gray developed her own ethics rules (she wrote to the subjects to ask permission), but she was struck by what was and wasn’t considered important by the IRB.
To the IRB, “distance read as objectivity” and so was by definition “good,” she said. Never mind that what her subjects shared about themselves online was as important as the thoughts they shared in person. This points to Gray’s broader critique of the IRB process. Social scientists frequently complain about IRB’s failing to understand their studies, but Gray suggested it was time to move beyond the idea of just adding more social scientists to the panel. Rather, she said it was time to question certain underlying assumptions of IRB’s and whether they even make sense for social science. It’s not that Gray doesn’t think there are ethical issues researchers must consider, but whether the medical model can ever work for projects that don’t follow the pattern of having a hypothesis designed to lead to the dispassionate creation of generalizable knowledge.
Gray said that “IRB fatigue” is discouraging researchers — especially graduate students — from even trying to get projects approved.
I can't say that I'd want a graduate student in any field asking minors about their sex lives without some kind of supervision. But it sounds as though UC San Diego's IRB lacked the expertise to give Gray meaningful guidance. That lack of expertise is built into the system of local IRB review, and it can produce decisions that are too lax as well as those that are too strict.
Evolving Research
University of Winnipeg professor of politics Christopher Leo poses and then answers positively the question, "Does the Ethics Bureaucracy Pose a Threat to Critical Research?" This provocative essay raises so many key questions that I plan to return to it in future postings. For now, note Leo's description about the evolving nature of his work:
Many researchers concerned with politics and policy stay in regular touch with politicians and public servants and, in the process, ask them questions the answers to which may well be used in future publications. That is an essential part of the research process because regular contact with well-informed people makes it possible for researchers to stay abreast of events and identify important issues as they arise.
So when does a query become a research question and a conversation an interview that requires ethics review? The guidelines are little help in answering that question, but, if we take them literally, they would appear to have taken from university researchers a right that every ordinary citizen enjoys, namely that of picking up the phone and talking to a politician or public servant without applying for bureaucratic permission to do so.
Meanwhile, over at Savage Minds, Alex Golub, assistant professor of anthropology at the University of Hawai’i Manoa (aka Rex), touches on the same question in the posting, "Using informed consent forms in fieldwork." He writes, "In some cases I interviewed people I’d known for years. I’d have breakfast or lunch with them and then schedule the official ‘interview’ for later on in the week."
IRBs that rigidly follow a biomedical model for ethics may insist that research protocols be spelled out in advance--even demanding sample questions. Such demands are inappropriate for the kind of work described by Leo and Golub: keeping in touch with knowledgeable people over a period of time.
An alternative appears in the University of Pennsylvania's Policy Regarding Human Subject Research in the Sociobehavioral Sciences. That policy accommodates such projects by freeing researchers from the requirement to submit "a fixed research protocol":
Evolving Research
Evolving research is a class of research in the sociobehavioral sciences in which the questions that are posed evolve in the course of investigation. An example is ethnography, where research questions may only be clarified after a period of observation and where current findings drive the next steps in the study. This class of research typically involves studying human behavior in non experimental settings, with or without active participation by the investigator; but it can also occur in more structured observational settings (e.g., oral histories, focus groups). In specific cases, such research does not pose more than minimal risk to human subjects and is considered to be “exempt from review,” as stated below. An approved mechanism is necessary for presenting to the IRB a research protocol that will evolve in the course of investigation. This policy institutes such a mechanism via certification.
4a. Research involving only non-interventionist observation of behavior occurring in public (including domains of the Internet clearly intended to be publicly accessible), for which no identifying information is recorded, is exempt from review.
4b. Investigators are allowed to use their certification, as per policy item 1, as a reference for describing evolving research activities to the IRB in lieu of a fixed research protocol.
This policy eliminates the need for investigators doing evolving research to spell out the details of a dynamic research protocol. The IRB can be assured that the research will be conducted in an ethically appropriate fashion, with full protection of human subjects, when certified investigators attest that their pre-registered research plan will be conducted within the ethical framework laid out in the training program for which they are certified.
In other words, if you have shown you know what you are doing, you don't have to get the IRB's approval for specific questions or topics.
I must note, however, that the Penn policy includes this disclaimer: "Note that different studies by the same investigator(s) must be submitted to the IRB as separate research protocols. These must not be viewed as a single study evolving from one investigation into another." I wonder what the Penn IRB would do with someone like Leo or Golub, who has the audacity to keep in touch with people for years.
