1 Institute for Ethics, the Vrije Universiteit, Amsterdam; 2 Department of Radiotherapy, The Netherlands Cancer Institute/Antoni van Leeuwenhoek Hospital, Amsterdam, 4 Division of Psychosocial Research and Epidemiology, The Netherlands Cancer Institute/Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands; 3 Department of Social Medicine, Division of Medical Ethics, Harvard Medical School, Boston, MA, USA
Received 28 August 2001; revised 30 January 2002; accepted 11 March 2002
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
This study examined the assessment of risk/benefit ratios for phase II cancer clinical trials by Institutional Review Board (IRB) members.
Patients and methods:
Semi-structured interviews were conducted with 53 IRB members from six research hospitals and specialized cancer centers in The Netherlands.
Results:
While the toxicity and side-effects of treatment were most often identified as risks associated with participating in a phase II trial, approximately two-thirds of IRB members also cited psychosocial and/or quality-of-life risks. Conversely, 68% of the respondents identified psychosocial benefits of trial participation, while 25% cited treatment effectiveness as a possible benefit. Between one-quarter and two-thirds of respondents indicated that trial protocols provide insufficient information regarding the likelihood, magnitude and duration of both risks and benefits. Between 15% and 34% of IRB members reported feeling less than fully competent at evaluating various aspects of phase II protocols (e.g. originality and feasibility of the study, adequacy of the methods and analysis procedures, etc.). This was particularly the case for non-physician IRB members. Few IRB members reported weighing risks and benefits in a systematic manner, but rather relied on global impressions or preferred to leave such matters to the IRB as a whole or to their patients.
Conclusions:
A substantial minority of IRB members believes that trial protocols provide too little information relevant to evaluating various cost/benefit and scientific issues, and feels less than fully competent in carrying out such evaluations. IRB members are more likely to identify psychosocial benefits than physical health benefits that may accrue to patients participating in phase II trials.
Key words: cancer, decision making, Institutional Review Boards, phase II protocols
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The study reported here is the first of a four-part investigation designed to gain greater insight into risk/benefit ratio (RBR) assessments of phase II and phase III cancer clinical trials. This first stage of the research involved conducting semi-structured interviews with IRB members to determine their attitudes, beliefs and experiences in evaluating the RBR of phase II cancer clinical trials in general [16]. In the second stage of the study, IRB members were asked to evaluate two protocols (a phase II and a phase III clinical trial) by means of a questionnaire. In the third stage, in-depth interviews were conducted with the IRB members about the protocol evaluations that they had carried out in stage two. The final stage of the study involved observation and analysis of the meetings of the full IRBs of two of the participating hospitals, during which the two protocols evaluated in stages two and three were discussed. The results from the second to the fourth stages of this investigation will be reported in subsequent papers.
In the first stage of this research, we evaluated four aspects of the risk/benefit assessment process: (i) identification of the risks and benefits of phase II cancer clinical trial protocols; (ii) estimation of the amount of information needed to make a risk/benefit assessment and whether such information is typically available in phase II clinical trial protocols; (iii) self-reported competence of IRB members to make risk/benefit assessments; and (iv) evaluation of specific risks and benefits for patients participating in phase II clinical trials, for future cancer patients, and for medical science.
![]() |
Patients and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Study measures: the semi-structured interview
The semi-structured interviews, including a combination of open-ended and closed questions, focused on the identification, estimation and evaluation of the risks and benefits associated with phase II clinical trials. On average, the interviews took 1 h to complete.
Identification of risks and benefits
A series of open-ended questions was asked about the identification of risks and benefits associated with phase II studies: (i) "How do you determine what risks and benefits to participating patients are associated with phase II studies?"; (ii) "What type of risks and benefits (to participating patients) do you identify in phase II studies?; and (iii) "What type of benefits do you identify in phase II studies for future patients and for medical science?".
Estimation of risks and benefits
Two types of questions with closed response categories were asked about the estimation of risks and benefits associated with phase II studies: (i) on the adequacy of information contained in trial protocols for evaluating risk/benefit issues; and (ii) on the perceived competence of IRB members to evaluate scientific and risk/benefit ratio issues. The adequacy of the information contained in trial protocols was assessed with the question: "In general, do you think that you are given enough information (empirical data) in phase II protocols to judge the following issues?". Responses were scored on a 4-point Likert-type scale, ranging from more than enough to very insufficient. The specific issues addressed are listed in Table 1.
|
|
Data analysis
Open-ended questions with respect to the identification and evaluation of risks and benefits were organized into categories and reported as percentages (see Tables 3, 4 and 5). The resulting categories were not mutually exclusive because IRB members could mention more than one risk or benefit (Table 3), method for evaluating risks and benefits (Table 4) or decisive factor (Table 5).
|
|
|
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Table 3 displays the types of risks and benefits to patients participating in phase II clinical trials identified by IRB members. All of the respondents identified the toxicity and side-effects of treatment, and nearly all identified the additional burden associated with trial participation (e.g. frequent visits to the hospital, extra tests) as a common risk associated with trial participation. Less self-evident was that 65% of the respondents identified psychosocial risks associated with trial participation. These included uncertainty about what is going to happen, a potential (and false) sense of hope about treatment efficacy, and confrontation with the fact that the disease cannot be cured and that the treatment may be of only limited or no direct benefit. Additionally, 20% of the respondents reported a decrease in quality of life as a risk of treatment.
Conversely, 68% of the respondents indicated a number of specific psychological benefits associated with participation in a phase II trial. These included an increase in the amount of attention and support received from medical and ancillary health care providers, a sense that there is still something that can be done to actively treat the disease, as well as a personal feeling of being able to fight back against the disease. Approximately one-third of the respondents identified improved quality of life as a potential benefit of trial participation. Approximately one-quarter of the IRB members interviewed identified treatment efficacy as a possible benefit. As expected, all respondents identified the potential for developing more effective cancer therapies as the primary benefit to future patients and to the scientific community.
Estimation of risks and benefits
Perceived adequacy of information provided in protocols
As indicated in Table 1, only a small minority of the respondents (8%) indicated that phase II protocols typically contain too little information about the types of benefits that might accrue to participating patients. However, between approximately one-quarter and one-half of the respondents reported that there is frequently insufficient information provided about the likelihood, the magnitude and the duration of such benefits.
Similarly, most IRB members believed that protocols provide sufficient information about the types of risks involved in phase II clinical trials. However, a substantial percentage of the respondents indicated that too little information is available regarding the likelihood, seriousness, duration and reversibility of those risks (44, 36, 60 and 36%, respectively). Approximately 90% of the respondents indicated that sufficient information is provided regarding the potential importance of the clinical trial for future patients and for medical science.
Perceived competence in protocol evaluation
Between 15% and 34% of the IRB members reported that it was (very) difficult to judge the various risks and benefits associated with phase II clinical trials, both for participating patients and for future patients, and also for society at large. While nearly all of the respondents indicated that they understood the aims (92%) and scientific relevance (94%) of phase II clinical trials, a substantial minority felt less than fully competent at evaluating the originality of the research (40%), the feasibility of the trial (34%), the scientific methodology employed (30%), the way in which the trial data were to be analyzed (42%) and how the trial relates to previous studies (32%).
Evaluation of risks and benefits
Table 4 shows the different ways in which IRB members go about evaluating the risks and benefits of phase II studies. Most of the decision strategies followed do not reflect a process of weighing risks and benefits against each other in a systematic way, but rather involve gaining an overall impression (20%), considering what alternative treatments are available (15%), or considering whether one would be willing to undergo the trial-based treatment oneself or would advise a family member to do so (10%). Seventeen percent of the respondents indicated that they typically leave the decision as to whether the benefits of a trial outweigh the risks to the patients themselves, and 12% reported that it is a task for the IRB as a whole, rather than for himself or herself as an individual IRB member.
The factors considered by IRB members to be decisive in assessing the risk/benefit ratio in phase II clinical trials are displayed in Table 5. Interestingly, one-third of the respondents were unable to identify such a factor in that they do not typically make such assessments themselves. The issue reported most frequently as being decisive was the potential value of the trial to future patients and to medical science (i.e. the potential of finding a more efficacious treatment) (21%). This was followed by the risks, burdens and inconvenience to participating patients (18%), the expectation that the treatment would be beneficial to the participating patients (16%), and feeling comfortable in proposing the trial-based treatment to patients (11%).
Professional status, length of IRB membership and RBR assessment
All respondents, regardless of professional background, identified the potential for developing better cancer therapies, with the resulting benefits to future patients and to medical science as possible benefits of phase II trials. Physicians and other professionals also did not differ significantly with respect to the perceived adequacy of information contained in phase II trial protocols. However, a significantly greater percentage of the IRB members who were physicians reported feeling competent in assessing the originality of the research (58% compared with 22% of non-physicians; P <0.05), the methodology employed (62% compared with 41%; P <0.01) and whether the trial would yield new insights (77% compared with 37%; P <0.05) (data not shown in tabular form).
When comparing oncology specialists (i.e. medical, radiation or surgical oncologists) with other IRB members (including family physicians), statistically significant differences were observed in: (i) the perceived adequacy of information provided in trial protocols about the likelihood of benefits (100% and 68%, respectively; P <0.05) and risks (89% and 49%; P <0.01) to patients, and (ii) perceived competence in evaluating the toxicity of the treatment (100% and 50%; P <0.05), the invasiveness of the treatment (100% and 55%; P <0.01), the originality of the trial (89% and 30%; P <0.05) and the place of the trial in relation to previous research (89% and 34%; P <0.05) (data not shown in tabular form).
Relatively new IRB members (i.e. those who had been members for 4 years) were more likely than members of longer standing to report that trial protocols contain insufficient information on the likelihood of benefits (40% compared with 9%; P <0.05) and the type of risks (23% compared with 14%, P <0.05) to patients. No statistically significant association was found between length of IRB membership and perceived competence to evaluate trial protocols (data not shown in tabular form).
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Secondly, we investigated the perceived adequacy of the information typically available in phase II protocols, and the perceived competence of IRB members to make risk/benefit assessments. Although the large majority of IRB members indicated that trial protocols contain sufficient information about the types of risks and benefits involved in phase II trials in general, a substantial percentage (between approximately 25% and 60%) reported that too little information is available about more specific issues such as the likelihood, magnitude and duration of such risks and benefits. This does not necessarily mean that phase II clinical trial protocols are poorly written. In some cases, it may not be possible to estimate accurately the exact nature of the risks and benefits involved. In other cases, however, protocol authors may not recognize the need for providing such information. Unfortunately, respondents were not asked whether protocols explicitly state that such detailed information about risks and benefits is not available, or whether they believe that the need for this detail was simply overlooked. In either case, the lack of detail provided in protocols may explain, at least in part, the finding that approximately one-third of IRB members do not make a risk-benefit calculation at all, and that 17% leave such matters up to the patients. This latter finding is somewhat disconcerting in light of the evidence that patients are often unable to understand fully the information provided to them about the clinical trials in which they are invited to participate [1921].
A substantial minority of IRB members (ranging from 15% to 40%) reported feeling less than fully competent to evaluate the risks and benefits associated with phase II trials and the scientific details of phase II trial protocols. One possible means of increasing the (perceived) competence of IRB members is through formal training programs. Such training courses are available in The Netherlands. However, IRB members are not required to follow them. While these findings may give cause for concern, they need to be tempered by the fact that protocol evaluation is typically a multidisciplinary, group process. It may not be realistic or necessarily efficient to expect that every individual member of an IRB is capable of evaluating all aspects of a clinical trial. In addition, although this is not typically the case in The Netherlands, some hospitals have separate committees for evaluating the scientific compared with ethical/human subject protection aspects of clinical trial protocols. This is the case for one of the hospitals (Vrije Universiteit, Amsterdam) from which IRB members were recruited for this study. Other participating hospitals have separate scientific committees within some, but not all, of their medical departments.
Thirdly, we inquired into the methods employed by IRB members to arrive at a risk/benefit estimate and the most important factors taken into consideration in such estimates. Only a small minority of the respondents indicated that they weighed risks and benefits against one another in a systematic way. More typically, such evaluations are made on a more global level, based on both individual judgments regarding the acceptability of the trial, whether one would participate oneself, and on the results of the decision-making process of the IRB as a whole. It is unclear as to whether a more formal and systematic form of individual risk/benefit evaluation would yield different decisions. It might, however, foster greater clarity in the criteria used to reach decisions, and greater consistency in the application of such criteria across IRBs.
Finally, we investigated whether there are systematic differences in risk/benefit assessment as a function of the professional background and number of years of committee experience of IRB members. As might be expected, physicians (particularly those specialized in oncology) and IRB members of longer standing were less likely than other IRB members to report inadequacies in the information provided by phase II trial protocols, and indicated having less difficulty in evaluating the various scientific aspects of those protocols. Similar results have been reported in studies of IRBs in general (i.e. those not specifically focused on cancer clinical trials) [10].
The current results need to be interpreted in the light of certain study limitations. First, we did not distinguish between different types of phase II protocols (e.g. chemotherapy, radiotherapy or surgery). While we have no reason to believe that this would have a significant impact on the findings, it is something that merits further study. Additionally, respondents were not asked to distinguish between phase II protocols that arise from within their own institutions and those from other sources, either national or international. Attempts to obtain systematic data on this matter from the participating IRBs proved unsuccessful due to the fact that such information is not routinely registered in databases and thus is not readily accessible. Thus additional research is needed to determine whether the quality of protocols varies significantly as a function of their source (e.g. local versus national/international; academic versus industry).
Thirdly, issues surrounding potential conflict of interest were not addressed in this study. This deserves additional attention, in that the assessment by medical IRB members of the risks and benefits of protocols might be influenced by their involvement in specific trials. We would note that although there are no official regulations in The Netherlands regarding how potential conflicts of interest should be handled, in practice, IRB members are asked to either withdraw from deliberations when their protocol is being discussed, or they may be present but are not allowed to cast a vote.
Fourthly, as noted above, the focus of our research was on the individual members of IRBs. The decisions made by IRBs, and the discussions that form the basis of such decisions, are of a collective nature. Each IRB member contributes to the decision-making process from his or her professional perspective and, undoubtedly, the whole is more than the sum of its parts. Thus, a more dynamic, group-oriented research approach is needed to obtain a comprehensive picture of both the process and content of risk/benefit assessment. Hopefully, the fourth phase of our investigation, in which we observed and analyzed the meetings of several IRBs, will shed additional light on the decision-making process and the role of individual members in that process.
Finally, our study was restricted to members of IRBs from Dutch academic hospitals and specialized cancer treatment centers. IRBs in various European countries and in North America may differ in some respects (e.g. with regard to informed consent requirements, the need for hospitals and physicians to protect themselves against potential legal risks, etc.). Nevertheless, the basic structure, objectives and procedures for protocol review are quite similar across countries, and thus we believe that our results can reasonably be extended to other national systems of research oversight. Nevertheless, this needs to be confirmed empirically.
In summary, the results of this study indicate that there is a good deal of variability in the ways in which individual members of IRBs identify, estimate and evaluate the risks and benefits associated with phase II clinical trials in oncology. A substantial minority of IRB members believes that trial protocols provide too little information relevant to evaluating various cost/benefit and scientific issues, and feels less than fully competent in carrying out such evaluations. In general, IRB members are more likely to identify psychosocial benefits than physical health benefits that may accrue to patients participating in phase II trials. They view such trials primarily as a vehicle for testing new therapies that may be of benefit to future patients and to medical science in general. While this is in line with the goals and objectives of the vast majority of phase II trials, it may not reflect the expectations of patients themselves, who often choose to participate in such trials in the hope that the treatment will prove to be clinically effective. This underscores the need to provide potential patients in phase II clinical trials with sufficient information to make informed decisions, and to ensure that such decisions are appropriately motivated.
![]() |
Acknowledgements |
---|
![]() |
Footnotes |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2. Gilbert C, Fulford KWN, Parker C. Diversity in the practice of district ethics committees. Br Med J 1989; 299: 14371439.[ISI][Medline]
3. Hall D. The research imperative and bureaucratic control: the case of clinical research. Soc Sci Med 1991; 32: 333342.[ISI][Medline]
4. Miller J. Ethics review in Canada: highlights from a national workshop, part 1. Ann R Coll Physicians Surg Can 1989; 22: 515523.[Medline]
5. While AE. Research ethics committees at work: the experience of one multi-location study. J Med Ethics 1996; 22: 352355.[Abstract]
6. Goldman J, Katz MD. Inconsistency and Institutional Review Boards. JAMA 1982; 248: 197202.[Abstract]
7. Levine RJ, Katz MD, Goldman J. Inconsistency and IRBs. Flaws in the Goldman and Katz study. IRB 1984; 6: 48.[Medline]
8. Foster CG, Marschall T, Moodie P. The annual reports of local Research Ethics Committees. J Med Ethics 1995; 21: 214219.[Abstract]
9. Harries UJ, Fenthem PH, Tuxworth W, Hoinville GW. Local research ethics committees. Widely differing responses to a national survey protocol. J R Coll Physicians Lond 1994; 28: 150154.[ISI][Medline]
10. Berghmans RLP, ter Meulen RHJ, Wachter de MAM. Medisch-ethische toetsing van geneesmiddelenonderzoek in Nederland [Medical ethical review of research in pharmaceuticals in The Netherlands]. Maastricht: The Institute of Health Ethics 1996.
11. Bjune G, Gedde-Dahl TW. Some problems related to risk-benefit assessments in clinical testing of new vaccines. IRB 1993; 15: 15.
12. Meslin EM. Protecting human subjects from harm through improved risk judgements. IRB 1990; 12: 710.
13. Meslin EM, Lavery JV, Sutherland HJ, Till JE. Judging the ethical merit of clinical trials: What criteria do research ethics board members use? IRB 1994; 16: 610.
14. Meslin EM. Ethical issues in the substantive and procedural aspects of research ethics review. Health Law Can 1993; 13: 179191.[Medline]
15. Meslin EM. Philosophical considerations about risks and risks assessment in medical research. In Koren G (ed.): Textbook of Ethics in Pediatric Research. Malabar: Krieger Publishing 1993; 3755.
16. van Luijn H. Evaluation of oncological phase II clinical studies by Institutional Review Board members in The Netherlands. Proceedings of the 13th World Congress on Medical Law, 610 Aug 2000, Helsinki, Finland.
17. Kodish E, Stocking C, Ratain MJ et al. Ethical issues in phase I oncology research: a comparison of investigators and Institutional Review Board chairpersons. J Clin Oncol 1992; 10: 18101816.[Abstract]
18. Miller M. Phase I cancer trials: a collusion of misunderstanding. Hasting Cent Rep 2000; 30: 3442.[Medline]
19. Daugherty C, Ratain MJ, Grochowski E. Perceptions of cancer patients and their physicians involved in phase I trials. J Clin Oncol 1995; 13: 10621072.[Abstract]
20. Schaeffer MH, Krantz DS, Wichman A et al. The impact of disease severity on the informed consent process in clinical research. Am Med J 1996; 100: 261268.[ISI][Medline]
21. Aaronson NK, Visser-Pol E, Leenhouts GH et al. Telephone-based nursing intervention improves the effectiveness of the informed consent process in cancer clinical trials. J Clin Oncol 1996; 14: 984996.[Abstract]