EDITORIAL

Selection Bias, Phase II Trials, and the FDA Accelerated Approval Process

Stephen L. George

Correspondence to: Stephen L. George, PhD, Duke University Medical Center, Cancer Center Biostatistics, Box 3958, Durham NC 27710 (e-mail: stephen.george{at}duke.edu).

Several observational studies have found that the distance traveled from a patient’s home to the treatment center is an important prognostic variable. In this issue of the Journal, Lamont et al. (1) investigated this phenomenon for clinical trials at a specialized cancer treatment center and reported that patients with head and neck cancer enrolled on phase II clinical trials experienced better survival the further they lived from the treatment center. This finding could not be explained away by adjustment for other observable demographic, medical, or socioeconomic factors (e.g., age, stage of disease, performance status, income level) that are known to influence outcomes. We are left with the conclusion that distance traveled for care—something rarely reported in the analysis of trials—is a potentially important prognostic variable, probably reflecting the effect of other more fundamental but unmeasured variables. This conclusion is unsettling because it suggests that other unknown or unmeasured variables have important prognostic impact.

The results of Lamont et al. also provide an important reminder of the degree to which selection factors can influence the outcomes of trials. If the studies they considered had been restricted to distant patients, the overall results would have been impressively positive. Conversely, had they been restricted to local patients, the results would have been discouragingly negative. Selection bias, the bias induced by the enrollment process on clinical trials, can seriously damage the external validity of trials. Simply put, patients enrolled on clinical trials often bear little resemblance to the larger population of patients to which we wish to generalize the results because of the complicated processes by which patients are identified and recruited for clinical trials. Although randomized trials are not immune from selection bias, they have the strong advantage that the treatment comparisons are unbiased because of the randomization. But nonrandomized trials, including phase II trials, do not have this advantage. Thus, the results from nonrandomized trials in general, and phase II trials in particular, depend heavily on the specific characteristics of patients studied, including the known and unknown variables. Moreover, the relatively small number of patients enrolled on these trials makes it difficult to adjust adequately for even the known variables when, say, comparing the results to those from other phase II trials.

Selection bias is a well-known concern in the traditional drug development paradigm, in which promising but potentially misleading results from phase II trials require subsequent larger, expensive randomized phase III trials to confirm the results. It is also true, as Lamont et al. note in their article, that misleadingly negative results for a beneficial drug from phase II trials would not be followed up. The concern about false-positive results is especially acute in light of a relatively recent mechanism for U.S. Food and Drug Administration (FDA) approval of oncology drugs called "accelerated approval." This mechanism, first used in 1995, allows approval based on a surrogate endpoint considered reasonably likely to predict patient benefit (2). Thus, because clear evidence of patient benefit is not required, accelerated approval is inevitably based on evidence that is less than sufficient to grant full approval. The majority of the more than 20 accelerated approvals granted to date have been based on the results of phase II trials.

The purpose of accelerated approval is to provide rapid access to potentially effective agents in serious or life-threatening diseases when no other effective therapies exist. Unfortunately, no drug is completely safe. The risk of approving a "toxic placebo" increases as the standards of approval are lowered. To mitigate this risk, FDA regulations appropriately require the completion of additional, more definitive studies to confirm or deny the patient benefit suggested by the results with the surrogate endpoint. Thus, there is the possibility of rescinding the approval if the sponsor fails to complete the confirmatory studies with due diligence or if the results do not confirm the initial promise. However, to date, no accelerated approval in oncology has been withdrawn, and only four accelerated approvals have been converted to full approval.

Many issues have been raised about the accelerated approval process, including the validation of the surrogate endpoint used, the assessment of "reasonably likely" in predicting benefit, and the meaning of "due diligence" in completing the required additional studies. But perhaps the most fundamental concern is that accelerated approvals are based on evidence that is less reliable evidence than that required for full approval. The accelerated approval of Iressa (gefitinib) in May 2003 as monotherapy for third-line treatment of advanced non–small-cell lung cancer (NSCLC) is instructive in this regard. The basis for approval was a phase II study involving 139 patients refractory to both platinum-based and docetaxel therapy, which are currently considered to be the standard of care. The overall response rate on this trial was approximately 10%. However, for the reasons discussed above, the response rate itself may have depended heavily on the characteristics of patients enrolled, in ways that cannot be adequately assessed. In particular, no analysis by distance traveled to the treatment center was available. And, as is always the case in non-comparative studies, no reliable or comprehensive assessment of overall survival, time to progression, or toxicity compared with, say, best available therapy was possible. Moreover, results from two phase III trials using Iressa in combination with standard chemotherapy as first-line therapy were strikingly negative (3). Nevertheless, both FDA’s Oncologic Drugs Advisory Committee and, eventually, the FDA itself were convinced that the 10% response rate in patients with advanced disease refractory to standard therapy represented a clinically important finding and that the results were "reasonably likely" to predict patient benefit. More definitive studies are already underway to confirm or deny this conclusion, as required as part of the granting of accelerated approval, but in the meantime Iressa is approved for third-line treatment of advanced NSCLC, and a large number of patients will undoubtedly receive it without much reliable evidence of its risk–benefit profile.

These considerations suggest that the results of phase II trials need to be interpreted with great caution and that their use in the regulatory process can be especially problematic. Randomized trials comparing a new regimen against a control or placebo offer the best protection against the risks of erroneous conclusions inherent in phase II trials, especially for regulatory decisions. Accelerated approval may provide earlier access to potentially beneficial agents, but the evidence required for granting it should not come solely from phase II trials.

REFERENCES

1 Lamont EB, Hayreh D, Pickett KE, Dignam JJ, List MA, Stenson KM, et al. Is patient travel distance associated with survival on phase II clinical trials in oncology? J Natl Cancer Inst 2003;95:1370–5.[CrossRef]

2 Hirschfeld S, Pazdur R. Oncology drug development: United States Food and Drug Administration perspective. Crit Rev Oncol Hematol 2002;42:137–43.[ISI][Medline]

3 U.S. Food and Drug Administration. Oncologic Drugs Advisory Committee Briefing Document. September 24, 2002. Available at: http://www.fda.gov/ohrms/dockets/ac/02/briefing/3894b1.htm. [Last accessed: August 11, 2003.]


This article has been cited by other articles in HighWire Press-hosted journals:


             
Copyright © 2003 Oxford University Press (unless otherwise stated)
Oxford University Press Privacy Policy and Legal Statement