Meta-Analysis Group, MRC Clinical Trials Unit, London, UK.
C Vale, Meta-Analysis Group, MRC Clinical Trials Unit, 222 Euston Road, London NW1 2DA, UK. E-mail: cv{at}ctu.mrc.ac.uk
Abstract
Background Systematic reviews of published time-to-event outcomes commonly rely on calculating odds ratios (OR) at fixed points in time and where actual numbers at risk are not presented. These estimates are usually based on the total numbers included in the published analysis and take no account of censoring. We have assessed the impact of adjusting for censoring on weighting, estimates and statistical heterogeneity of meta-analyses in cancer.
Methods Meta-analyses of survival data for five meta-analyses of published trials in cancer were conducted. The OR and associated statistics were calculated based on unadjusted total numbers of participants and events. These were compared with calculations that first adjusted the numbers at risk for censoring using a simple model.
Results Pooled OR were changed in 17/24 cases. On average, there was a 2.6% difference between the adjusted and unadjusted OR. Confidence intervals were frequently wider for the adjusted OR. Adjusting also reduced weighting of individual trials with immature follow-up. In 18/24 cases, adjusting reduced statistical heterogeneity and affected the associated P-values.
Conclusions Reviewers conducting meta-analyses of published time-to-event data where actual numbers at risk are not available should adjust the numbers at risk, estimated from total numbers analysed, to account for immature data and censoring.
Keywords Meta-analysis, randomized controlled trials, neoplasms, survival analysis, censoring, follow-up, time-to-event
Accepted 25 September 2001
Time-to-event outcomes, including time to death, time to disease progression or time to recovery are important in many areas of healthcare. They take into account not only that an event has occurred, but also the time at which it took place. A key characteristic of time-to-event data is censoring (reviewed in ref. 1), where incomplete information is available, for example where a participant leaves the study before it ends, or dies of a cause unrelated to the study, and so becomes lost to follow-up. Alternatively, if an event has not taken place, then a patient may be censored on the date of last follow-up. For meta-analyses of these outcomes, the most appropriate statistic to use is the hazard ratio (HR), which takes into account both the number of events and the time to these events, and also the data from those patients who have been censored. However, this is possible only if individual patient data (IPD) has been collected, or if particular statistics (log hazard ratio and variance) have been reported. Alternatively, the HR can be estimated from a variety of summary statistics provided that sufficient information is presented in the trial report.2 In practice, the former is quite rare, and the latter is often difficult because the necessary statistics are frequently absent from trial reports. More commonly, meta-analyses of time-to-event outcomes rely on estimating the odds ratios (OR) at fixed points in time, or a series of fixed time points. Numbers of patients dead and alive at time points of interest are usually estimated using readings from survival curves. If no (or very few) patients are lost to follow-up then observed and expected deaths can be calculated using the total numbers of patients in the trial. However, censoring patterns will vary from trial to trial. Where significant numbers of patients have been censored then this should be accounted for, otherwise the comparability, and possibly the reliability, of the individual trial results may be affected. This in turn could impact on the quality of any meta-analysis of these trials.
Methods
In order to look at the effects of adjusting for follow-up, meta-analyses of time-to-event outcomes were conducted in bladder, lung and cervix cancer and in soft tissue sarcoma (Table 1). Systematic searches for all known trials had already been carried out.36 However, for each meta-analysis, only trials for which data were available for all time-points were used, such that the number of trials at each time-point was consistent, thus isolating the effect of adjusting for censoring. Survival data at 1, 2, 3, 4 and 5 years were extracted either from the text or estimated from the survival curves in trial reports.
|
Box 1 Method for adjusting for censoring If the chosen time-point for the meta-analysis is T1 where Fmin< T1 N = total patients in trial/analysis Fmin = minimum follow-up Fmax = maximum follow-up T1 = chosen time point of meta-analysis Q = proportion of patients with complete follow-up (dead or alive) until T 1 n = effective number of patients followed until T 1 Then:
If the meta-analysis is performed at T2 in cases where T2 > Fmax
|
Results
Effects on estimates and precision
In 17/24 of the meta-analyses conducted, adjusting for variable follow-up had modest effects on the OR (average = 2.6%, range 19%) and associated CI (Table 2). The most extreme example was seen in the meta-analysis of adjuvant chemotherapy for advanced bladder cancer, the OR for the unadjusted data was 0.79 (95% CI : 0.581.08), whereas the adjusted OR was 0.88 (95% CI : 0.821.13); a 9% difference in the risk of dying on treatment versus control. However, neither result was conventionally significant.
|
|
|
Effects on statistical heterogeneity
For most time points, in each meta-analysis, adjusting for variable follow-up reduced the level of statistical heterogeneity. In the cervix cancer meta-analysis, the level of heterogeneity was unacceptably high at most time points, irrespective of whether the data was adjusted (2 = 7.9415.54, P = 0.0080.16) or not (
2 = 6.9313.95, P = 0.020.23). However, in the soft tissue sarcoma meta-analysis, adjusting for variable follow-up not only reduced the level of heterogeneity but could have influenced the decision on whether to pool these trials. For example if a 10% cut-off had been chosen for the test for heterogeneity (as is common due to its low power), based on the unadjusted 3-year (P = 0.06) or 5-year (P = 0.09) analysis, the trials would not have been pooled. Instead a positive decision to derive a pooled OR would have been made in the adjusted 3-year (P = 0.12) or 5-year (P = 0.19) analysis.
Discussion
Randomized trials that examine time-to-event outcomes are likely to be published with different ranges of follow-up. This may occur where a therapeutic question has been studied over many years, or where some of the trials have reported preliminary results. Simply using the number of events and patients at risk at different time points to calculate individual and pooled OR, assumes that all follow-up is equal and complete to the relevant time-point. This method is therefore, over-simplistic and perhaps unreliable. It has been argued that the methodological problems associated with meta-analyses of time-to-event outcomes are sufficient to argue the case for collecting and analysing individual patient data,8,9 however this may not always be possible. Where a meta-analysis of published data is the only practical option, simple methods like that described here, which take account of censoring, may provide a useful means of standardization across trials. Using details of the minimum and maximum follow-up for each trial to adjust the numbers at risk, weights trials appropriately according to the reliable information they contribute. This means, for example, that a large trial with poor follow-up is not given undue weight, as would otherwise be the case. The method also ensures that calculated OR, CI and P-values for individual trials and pooled over all trials, reflect the uncertainty of trial results that have been extrapolated to distant time points. Because it is a more conservative approach, increasing the overall level of uncertainty within a trial has the secondary effect of reducing the level of statistical heterogeneity between trials.
An alternative and potentially more reliable approach is to use reported numbers at risk to estimate the levels of censoring. However, only 8/24 included in these meta-analyses reported the actual numbers at risk. We would therefore recommend that authors of papers presenting randomized controlled trials should routinely report the numbers at risk for all time-points.
Conclusions
Where time-to-event outcomes are of interest in a meta-analysis and where they are likely to be affected by the duration of follow-up, we recommend that some method be used to take variable censoring into account. This should provide a more conservative, but more reliable answer to the question posed.
KEY MESSAGES
|
Acknowledgments
The authors would like to thank the UK Medical Research Council for funding this project.
References
1 Leung K-M, Elashoff R, Afifi A. Censoring issues in survival analysis. Annu Rev Public Health 1997;18:83104.[CrossRef][ISI][Medline]
2 Parmar MKB, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med 1998;17:281534.[CrossRef][ISI][Medline]
3 Advanced Bladder Cancer Overview Collaboration. Does neoadjuvant cisplatin-based chemotherapy improve the survival of patients with locally advanced bladder cancer: a meta-analysis of individual patient data from randomised clinical trials. Br J Urol 1995;75:20613.[ISI][Medline]
4 Parmar MKB, Burdett S. Neoadjuvant and adjuvant chemotherapy. In: Hall RR (ed.). Clinical Management of Bladder Cancer. 1st Edn. London: Arnold, 1999, pp.24963.
5 PORT Meta-analysis Trialists Group. Postoperative radiotherapy in non-small cell lung cancer: systematic review and meta-analysis of individual patient data from nine randomised controlled trials. Lancet 1998;352:25763.[CrossRef][ISI][Medline]
6 Tierney JF, on behalf of the Sarcoma Meta-analysis Collaboration (SMAC). A meta-analysis using individual patient data from randomised clinical trials (RCTs) of adjuvant chemotherapy for soft-tissue sarcomas (STS). Proc Am Soc Clin Oncol 1996;15:2024.
7 Early Breast Cancer Trialists Collaborative Group. Treatment of Early Breast Cancer Vol. 1. Worldwide Evidence 19851990. Oxford: Oxford University Press, 1990.
8 Tierney JF, Stewart LA, Parmar MKB. Can the published data tell us about the effectiveness of neoadjuvant chemotherapy for locally advanced cancer of the uterine cervix? Eur J Cancer 1999;35:40609.[CrossRef][ISI][Medline]
9 Stewart LA, Parmar MKB. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet 1993;341: 41822.[ISI][Medline]