Effects of adjusting for censoring on meta-analyses of time-to-event outcomes

Claire L Vale, Jayne F Tierney and Lesley A Stewart

Meta-Analysis Group, MRC Clinical Trials Unit, London, UK.

C Vale, Meta-Analysis Group, MRC Clinical Trials Unit, 222 Euston Road, London NW1 2DA, UK. E-mail: cv{at}ctu.mrc.ac.uk

Abstract

Background Systematic reviews of published time-to-event outcomes commonly rely on calculating odds ratios (OR) at fixed points in time and where actual numbers at risk are not presented. These estimates are usually based on the total numbers included in the published analysis and take no account of censoring. We have assessed the impact of adjusting for censoring on weighting, estimates and statistical heterogeneity of meta-analyses in cancer.

Methods Meta-analyses of survival data for five meta-analyses of published trials in cancer were conducted. The OR and associated statistics were calculated based on unadjusted total numbers of participants and events. These were compared with calculations that first adjusted the numbers at risk for censoring using a simple model.

Results Pooled OR were changed in 17/24 cases. On average, there was a 2.6% difference between the adjusted and unadjusted OR. Confidence intervals were frequently wider for the adjusted OR. Adjusting also reduced weighting of individual trials with immature follow-up. In 18/24 cases, adjusting reduced statistical heterogeneity and affected the associated P-values.

Conclusions Reviewers conducting meta-analyses of published time-to-event data where actual numbers at risk are not available should adjust the numbers at risk, estimated from total numbers analysed, to account for immature data and censoring.

Keywords Meta-analysis, randomized controlled trials, neoplasms, survival analysis, censoring, follow-up, time-to-event

Accepted 25 September 2001

Time-to-event outcomes, including time to death, time to disease progression or time to recovery are important in many areas of healthcare. They take into account not only that an event has occurred, but also the time at which it took place. A key characteristic of time-to-event data is censoring (reviewed in ref. 1), where incomplete information is available, for example where a participant leaves the study before it ends, or dies of a cause unrelated to the study, and so becomes lost to follow-up. Alternatively, if an event has not taken place, then a patient may be censored on the date of last follow-up. For meta-analyses of these outcomes, the most appropriate statistic to use is the hazard ratio (HR), which takes into account both the number of events and the time to these events, and also the data from those patients who have been censored. However, this is possible only if individual patient data (IPD) has been collected, or if particular statistics (log hazard ratio and variance) have been reported. Alternatively, the HR can be estimated from a variety of summary statistics provided that sufficient information is presented in the trial report.2 In practice, the former is quite rare, and the latter is often difficult because the necessary statistics are frequently absent from trial reports. More commonly, meta-analyses of time-to-event outcomes rely on estimating the odds ratios (OR) at fixed points in time, or a series of fixed time points. Numbers of patients dead and alive at time points of interest are usually estimated using readings from survival curves. If no (or very few) patients are lost to follow-up then observed and expected deaths can be calculated using the total numbers of patients in the trial. However, censoring patterns will vary from trial to trial. Where significant numbers of patients have been censored then this should be accounted for, otherwise the comparability, and possibly the reliability, of the individual trial results may be affected. This in turn could impact on the quality of any meta-analysis of these trials.

Methods

In order to look at the effects of adjusting for follow-up, meta-analyses of time-to-event outcomes were conducted in bladder, lung and cervix cancer and in soft tissue sarcoma (Table 1Go). Systematic searches for all known trials had already been carried out.3–6 However, for each meta-analysis, only trials for which data were available for all time-points were used, such that the number of trials at each time-point was consistent, thus isolating the effect of adjusting for censoring. Survival data at 1, 2, 3, 4 and 5 years were extracted either from the text or estimated from the survival curves in trial reports.


View this table:
[in this window]
[in a new window]
 
Table 1 Characteristics of the meta-analyses (Note: For the purposes of this study only those trials for which data were available at all time points were included)
 
Few methods have been reported to adjust the numbers of patients at risk to allow for variable follow-up. These are rarely applied in meta-analysis of published data. We have used one of these methods2 to adjust for variable follow-up in the five meta-analyses and examined the effects on the results. The method is based upon a simple model that assumes the rate of censoring is constant and non-informative, and allows the effective number of patients at risk to be calculated. In effect the method sums the partial follow-up of all patients in a trial to obtain an adjusted (reduced) total number at risk. For example, if 100 patients are censored at a constant rate over a year, the average follow-up will be 6 months, and the adjusted total number of patients at risk is 50. The method is described in Box 1Go and using different notation in Parmar et al.2 The OR and associated statistics were derived for each of the meta-analyses in cancer, using the numbers of patients that had died and the numbers at risk. Using the model described above, these calculations were then repeated after first adjusting (reducing) the number of patients at risk, based on the extent of follow-up. Adjusted OR and associated statistics were derived.7


Box 1 Method for adjusting for censoring

If the chosen time-point for the meta-analysis is T1 where Fmin< T1 <= Fmax

N = total patients in trial/analysis

Fmin = minimum follow-up

Fmax = maximum follow-up

T1 = chosen time point of meta-analysis

Q = proportion of patients with complete follow-up (dead or alive) until T 1

n = effective number of patients followed until T 1

Then:



If the meta-analysis is performed at T2 in cases where T2 > Fmax


 

Results

Effects on estimates and precision
In 17/24 of the meta-analyses conducted, adjusting for variable follow-up had modest effects on the OR (average = 2.6%, range 1–9%) and associated CI (Table 2Go). The most extreme example was seen in the meta-analysis of adjuvant chemotherapy for advanced bladder cancer, the OR for the unadjusted data was 0.79 (95% CI : 0.58–1.08), whereas the adjusted OR was 0.88 (95% CI : 0.82–1.13); a 9% difference in the risk of dying on treatment versus control. However, neither result was conventionally significant.


View this table:
[in this window]
[in a new window]
 
Table 2 Summary of results (OR, 95% CI and P-values) at all time points for all meta-analyses
 
A more dramatic difference between the unadjusted and adjusted data was observed in the meta-analysis of post-operative radiotherapy for non-small cell lung cancer (Figure 1Go). The OR for the unadjusted meta-analysis was 1.28 (95% CI : 0.96–1.70), indicating a 28% increase in the risk of death with this treatment. This was not a statistically significant result (P = 0.10). However, the adjusted OR of 1.37 (95% CI : 1.00–1.89) suggested a 37% increase in the risk of death. This result was conventionally significant (P = 0.05). In this case, adjusting for variable follow-up had the potential to alter both the size of the effect and its interpretation.



View larger version (14K):
[in this window]
[in a new window]
 
Figure 1 The impact of adjusting for censoring on the results of the meta-analysis of post-operative radiotherapy in lung cancer (results at 5 years)

 
Effects on weighting of trials
Adjusting for variable follow-up can have a marked effect on the contribution of individual trial results to a meta-analysis, its weighting. Obviously, a trial with poor follow-up will be affected to a greater extent by adjusting than a trial with good follow-up. In the meta-analysis of neoadjuvant chemotherapy for locally advanced bladder cancer, one of the trials, which was relatively small, had a minimum follow-up of 48 months. Owing to its good follow-up, relative to other trials in the meta-analysis, adjusting had the effect of increasing the contribution made by this trial to the pooled meta-analysis result (Figure 2aGo).



View larger version (36K):
[in this window]
[in a new window]
 
Figure 2 The impact of adjusting for censoring on the weight of an individual trial result in (a) a meta-analysis of neoadjuvant chemotherapy for bladder cancer and (b) a meta-analysis of adjuvant chemotherapy in soft tissue sarcoma

 
In contrast, one of the larger trials in the meta-analysis of adjuvant chemotherapy in soft tissue sarcoma had a minimum follow-up of just 10 months and a maximum of 70 months. Once adjusted, the relative weight of this trial decreased in the meta-analysis, most notably at the later time points (Figure 2bGo).

Effects on statistical heterogeneity
For most time points, in each meta-analysis, adjusting for variable follow-up reduced the level of statistical heterogeneity. In the cervix cancer meta-analysis, the level of heterogeneity was unacceptably high at most time points, irrespective of whether the data was adjusted ({chi}2 = 7.94–15.54, P = 0.008–0.16) or not ({chi}2 = 6.93–13.95, P = 0.02–0.23). However, in the soft tissue sarcoma meta-analysis, adjusting for variable follow-up not only reduced the level of heterogeneity but could have influenced the decision on whether to pool these trials. For example if a 10% cut-off had been chosen for the test for heterogeneity (as is common due to its low power), based on the unadjusted 3-year (P = 0.06) or 5-year (P = 0.09) analysis, the trials would not have been pooled. Instead a positive decision to derive a pooled OR would have been made in the adjusted 3-year (P = 0.12) or 5-year (P = 0.19) analysis.

Discussion

Randomized trials that examine time-to-event outcomes are likely to be published with different ranges of follow-up. This may occur where a therapeutic question has been studied over many years, or where some of the trials have reported preliminary results. Simply using the number of events and patients at risk at different time points to calculate individual and pooled OR, assumes that all follow-up is equal and complete to the relevant time-point. This method is therefore, over-simplistic and perhaps unreliable. It has been argued that the methodological problems associated with meta-analyses of time-to-event outcomes are sufficient to argue the case for collecting and analysing individual patient data,8,9 however this may not always be possible. Where a meta-analysis of published data is the only practical option, simple methods like that described here, which take account of censoring, may provide a useful means of standardization across trials. Using details of the minimum and maximum follow-up for each trial to adjust the numbers at risk, weights trials appropriately according to the reliable information they contribute. This means, for example, that a large trial with poor follow-up is not given undue weight, as would otherwise be the case. The method also ensures that calculated OR, CI and P-values for individual trials and pooled over all trials, reflect the uncertainty of trial results that have been extrapolated to distant time points. Because it is a more conservative approach, increasing the overall level of uncertainty within a trial has the secondary effect of reducing the level of statistical heterogeneity between trials.

An alternative and potentially more reliable approach is to use reported numbers at risk to estimate the levels of censoring. However, only 8/24 included in these meta-analyses reported the actual numbers at risk. We would therefore recommend that authors of papers presenting randomized controlled trials should routinely report the numbers at risk for all time-points.

Conclusions

Where time-to-event outcomes are of interest in a meta-analysis and where they are likely to be affected by the duration of follow-up, we recommend that some method be used to take variable censoring into account. This should provide a more conservative, but more reliable answer to the question posed.


KEY MESSAGES

  • Meta-analyses of time-to-event outcomes are likely to be affected by censoring and by the duration of follow-up of individual trials.
  • Where actual numbers at risk are not reported, a simple method, based on the minimum and maximum follow-up, exists that can adjust the numbers at risk to reflect censoring. This provides a more conservative but more reliable estimate of effect.
  • For meta-analyses of time-to-event outcomes, some method should be used to adjust for censoring.
  • We recommend that publications of randomized controlled trials should routinely report the actual numbers at risk for each time point.

 

Acknowledgments

The authors would like to thank the UK Medical Research Council for funding this project.

References

1 Leung K-M, Elashoff R, Afifi A. Censoring issues in survival analysis. Annu Rev Public Health 1997;18:83–104.[CrossRef][ISI][Medline]

2 Parmar MKB, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med 1998;17:2815–34.[CrossRef][ISI][Medline]

3 Advanced Bladder Cancer Overview Collaboration. Does neoadjuvant cisplatin-based chemotherapy improve the survival of patients with locally advanced bladder cancer: a meta-analysis of individual patient data from randomised clinical trials. Br J Urol 1995;75:206–13.[ISI][Medline]

4 Parmar MKB, Burdett S. Neoadjuvant and adjuvant chemotherapy. In: Hall RR (ed.). Clinical Management of Bladder Cancer. 1st Edn. London: Arnold, 1999, pp.249–63.

5 PORT Meta-analysis Trialists Group. Postoperative radiotherapy in non-small cell lung cancer: systematic review and meta-analysis of individual patient data from nine randomised controlled trials. Lancet 1998;352:257–63.[CrossRef][ISI][Medline]

6 Tierney JF, on behalf of the Sarcoma Meta-analysis Collaboration (SMAC). A meta-analysis using individual patient data from randomised clinical trials (RCTs) of adjuvant chemotherapy for soft-tissue sarcomas (STS). Proc Am Soc Clin Oncol 1996;15:2024.

7 Early Breast Cancer Trialists Collaborative Group. Treatment of Early Breast Cancer Vol. 1. Worldwide Evidence 1985–1990. Oxford: Oxford University Press, 1990.

8 Tierney JF, Stewart LA, Parmar MKB. Can the published data tell us about the effectiveness of neoadjuvant chemotherapy for locally advanced cancer of the uterine cervix? Eur J Cancer 1999;35:406–09.[CrossRef][ISI][Medline]

9 Stewart LA, Parmar MKB. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet 1993;341: 418–22.[ISI][Medline]