Substantial Overestimation of Standard Errors of Relative Survival Rates of Cancer Patients
Hermann Brenner1 and
Timo Hakulinen2
1 German Centre for Research on Ageing, Heidelberg, Germany
2 Finnish Cancer Registry, Helsinki, Finland
Correspondence to Dr. Hermann Brenner, Department of Epidemiology, German Centre for Research on Ageing, Bergheimer Str. 20, D-69115 Heidelberg, Germany (e-mail: brenner{at}dzfa.uni-heidelberg.de).
Received for publication September 27, 2004.
Accepted for publication December 9, 2004.
 |
ABSTRACT
|
---|
Relative survival rates are among the most commonly reported outcome measures of cancer patients. They are calculated as ratios of observed survival rates and the expected survival rates in the absence of cancer. Standard errors of relative survival rates are commonly calculated by dividing the standard error for absolute survival rates by the expected survival, without taking possible random variation of the latter into account. The aim of this study was to empirically assess the validity of these commonly reported standard errors. Using data from the nationwide Finnish Cancer Registry, the authors calculated 5- and 10-year absolute, expected, and relative survival rates for patients with 25 common forms of cancer in Finland in 1989. The authors used bootstrap analysis to empirically assess the random error of absolute and relative survival rates and then compared the results with conventionally derived estimates of standard errors. The conventional and bootstrap standard errors were closely similar for all estimates of absolute survival. By contrast, the conventional estimates of standard errors of 5- and 10-year relative survival exceeded the bootstrap estimates by up to 17% and 32%, respectively. The authors conclude that conventional derivation may substantially overestimate standard errors for relative survival.
epidemiologic methods; registries; survival
 |
INTRODUCTION
|
---|
Relative survival rates are commonly reported by cancer registries. They are calculated as ratios of observed survival rates and the expected survival rates in the absence of cancer (1
). The latter are typically derived from life tables of the general population. Standard errors of relative survival rates are commonly calculated by dividing the standard error for absolute survival rates by the expected survival, without taking possible random variation of the latter into account (2
4
). The rationale for this procedure is that random variation in the expected survival can be neglected, as the database underlying the life tables from which it is derived is usually very large. This type of reasoning neglects the fact, however, that expected survival of a sample of patients is still subject to random variation due to random variation of the age distribution of the sample even if random variation of the life table estimates is negligible. Moreover, random errors of absolute and expected survival may often be positively correlated, because the age distribution of a sample of cancer patients may affect estimates of absolute and expected survival in a similar manner, as both absolute and expected survival are typically decreasing with increasing age. The aim of this study was to empirically assess the random error of expected survival rates resulting from random variation in the age distribution of cancer patients, its correlation with the random error of absolute survival rates, and the random error of relative survival rates, as well as to compare the results with estimates of standard errors obtained by neglecting random variation in expected survival rates altogether.
 |
MATERIALS AND METHODS
|
---|
This analysis is based on data from the nationwide Finnish Cancer Registry. This population-based registry has been operating for more than 50 years, covers the whole population of Finland (about 5.3 million people), and is well known for its data quality and completeness (5
). Our empirical analysis was carried out among patients aged 15 or more years with a first diagnosis of cancer in 1989. We calculated 5- and 10-year absolute, expected, and relative survival rates and their standard errors for each of 25 common forms of cancer separately. Expected survival rates were calculated from age-, sex-, and calendar period-specific population life tables of Finland (which were assumed to have negligible random error) using Hakulinen's method (6
).
The standard errors of survival rates were determined in two ways. First, the standard error of the absolute survival rates was calculated according to Greenwood's method (7
), the standard errors of the expected survival rates were assumed to be zero, and the standard errors of the relative survival rates were obtained by dividing the standard errors of the absolute survival rates by the expected survival rates. This approach reflects common practice in the analysis of population-based cancer registry data. Second, all standard errors were estimated nonparametrically by bootstrap analysis as the standard deviation of the respective point estimates obtained in 10,000 bootstrap samples (full resamples) for each cancer site (8
). The standard errors obtained by the bootstrap analysis and those obtained by the standard procedure were compared by calculating their ratios. Furthermore, Pearson's correlation coefficients of the point estimates of absolute and expected survival rates in the bootstrap samples were calculated to assess the correlation of sampling errors in the absolute and expected survival rates.
All survival analyses were carried out with publicly available SAS (SAS Institute, Inc., Cary, North Carolina) macros for both absolute and relative survival rates (9
, 10
), which were extended to allow for bootstrap analyses (a copy of the extended program may be obtained from the first author by request).
 |
RESULTS
|
---|
Table 1 shows the numbers and age distribution of cancer patients included in the analysis by cancer site. Breast cancer was the most common form of incident cancer in Finland in 1989, followed by lung cancer, prostate cancer, stomach cancer, and colon cancer. The median age at diagnosis was above 60 years for most of the cancers included in the analysis, and it ranged from 74 years for patients with prostate cancer to 35 years for patients with cancer of the testis. Furthermore, there was a wide age range of patients even within cancer sites.
View this table:
[in this window]
[in a new window]
|
TABLE 1. Numbers and age distribution of patients aged 15 or more years with a first diagnosis of common forms of cancer, Finland, 1989
|
|
The point estimates and their standard errors of 5- and 10-year absolute survival rates are shown in table 2. Prognosis strongly varied by cancer site, with 5-year and 10-year absolute survival rates ranging from 88.5 percent and 86.9 percent for patients with testicular cancer to 1.4 percent and 1.1 percent for patients with pancreatic cancer. Standard errors for 5-year and 10-year absolute survival rates also strongly vary by cancer site. The standard errors determined by Greenwood's method and by bootstrap analysis are generally very similar (within ±1 percent in all cases for 5-year absolute survival rates and in 21 of 25 cases for 10-year absolute survival rates).
View this table:
[in this window]
[in a new window]
|
TABLE 2. Absolute and expected survival after 5 years and 10 years of patients aged 15 or more years with a first diagnosis of common forms of cancer, Finland, 1989
|
|
Table 2 also shows the point estimates and their standard errors according to bootstrap analysis of 5-year and 10-year expected survival rates. The point estimates are highest for patients with testicular cancer because of their relatively young age and lowest for patients with prostate cancer because of their relatively old age. Although the bootstrap estimates of the standard errors of expected survival are lower than those of absolute survival for most forms of cancer, they are not entirely negligible and range from 0.31 to 2.24 percent and from 0.52 to 3.48 percent for 5-year and 10-year expected survival, respectively. Table 2 also shows that the correlation between absolute and expected survival rates in the bootstrap samples is substantial. Pearson's correlation coefficients varied from 0.11 to 0.76 for 5-year survival and from 0.09 to 0.76 for 10-year survival. The highest correlations are seen for cancers of the testis, thyroid gland, cervix and corpus uteri, and urinary bladder and for melanoma.
As table 3 shows, differences between standard errors for 5-year and 10-year relative survival rates derived by the standard procedure and by bootstrap analysis are often much larger than the very small differences seen for 5-year and 10-year absolute survival rates. With few exceptions, the standard errors obtained by the standard procedure exceed the standard errors obtained by bootstrap analysis. The former are up to 17 percent higher than the latter for 5-year relative survival, and the differences range up to 32 percent for 10-year relative survival rates. For nine of 25 cancers, the differences in standard errors of 5-year relative survival are 5 percent or higher. For three cancers, the differences are 10 percent or higher. Pertinent differences in standard errors of 10-year relative survival rates are seen for 15 and seven forms of cancer, respectively. The differences between the two types of standard errors are largest for those cancers with the highest correlation between absolute and expected survival rates, most notably for cancers of the thyroid gland and the testis.
View this table:
[in this window]
[in a new window]
|
TABLE 3. Relative survival after 5 years and 10 years of patients aged 15 or more years with a first diagnosis of common forms of cancer, Finland, 1989
|
|
To assess reproducibility of the observed patterns in different samples, we repeated all analyses separately for patients diagnosed in 1988 and 1987. Despite some variation in the survival rates between the years, the patterns regarding the standard errors were generally very similar, and results are therefore not shown separately to save space.
 |
DISCUSSION
|
---|
This analysis demonstrates that the standard errors of relative survival rates may be overestimated to some nonnegligible extent with traditional methods of analysis. Potential overestimation appears to be more pronounced for 10-year relative survival rates than for 5-year relative survival rates, and it ranged up to 32 percent in our empirical evaluation for patients diagnosed with 25 common forms of cancer in Finland in 1989.
The rationale behind traditional estimation of the standard error of relative survival rates is that random variation in expected survival rates is negligible for practical purposes, as the latter are determined from population mortality figures that are typically based on very large numbers. However, even though random variation of the latter can typically be neglected (an assumption also made in our bootstrap analyses), the expected survival of a sample of cancer patients is still dependent on the age distribution of this sample, which is subject to random variation. As a result, there is also random variation in the expected survival of samples of cancer patients.
In theory, the additional random variation in expected survival rates might increase or reduce the standard error of relative survival rates (or leave them unchanged). More formally, let A and E denote absolute and expected survival with variances var(A) and var(E), respectively, and covariance cov(A,E). Then, applying the delta method, the variance of relative survival, var(R), is given as
 |
If random variation in absolute and expected survival rates were independent or negatively correlated, one might expect random error of relative survival rates to be higher than in the absence of random error in expected survival rates. However, a positive correlation in random variation of absolute and expected survival rates was observed for all forms of cancer assessed in this analysis. This result is not surprising, because both absolute and expected survival rates decrease with increasing age, and random variation in the age distribution of the samples of cancer patients would typically alter absolute and expected survival rates in the same direction. According to our analyses, this positive correlation is strong enough that relative survival rates typically have lower standard errors than suggested by conventional modes of calculation, which ignore random variation in expected survival rates altogether.
We are not aware of a possibility to calculate standard errors of relative survival rates that takes the random variation of both absolute and expected survival rates and their correlation into account, in a straightforward manner. Such standard errors can be empirically derived, however, by resampling techniques, of which bootstrap analysis is a particularly useful method (8
). Application of resampling techniques in this context requires that the contributions of all patients to the calculations of expected survival are determined on an individual basis, an approach used in our recently developed computer programs for relative survival (9
, 10
), and not on an aggregate basis, as it had mostly been done in previous commonly used computer programs, such as that by Hakulinen and Abeywickrama (11
).
A disadvantage of resampling techniques is that a very large number of replications is necessary to come up with reliable estimates of standard errors, requiring substantial computing time. In our analyses, we decided to use 10,000 replications per analysis to ensure very high levels of reliability. The almost perfect agreement of standard errors obtained by Greenwood's formula and by bootstrap analysis for the absolute survival rates in all cases reassures us that this number of replications is sufficient to come up with reliable bootstrap estimates. It took about 4 hours to run all the bootstrap analyses shown in this paper on a standard laptop (Pentium 4 processor (Intel Corporation, Santa Clara, California), 3.06 GHz, 512 MB RAM). Obviously, there is a tradeoff between computation time and precision of the estimates, and lower numbers of bootstrap replications may often be sufficient for practical purposes. When the analyses were repeated with 1,000 bootstrap replications per analysis, the agreement of standard errors obtained by Greenwood's formula and by bootstrap analysis for the absolute survival rates was still within ±5 percent in all cases (and within ±2 percent in 40 of 50 cases), and the computing time was reduced to about half an hour. Although these computing times appear to be quite long, computing time most likely will further rapidly decrease with more powerful hardware equipment in the future, and the relevance of computing time (which can usually be easily allocated during nights or weekends) appears to be very small if compared with the time and effort it takes to collect pertinent cancer registry data.
In summary, our analyses suggest that conventional estimates of standard errors for relative survival rates may often be too high, particularly for long-term relative survival rates. Although more time and computer intensive, bootstrap analysis may be a useful tool to estimate standard errors of relative survival rates.
 |
References
|
---|
- Ederer F, Axtell LM, Cutler SJ. The relative survival rate: a statistical methodology. Natl Cancer Inst Monogr 1961;6:10121.
- Parkin DM, Hakulinen T. Cancer registration: principles and methods. Analysis of survival. IARC Sci Publ 1991;(95):15976.[Medline]
- Estève J, Benhamou E, Raymond L. Statistical methods in cancer research. Vol IV. Descriptive epidemiology. IARC Sci Publ 1994;(128):1302.[Medline]
- Mathew A, Pandey M. Attributing death to cancer: cause-specific survival estimation. J Postgrad Med 2002;48:3226.[Medline]
- Teppo L, Pukkala E, Lehtonen M. Data quality and quality control of a population-based cancer registry. Experience in Finland. Acta Oncol 1994;33:3659.[ISI][Medline]
- Hakulinen T. Cancer survival corrected for heterogeneity in patient withdrawal. Biometrics 1982;39:93342.
- Greenwood M. The natural duration of cancer. In: Reports on public health and medical subjects. London, England: His Majesty's Stationery Office, 1926:126.
- Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci 1986;1:5477.
- Brenner H, Hakulinen T, Gefeller O. Computational realization of period analysis for monitoring cancer patient survival. Epidemiology 2002;13:61112.[CrossRef][ISI][Medline]
- Arndt V, Talbäck M, Gefeller O, et al. Modification of SAS macros for more efficient analysis of relative survival rates. Eur J Cancer 2004;40:7789.[CrossRef][ISI][Medline]
- Hakulinen T, Abeywickrama KH. A computer program package for relative survival analysis. Comput Programs Biomed 1985;19:197207.[CrossRef][ISI][Medline]