Utility of the Chronic Disease Score and Charlson Comorbidity Index as Comorbidity Measures for Use in Epidemiologic Studies of Antibiotic-resistant Organisms
Jessina C. McGregor1 ,
Peter W. Kim2,
Eli N. Perencevich1,3,
Douglas D. Bradham1,3,
Jon P. Furuno1,
Keith S. Kaye4,
Jeffrey C. Fink1,5,
Patricia Langenberg1,
Mary-Claire Roghmann1,3 and
Anthony D. Harris1,3
1 Department of Epidemiology and Preventive Medicine, School of Medicine, University of Maryland, Baltimore, MD.
2 Division of Anti-Infective Drug Products, Office of Drug Evaluation IV, Center for Drug Evaluation and Research, Food and Drug Administration, Rockville, MD.
3 VA MD Health Care System, Baltimore, MD.
4 Department of Medicine, Duke University Medical Center, Durham, NC.
5 Division of Nephrology, Department of Medicine, University of Maryland Medical Center, Baltimore, MD.
Received for publication August 5, 2004; accepted for publication October 7, 2004.
 |
ABSTRACT
|
---|
Comorbidity is a known risk factor for antibiotic-resistant bacterial infections. Although aggregate comorbidity measures are useful in epidemiologic research, none of the existing measures was developed for use with this outcome. This study compared the utility of two comorbidity measures, the Charlson Comorbidity Index and the Chronic Disease Score, in assessing the comorbidity-attributable risk of nosocomial infections with methicillin-resistant Staphylococcus aureus (MRSA) or vancomycin-resistant enterococci (VRE). Two case-control studies were conducted at the University of Maryland Medical System in Baltimore, Maryland. Cases were inpatients with a first positive clinical culture of MRSA or VRE at least 48 hours postadmission (July 1, 1998July 1, 2001). Three inpatient controls were randomly selected per case. The MRSA study included 2,164 patients, and the VRE study included 1,948. The scores discrimination and calibration were measured by using the c statistic and Hosmer-Lemeshow chi-square test. The Charlson Comorbidity Index (c = 0.653) and Chronic Disease Score (c = 0.608) were similar discriminators of MRSA and VRE (c = 0.670 and c = 0.647, respectively). Calibration of the scores was poor for both outcomes (p < 0.05). A revised comorbidity measure specific to resistant infections would likely provide a better assessment of the comorbidity-attributable risk of antibiotic-resistant infections.
comorbidity; drug resistance, bacterial; predictive value of tests; ROC curve; sensitivity and specificity
Abbreviations:
ICD-9, International Classification of Diseases, Ninth Revision; MRSA, methicillin-resistant Staphylococcus aureus; ROC, receiver operator characteristic; UMMS, University of Maryland Medical System; VRE, vancomycin-resistant enterococci.
 |
INTRODUCTION
|
---|
Comorbid conditions, such as diabetes and vascular disease, are among the many factors that contribute to the risk of infection with antibiotic-resistant bacteria (14). Risk factor studies of antibiotic-resistant bacteria often attempt to control for the risk attributable to comorbidity by including in a statistical model either a dichotomous variable, such as the presence or absence of any comorbid condition, or individual conditions (1, 517). For statistical reasons, it is often difficult to include several comorbid conditions in one statistical model without the concern of overfitting (1823). This concern is particularly important when assessing risks for rare events, such as an infection with a single species of resistant organism where the number of cases may be low, thus making it difficult to stratify or otherwise adjust for multiple variables (23, 24). We believe greater utility may be found in using a single aggregate measure of a persons risk due to comorbid conditions (2527). Although several such comorbidity measures exist, none has been developed specifically for use with infectious disease outcomes and, in particular, antibiotic-resistant bacterial outcomes.
The Charlson Comorbidity Index was originally designed as a measure of the risk of 1-year mortality attributable to comorbidity in a longitudinal study of general hospitalized patients. It was then validated for the same outcome in a cohort of breast cancer patients. Its contents and weighting scheme were created on the basis of Cox proportional hazards modeling (28). It was subsequently adapted so that International Classification of Diseases, Ninth Revision (ICD-9), codes could be used to calculate the Charlson Comorbidity Index with existing administrative data (29). Although this index has been used in numerous studies of antibiotic-resistant bacteria, to our knowledge it has never been validated for use with these outcomes.
The Chronic Disease Score is an aggregate comorbidity measure based on current medication use. A panel of health professionals originally created the Chronic Disease Score by using a pharmaceutical database to reach consensus decisions as to which classes of medications should be included in the score and how they should be weighted to correspond to disease complexity and severity. The Chronic Disease Score was originally validated for use as a predictor of physician-rated disease status, self-rated health status, hospitalization, and mortality (30). Like the Charlson Comorbidity Index, the Chronic Disease Score is not known to have been validated for use with antibiotic-resistant bacterial outcomes.
Numerous authors have stressed the importance of reevaluating the validity of risk measurement tools when they are used in different populations or when the outcome of interest is varied. The effectiveness of a risk-adjustment tool may vary under these circumstances (2527, 31, 32).
Methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin-resistant enterococci (VRE) are both causes of nosocomial infections that have increasingly been the focus of epidemiologic research, not only because they have been associated more often with increased morbidity and mortality than their antibiotic-susceptible counterparts but also because of the rising prevalence of these bacteria in the hospital setting (3339). Yet because the etiologies of MRSA and VRE infections are different, there may be true differences in the performance of the scores for these different outcomes. Therefore, it is important to evaluate these outcomes separately.
The purpose of this study was to compare the discrimination and calibration of two aggregate measures of comorbidity, the Charlson Comorbidity Index and the Chronic Disease Score, when used as predictors of nosocomial clinical culture positivity with MRSA or VRE among hospitalized patients.
 |
MATERIALS AND METHODS
|
---|
Two case-control studies were performed by using the population of patients, aged 18 years or older at admission, hospitalized at the University of Maryland Medical System (UMMS) between July 1, 1998, and July 1, 2001. Patients admitted directly to the R. Adams Cowley Shock Trauma Center, obstetrics/gynecology, or psychiatric services were excluded because they represent distinct subpopulations in which infection rates and transmission patterns may differ from those in the larger, general population. Patient eligibility was determined electronically by using the UMMS Central Data Repository. This repository is maintained by the UMMS Information Technology Group. The data contained in pharmacy, microbiology, and medical demographics tables in UMMS Central Data Repository were validated against the medical records of over 400 patients between October 1997 and January 2000 for previous research studies (4043). For the present study, the ICD-9 codes (used to calculate the Charlson Comorbidity Index) and pharmacy order records (used to calculate the Chronic Disease Score) for 30 patients and over 100 medication records were also validated against medical records. The electronically abstracted medications were 93 percent accurate (when compared with nursing medication administration records), and the electronically abstracted ICD-9 codes were 98 percent accurate. The components, and the weights for each component, of the Chronic Disease Score and Charlson Comorbidity Index can be found in the Appendix. Approval for this study was obtained from the University of Maryland, Baltimore Institutional Review Board.
For the first case-control study, cases were all eligible patients with a positive clinical culture for MRSA obtained after the first 48 hours of hospital admission; the use of cultures obtained beyond 48 hours is an accepted criterion for classification as a nosocomial infection (44). Only the first positive clinical culture for MRSA led to including a patient as a case. Subsequent positive cultures were disregarded; that is, each case was included in the study only once. Patients with a positive clinical culture for MRSA obtained within the first 48 hours of admission were excluded from the study because they most likely did not acquire MRSA during their current hospital stay. Controls were randomly sampled in a 3:1 ratio to cases from the population of eligible patients who did not have a positive clinical culture for MRSA during their hospital stay (45, 46).
The second case-control study was conducted in the same manner as the first, except that the outcome of interest was a positive clinical culture for VRE. Thus, for this study, cases were all eligible patients with a positive clinical culture for VRE obtained after the first 48 hours of hospital admission. Only the first positive clinical culture for VRE led to including a patient as a case so that each case was included in the study only once. Again, for the same reason as above, patients with a positive clinical culture for VRE obtained within the first 48 hours of admission were excluded from the study. Controls were randomly sampled in a 3:1 ratio of cases to controls from the same population of eligible patients who did not have a positive clinical culture for VRE during their hospital stay (45, 46).
Data used to calculate the Charlson Comorbidity Index and the Chronic Disease Score were obtained from the UMMS Central Data Repository as follows. For each study participant, discharge ICD-9 codes and information on the medications prescribed within the first 24 hours of admission were extracted electronically from medical records via the Central Data Repository. Discharge ICD-9 codes were used to calculate the Charlson Comorbidity Index for each study participant (29). The potential range of values for this index is 037. Medications prescribed in the first 24 hours of admission were used to calculate the Chronic Disease Score for each study participant (30); the potential range of values is 035. Clinical culturing information was collected to determine case/control status. Data were also extracted regarding admission and discharge dates, demographics, location within the hospital, and whether the patient had been transferred from another institution, had been in an intensive care unit, had had surgery, or had received any antibiotic (note: information on antibiotic use for cases was collected for only the time period prior to clinical culture positivity and, for controls, was collected if the antibiotic was used any time before discharge, i.e., the at-risk period). These descriptive data were used to characterize the cases and controls in each study (chi-square tests, t tests, and Wilcoxon rank-sum tests were used where appropriate).
For each case-control study, two logistic regression models were created by using each of the comorbidity measures (the Charlson Comorbidity Index and the Chronic Disease Score) as single, independent predictors of the outcome (MRSA or VRE clinical culture positivity); patient age and sex were then subsequently added to each model. For each model, the c statistic, along with its corresponding 95 percent confidence interval, was calculated. The c statistic is a measure of model discrimination that is defined as the area under the receiver operator characteristic (ROC) curve, a plot of sensitivity, and 1 specificity for all values of the predictor variable. Alternatively, the c statistic can be interpreted as an estimate of the probability that, for a randomly selected case, the value of the predictor variable will be higher than that for a randomly selected control. A c statistic value of 0.5 is equivalent to random prediction; a value of 1 is equivalent to perfect prediction (47). For example, if a predictor has some value at which all cases and controls are completely segregated (i.e., for all cases, values of the predictor are greater than the breakpoint, and, for all controls, values are below the breakpoint), then that predictor would have perfect discrimination (c = 1) for that outcome. The c statistic was calculated by using the nonparametric method; the standard error was calculated by using the methods suggested by DeLong et al. (48).
The discriminating ability of the Charlson Comorbidity Index and Chronic Disease Score were compared within each study by testing the equality of the two c statistics, while accounting for the correlation that occurs from using the same study sample, using the chi-square test (48). The Hosmer-Lemeshow goodness-of-fit chi-square test was also used to assess the calibration of each model. This test compares the expected and observed distribution of cases and controls across deciles of predicted risk. Therefore, for this test, a higher p value corresponds to better model fit (4951). The test of equality of c statistics and the Hosmer-Lemeshow goodness-of-fit chi-square test are the statistical methods commonly used for comparing both comorbidity measures and also for other risk-adjustment measures (31, 5159). Data analyses were performed by using SAS version 8.02 (SAS Institute, Inc., Cary, North Carolina) and Stata version 7.0 (Stata Corporation, College Station, Texas) software.
 |
RESULTS
|
---|
In the MRSA case-control study, 541 patients were included as cases and 1,623 as controls. Cases in this study were more likely than controls to have been in an intensive care unit, to have had surgery, and to have been transferred from another institution (table A1). A statistically significant difference between the mean age (in years) of cases (55.72; standard deviation, 15.41) and controls (51.10; standard deviation, 17.09) was also observed (t-test p < 0.01). The Chronic Disease Score ranged from 0 to 19, with 11.3 percent of cases and 25.5 percent of controls having a value of 0. The Charlson Comorbidity Index ranged from 0 to 14, with 26.1 percent of cases and 50.6 percent of controls having a value of 0. Because neither the Chronic Disease Score nor the Charlson Comorbidity Index was normally distributed within cases and controls, the Wilcoxon rank-sum test was used to compare the scores by case status. The median Chronic Disease Score for cases was statistically significantly different from that for controls (6 and 4, respectively; p < 0.01). The median Charlson Comorbidity Index for cases was also significantly different from that for controls (2 and 0, respectively; p < 0.01).
The c statistic was calculated for both the Chronic Disease Score and the Charlson Comorbidity Index modeled as single independent predictors of MRSA clinical culture positivity. The Chronic Disease Score yielded a c statistic (the area under the ROC curve) of 0.608 (95 percent confidence interval: 0.580, 0.634), and the Charlson Comorbidity Index yielded a c statistic of 0.653 (95 percent confidence interval: 0.627, 0.678). The chi-square test indicated a statistically significant difference in the two scores abilities to discriminate between cases and controls (p < 0.01). This difference remained significant even after age and sex were added to the models (p = 0.01; table B1). ROC curves were plotted for both age- and sex-adjusted models (figure 1). The Hosmer-Lemeshow chi-square test yielded significant values for both the Chronic Disease Score and the Charlson Comorbidity Index (p < 0.01 for both), indicating lack of fit. After age and sex were added to the models, results for this test were statistically significant for the Chronic Disease Score but not for the Charlson Comorbidity Index (p < 0.01 and p = 0.076, respectively). The observed numbers of cases were plotted with the expected numbers from the age- and sex-adjusted models within deciles of predicted risk to portray calibration (figure 2).

View larger version (14K):
[in this window]
[in a new window]
|
FIGURE 1. Receiver operator characteristic curves for the Chronic Disease Score and Charlson Comorbidity Index as predictors of methicillin-resistant Staphylococcus aureus clinical culture positivity with age (years) and sex included in the model, University of Maryland Medical System, 19982001.
|
|

View larger version (17K):
[in this window]
[in a new window]
|
FIGURE 2. Calibration plots of the Chronic Disease Score (a) and Charlson Comorbidity Index (b) as predictors of methicillin-resistant Staphylococcus aureus clinical culture positivity with age (years) and sex included in the models, University of Maryland Medical System, 19982001.
|
|
The VRE case-control study included 487 patients as cases and 1,461 as controls. In this study, cases were also more likely than controls to have been in an intensive care unit, to have had surgery, and to have been transferred from another institution (table 3). A statistically significant difference between the mean age (in years) of cases (55.20; standard deviation, 15.64) and controls (51.80; standard deviation, 17.33) was also observed (t-test p < 0.01). The Chronic Disease Score ranged from 0 to 18, with 7.8 percent of cases and 25.5 percent of controls having a value of 0. The Charlson Comorbidity Index ranged from 0 to 14, with 19.3 percent of cases and 46.7 percent of controls having a value of 0. Because, as in the previous study, neither the Chronic Disease Score nor the Charlson Comorbidity Index was normally distributed, the Wilcoxon rank-sum test was used to compare the median scores by case status. The median Chronic Disease Score for cases was again statistically significantly different from that for controls (6 and 4, respectively; p < 0.01). The median Charlson Comorbidity Index for cases was also significantly different from that for controls (2 and 1, respectively; p < 0.01).
View this table:
[in this window]
[in a new window]
|
TABLE 3. Characteristics of the cases and controls in the vancomycin-resistant enterococci study, University of Maryland Medical System, 19982001
|
|
The c statistic was calculated for both the Chronic Disease Score and the Charlson Comorbidity Index modeled as single, independent predictors of VRE clinical culture positivity. The Chronic Disease Score yielded a c statistic of 0.647 (95 percent confidence interval: 0.620, 0.674), and the Charlson Comorbidity Index yielded a c statistic of 0.670 (95 percent confidence interval: 0.644, 0.696). The chi-square test indicated no significant difference between the two scores abilities to discriminate between cases and controls; adding age and sex to these models did not change this result (table 4). ROC curves were plotted for the age- and sex-adjusted models (figure 3). The Hosmer-Lemeshow chi-square test yielded significant values for both the Chronic Disease Score and Charlson Comorbidity Index (p < 0.01 for both). After age and sex were added to the models, results for this test were still statistically significant for both the Chronic Disease Score and the Charlson Comorbidity Index (p = 0.014 and p < 0.01, respectively). Calibration plots for each of the age- and sex-adjusted models are presented in figure 4.
View this table:
[in this window]
[in a new window]
|
TABLE 4. Comparison of the discriminating abilities of the CDS* and the CCI* in the vancomycin-resistant enterococci study, University of Maryland Medical System, 19982001
|
|

View larger version (15K):
[in this window]
[in a new window]
|
FIGURE 3. Receiver operator characteristic curves for the Chronic Disease Score and Charlson Comorbidity Index as predictors of vancomycin-resistant enterococci clinical culture positivity with age (years) and sex included in the model, University of Maryland Medical System, 19982001.
|
|

View larger version (17K):
[in this window]
[in a new window]
|
FIGURE 4. Calibration plots of the Chronic Disease Score (a) and Charlson Comorbidity Index (b) as predictors of vancomycin-resistant enterococci clinical culture positivity with age (years) and sex included in the models, University of Maryland Medical System, 19982001.
|
|
 |
DISCUSSION
|
---|
Aggregate comorbidity measures in infectious disease research, and in particular in the realm of antibiotic resistance, are frequently utilized in the following two ways. First, they are used in case-control and cohort studies to determine the risk factors for colonization or infection with antibiotic-resistant bacteria (40, 6065). Often, the comorbidity measure represents important risk factors but also an important confounding variable that, when adjusted for correctly, allows for a more accurate assessment of the strength of the association (e.g., the odds ratio or relative risk) between other causal variables, such as previous antibiotic use, and infection with antibiotic-resistant bacteria. Second, comorbidity measures are utilized in prediction rules to predict colonization or infection with antibiotic-resistant bacteria (66, 67). In this application, comorbidity measures are used in real time as part of infection control interventions such as identifying patients for isolation or surveillance cultures.
For both applications of comorbidity measures, calibration and discrimination are important. Our results demonstrate that both the Chronic Disease Score and Charlson Comorbidity Index discriminate between patients with and without positive clinical cultures for MRSA or VRE better than would a random predictor (tables 2 and 4). For the MRSA data set, these data indicate a statistically significant difference between the discriminating ability of the Chronic Disease Score and Charlson Comorbidity Index, with the latter outperforming the former. In the VRE data set, the Charlson Comorbidity Index also yielded a higher c statistic than the Chronic Disease Score; however, this difference was not statistically significant. In both data sets, the absolute difference between the c statistics was not large (0.045 in the MRSA data and 0.024 in the VRE data).
View this table:
[in this window]
[in a new window]
|
TABLE 2. Comparison of the discriminating abilities of the CDS* and the CCI* in the methicillin-resistant Staphylococcus aureus study, University of Maryland Medical System, 19982001
|
|
Model calibration was tested by using the Hosmer-Lemeshow goodness-of-fit chi-square test. A perfectly calibrated model would be one in which the observed distribution of events (MRSA or VRE clinical culture positivity) matches the expected distribution of events based on the model. The Hosmer-Lemeshow test indicated that neither the Chronic Disease Score nor the Charlson Comorbidity Index was a well-calibrated predictor of either MRSA or VRE clinical culture positivity (the observed distribution of events was significantly different from the expected distribution).
After age and sex were included in the models, only the calibration of the Charlson Comorbidity Index model for predicting MRSA clinical culture status improved to no longer be statistically significant (p = 0.076). However, the calibration plots (figures 2 and 4) show that, for all of the age- and sex-adjusted models, the observed number of cases roughly followed the expected number of cases predicted by the models. Statistical significance for this test may, in part, be driven by the large sample sizes of these data sets. It is reasonable to expect differences in the performances of both comorbidity measures as predictors of MRSA versus VRE clinical culture positivity. Because S. aureus commonly resides in the respiratory tract while enterococci reside in the gastrointestinal tract, the etiology of the infections caused by these organisms (and thus the risk factors for each) is likely different. Thus, there may be true differences in the performance of the scores for these different outcomes.
Although the discriminating ability of the two scores is better than would be expected from a random predictor, the discrimination of both scores is still poor. It is typically desirable that model discrimination (the c statistic) be greater than 0.70. The lower c statistics observed in these data (point estimates were in the range of 0.600.68) are not unexpected. Other risk factors, such as antibiotic use, may have stronger causal associations with higher discriminatory abilities. To obtain an accurate measure of the strength of associations with these causal variables, adjustment for comorbidity is still important.
In both data sets included in this study, differences were observed between cases and controls (tables 1 and 3). Some of the observed differences, such as antibiotic use and stay in an intensive care unit, may be true differences associated with MRSA or VRE clinical culture positivity. Other observed differences may have reached statistical significance primarily because of the large sample size. The issue of matching in case-control studies of antibiotic resistance is controversial. To date, the majority of research in this field has not used matching (35, 68). Thus, matching was not performed in designing either data set to ensure that the estimates of discrimination and calibration measured in this study would be more generalizable to other risk-factor studies of MRSA and VRE.
View this table:
[in this window]
[in a new window]
|
TABLE 1. Characteristics of the cases and controls in the methicillin-resistant Staphylococcus aureus case-control study, University of Maryland Medical System, 19982001
|
|
Although the Charlson Comorbidity Index was significantly better than the Chronic Disease Score at discriminating MRSA clinical culture status, both scores appear to be functionally equivalent discriminators for nosocomial infection with either MRSA or VRE. Because the Deyo adaptation of the Charlson Comorbidity Index is calculated by using discharge ICD-9 codes, it may be based on patient comorbidities that develop after the infection has occurredeven though such comorbidities cannot contribute to the risk of infection. Thus, it would be expected that the values for this score would be higher than for the Chronic Disease Score for cases, since comorbidities may have developed or been detected as a result of either the infection or the patients lengthened stay in the hospital. This possibility may in part explain the difference in the discriminatory abilities between the two scores. Because discharge ICD-9 codes are used in the Deyo adaptation of the Charlson Comorbidity Index, it also makes an inappropriate choice for use in epidemiologic studies in which patients are observed for only one admission.
This method of comorbidity ascertainment also makes the Deyo adaptation of the Charlson Comorbidity Index impractical for potential real-time use in the field of hospital epidemiology and infection control. Although the Charlson Comorbidity Index can be calculated without using existing ICD-9 codes, the data for the alternate methods of calculation are not so easily collected. ICD-9 codes from a previous hospital admission have also been used to calculate the Charlson Comorbidity Index, but doing so can result in missing data for patients with no known previous admission (6973). In addition, the validity of such data may be questionable for patients with long time periods between admissions. A more appropriate measure of comorbidity would be one that is both easily calculated based on existing records and one that measures exposure prior to the occurrence of the outcome and may therefore be used for both epidemiologic studies and real-time interventions. The Chronic Disease Score, when calculated within the first 24 hours of admission, can function in a predictive capacity and is therefore more advantageous.
Because the Chronic Disease Score and Charlson Comorbidity Index were not originally designed for use as predictors of nosocomial infections with antibiotic-resistant bacteria, the comorbidities included in each of the scores may not be inclusive of all comorbid conditions associated with an increased risk of a nosocomial infection due to antibiotic-resistant bacteria. Furthermore, some conditions included in these scores may either have no association with such an outcome or be inappropriately weighted in the calculated scores. For example, the Charlson Comorbidity Index includes dementia as one of its constructs, and the Chronic Disease Score includes migraines. To date, research in this field has provided no evidence of or biologic plausibility for an association between either of these constructs and antibiotic-resistant nosocomial bacterial infections. Differences in the comorbidities, and the weights for each of the comorbidities, included in each score also may explain the observed differences in the performance of the Chronic Disease Score and Charlson Comorbidity Index.
An aggregate comorbidity score correctly parameterized for use as a measure of the risk of infection with antibiotic-resistant bacteria would be expected to have better (although not perfect) model calibration and discrimination and thereby serve as a better risk-adjustment measure in infectious disease epidemiology (2527, 31, 32). However, revising either comorbidity score would require considerable effort. The constructs used to measure the presence or absence of each condition would have to be evaluated, a literature search and consultations with infectious disease professionals would be necessary to evaluate which comorbidities could plausibly be included in the measure, and, optimally, new weights for the components of the score should be developed by using regression models. Then, the discrimination and calibration of the revised score would need to be evaluated before researchers could use the score to measure the contribution of comorbidity in risk-factor studies.
Our comparison of the ability of the Chronic Disease Score and the Charlson Comorbidity Index indicates that both are roughly equivalent as measures of the increased risk of a nosocomial, antibiotic-resistant infection that is attributable to preexisting comorbidities. The discriminatory abilities of both scores were functionally similar but below the standard level of acceptability. In addition, both scores were poorly calibratedmeaning that the predicted probabilities of being a case and the observed proportions of cases were not consistent across deciles of risk. On the basis of model discrimination and calibration, the Chronic Disease Score and Charlson Comorbidity Index appear to demonstrate no effective differences. However, because the Deyo adaptation of the Charlson Comorbidity Index is calculated by using data that were collected on discharge, it would not be appropriate to use in a real-time infection control intervention or in epidemiologic studies as a risk adjustment tool. Because the Chronic Disease Score is easily calculated within the first 24 hours of admission to the hospital, it is more practical as a comorbidity measure for use with antibiotic-resistant infectious diseases. Further refinement of this scoring algorithm may improve the discrimination and calibration so as to create a far better comorbidity measure for use with antibiotic-resistant infectious diseases.
 |
ACKNOWLEDGMENTS
|
---|
The views expressed in this article do not necessarily reflect those of the Food and Drug Administration.
 |
APPENDIX
|
---|
BA1
BB1
 |
NOTES
|
---|
Correspondence to Jessina C. McGregor, Department of Epidemiology and Preventive Medicine, University of Maryland, Baltimore, 100 North Greene Street, Lower Level, Baltimore, MD 21201 (e-mail: jmcgrego{at}epi.umaryland.edu). 
 |
REFERENCES
|
---|
- Tornieporth NG, Roberts RB, John J, et al. Risk factors associated with vancomycin-resistant Enterococcus faecium infection or colonization in 145 matched case patients and control patients. Clin Infect Dis 1996;23:76772.[ISI][Medline]
- Memmel H, Kowal-Vern A, Latenser BA. Infections in diabetic burn patients. Diabetes Care 2004;27:22933.[Abstract/Free Full Text]
- Baillargeon J, Kelley MF, Leach CT, et al. Methicillin-resistant Staphylococcus aureus infection in the Texas prison system. Clin Infect Dis 2004;38:e925.[CrossRef][ISI][Medline]
- Naylor AR, Hayes PD, Darke S. A prospective audit of complex wound and graft infections in Great Britain and Ireland: the emergence of MRSA. Eur J Vasc Endovasc Surg 2001;21:28994.[CrossRef][ISI][Medline]
- Chambers HF. The changing epidemiology of Staphylococcus aureus? Emerg Infect Dis 2001;7:17882.[ISI][Medline]
- Joels CS, Matthews BD, Sigmon LB, et al. Clinical characteristics and outcomes of surgical patients with vancomycin-resistant enterococcal infections. Am Surg 2003;69:51419.[ISI][Medline]
- Gomez-Barreto D, Calderon-Jaimes E, Rodriguez RS, et al. Clinical outcome of invasive infections in children caused by highly penicillin-resistant Streptococcus pneumoniae compared with infections caused by penicillin-susceptible strains. Arch Med Res 2000;31:5928.[CrossRef][ISI][Medline]
- Morrison L, Stolarek I. Does MRSA affect patient outcomes in the elderly? A retrospective pilot study. J Hosp Infect 2000;45:16971.[CrossRef][ISI][Medline]
- Garau J, Aguilar L, Rodriguez-Creixems M, et al. Influence of comorbidity and severity on the clinical outcome of bacteremic pneumococcal pneumonia treated with beta-lactam monotherapy. J Chemother 1999;11:26672.[ISI][Medline]
- Chang FY, MacDonald BB, Peacock JE Jr, et al. A prospective multicenter study of Staphylococcus aureus bacteremia: incidence of endocarditis, risk factors for mortality, and clinical impact of methicillin resistance. Medicine (Baltimore) 2003;82:32232.[CrossRef][ISI][Medline]
- Chang FY, Peacock JE Jr, Musher DM, et al. Staphylococcus aureus bacteremia: recurrence and the impact of antibiotic treatment in a prospective multicenter study. Medicine (Baltimore) 2003;82:3339.[CrossRef][ISI][Medline]
- Lautenbach E. Enterococcal bacteremia: risk factors for vancomycin resistance and predictors of mortality. Infect Control Hosp Epidemiol 1999;20:31823.[ISI][Medline]
- Roghmann MC, Fink JC, Polish L, et al. Colonization with vancomycin-resistant enterococci in chronic hemodialysis patients. Am J Kidney Dis 1998;32:2547.[ISI][Medline]
- Montesinos I, Salido E, Delgado T, et al. Epidemiology of methicillin-resistant Staphylococcus aureus at a university hospital in the Canary Islands. Infect Control Hosp Epidemiol 2003;24:66772.[ISI][Medline]
- Campbell AL, Bryant KA, Stover B, et al. Epidemiology of methicillin-resistant Staphylococcus aureus at a childrens hospital. Infect Control Hosp Epidemiol 2003;24:42730.[ISI][Medline]
- Husni R, Hachem R, Hanna H, et al. Risk factors for vancomycin-resistant Enterococcus (VRE) infection in colonized patients with cancer. Infect Control Hosp Epidemiol 2002;23:1023.[ISI][Medline]
- Terpenning MS, Bradley SF, Wan JY, et al. Colonization and infection with antibiotic-resistant bacteria in a long-term care facility. J Am Geriatr Soc 1994;42:10629.[ISI][Medline]
- Peduzzi P, Concato J, Kemper E, et al. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996;49:13739.[CrossRef][ISI][Medline]
- Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med,1996;15:36187.[CrossRef][ISI][Medline]
- Harrell FE Jr, Lee KL, Califf RM, et al. Regression modelling strategies for improved prognostic prediction. Stat Med 1984;3:14352.[ISI][Medline]
- Moss M, Wellman DA, Cotsonis GA, An appraisal of multivariable logistic models in the pulmonary and critical care literature. Chest 2003;123:9238.[Abstract/Free Full Text]
- Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med 2004;66:41121.[Abstract/Free Full Text]
- Concato J, Feinstein AR, Holford TR. The risk of determining risk with multivariable models. Ann Intern Med 1993;118:20110.[Abstract/Free Full Text]
- Harrell FE Jr, Lee KL, Matchar DB, et al. Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep 1985;69:10717.[ISI][Medline]
- Charlson M, Ales KL, Simon R, et al. Why predictive indexes perform less well in validation studies. Arch Intern Med 1987;147:215561.[Abstract]
- Lesens O, Methlin C, Hansmann Y, et al. Role of comorbidity in mortality related to Staphylococcus aureus bacteremia: a prospective study using the Charlson weighted index of comorbidity. Infect Control Hosp Epidemiol 2003;24:8906.[ISI][Medline]
- Cosgrove SE, Carmeli Y. Studies of bloodstream infection outcomes: reading between the lines. Infect Control Hosp Epidemiol 2003;24:8846.[ISI][Medline]
- Charlson ME, Pompei P, Ales KL, et al. A. new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:37383.[CrossRef][ISI][Medline]
- Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45:61319.[CrossRef][ISI][Medline]
- Von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol 1992;45:197203.[ISI][Medline]
- Schneeweiss S, Seeger JD, Maclure M, et al. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol 2001;154:85464.[Abstract/Free Full Text]
- Iezzoni LI, ed. Risk adjustment for measuring healthcare outcomes. 2nd ed. Chicago, IL: Health Administration Press, 1997.
- Nosocomial enterococci resistant to vancomycinUnited States, 19891993. MMWR Morb Mortal Wkly Rep 1993;42:5979.[Medline]
- Fridkin SK, Gaynes RP. Antimicrobial resistance in intensive care units. Clin Chest Med 1999;20:30316, viii.
- Cosgrove SE, Sakoulas G, Perencevich EN, et al. Comparison of mortality associated with methicillin-resistant and methicillin-susceptible Staphylococcus aureus bacteremia: a meta-analysis. Clin Infect Dis 2003;36:539.[CrossRef][ISI][Medline]
- Mutnick AH, Biedenbach DJ, Jones RN. Geographic variations and trends in antimicrobial resistance among Enterococcus faecalis and Enterococcus faecium in the SENTRY Antimicrobial Surveillance Program (19972000). Diagn Microbiol Infect Dis 2003;46:638.[CrossRef][ISI][Medline]
- Low DE, Keller N, Barth A, et al. Clinical prevalence, antimicrobial susceptibility, and geographic resistance patterns of enterococci: results from the SENTRY Antimicrobial Surveillance Program, 19971999. Clin Infect Dis 2001;32(suppl 2):S13345.[Medline]
- Pfaller MA, Jones RN, Doern GV, et al. Survey of blood stream infections attributable to gram-positive cocci: frequency of occurrence and antimicrobial susceptibility of isolates collected in 1997 in the United States, Canada, and Latin America from the SENTRY Antimicrobial Surveillance Program. SENTRY Participants Group. Diagn Microbiol Infect Dis 1999;33:28397.[CrossRef][ISI][Medline]
- Edmond MB, Wallace SE, McClish DK, et al. Nosocomial bloodstream infections in United States hospitals: a three-year analysis. Clin Infect Dis 1999;29:23944.[ISI][Medline]
- Harris AD, Smith D, Johnson JA, et al. Risk factors for imipenem-resistant Pseudomonas aeruginosa among hospitalized patients. Clin Infect Dis 2002;34:3405.[CrossRef][ISI][Medline]
- Harris AD, Perencevich E, Roghmann MC, et al. Risk factors for piperacillin-tazobactam-resistant Pseudomonas aeruginosa among hospitalized patients. Antimicrob Agents Chemother 2002;46:8548.[Abstract/Free Full Text]
- Kim PW, Harris AD, Roghmann MC, et al. Epidemiological risk factors for isolation of ceftriaxone-resistant versus -susceptible citrobacter freundii in hospitalized patients. Antimicrob Agents Chemother 2003;47:28827.[Abstract/Free Full Text]
- Harris AD, Nemoy L, Johnson JA, et al. Co-carriage rates of vancomycin-resistant Enterococcus and extended-spectrum beta-lactamase-producing bacteria among a cohort of intensive care unit patients: implications for an active surveillance program. Infect Control Hosp Epidemiol 2004;25:1058.[ISI][Medline]
- Garner JS, Jarvis WR, Emori TG, et al. CDC definitions for nosocomial infections. In: Olmstead RN, ed. APIC infection control and applied epidemiology: principles and practice. St. Louis, MO: Mosby, 1996: A1A20.
- Harris AD, Samore MH, Lipsitch M, et al. Control-group selection importance in studies of antimicrobial resistance: examples applied to Pseudomonas aeruginosa, Enterococci, and Escherichia coli. Clin Infect Dis 2002;34:155863.[CrossRef][ISI][Medline]
- Harris AD, Samore MH, Carmeli Y. Control group selection is an important but neglected issue in studies of antibiotic resistance. (Letter). Ann Intern Med 2000;133:159.[Free Full Text]
- Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:2936.[Abstract]
- DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:83745.[ISI][Medline]
- Lemeshow S, Hosmer DW Jr. A. review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982;115:92106.[Abstract]
- Diamond GA. What price perfection? Calibration and discrimination of clinical prediction models. J Clin Epidemiol 1992;45:859.[CrossRef][ISI][Medline]
- Lemeshow S, Le Gall JR. Modeling the severity of illness of ICU patients. A systems update. JAMA 1994;272:104955.[Abstract]
- Schneeweiss S, Maclure M. Use of comorbidity scores for control of confounding in studies using administrative databases. Int J Epidemiol 2000;29:8918.[Abstract/Free Full Text]
- Roos LL, Stranc L, James RC, et al. Complications, comorbidities, and mortality: improving classification and prediction. Health Serv Res 1997;32:22938; discussion 23942.[ISI]
- Ghali WA, Hall RE, Rosen AK, et al. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. J Clin Epidemiol 1996;49:2738.[CrossRef][ISI][Medline]
- Melfi C, Holleman E, Arthur D, et al. Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data. J Clin Epidemiol 1995;48:91726.[CrossRef][ISI][Medline]
- Johnston JA, Wagner DP, Timmons S, et al. Impact of different measures of comorbid disease on predicted mortality of intensive care unit patients. Med Care 2002;40:92940.[CrossRef][ISI][Medline]
- Putnam KG, Buist DS, Fishman P, et al. Chronic disease score as a predictor of hospitalization. Epidemiology 2002;13:3406.[CrossRef][ISI][Medline]
- Ottenbacher KJ, Smith PM, Illig SB, et al. Comparison of logistic regression and neural networks to predict rehospitalization in patients with stroke. J Clin Epidemiol 2001;54:115965.[CrossRef][ISI][Medline]
- Kizer JR, Berlin JA, Laskey WK, et al. Limitations of current risk-adjustment models in the era of coronary stenting. Am Heart J 2003;145:68392.[CrossRef][ISI][Medline]
- Troillet N, Carmeli Y, Venkataraman L, et al. Epidemiological analysis of imipenem-resistant Serratia marcescens in hospitalized patients. J Hosp Infect 1999;42:3743.[ISI][Medline]
- Troillet N, Samore MH, Carmeli Y. Imipenem-resistant Pseudomonas aeruginosa: risk factors and antibiotic susceptibility patterns. Clin Infect Dis 1997;25:10948.[ISI][Medline]
- Ostrowsky BE, Venkataraman L, DAgata EM, et al. Vancomycin-resistant enterococci in intensive care units: high frequency of stool carriage during a non-outbreak period. Arch Intern Med 1999;159:146772.[Abstract/Free Full Text]
- Perencevich EN, Sands KE, Cosgrove SE, et al. Health and economic impact of surgical site infections diagnosed after hospital discharge. Emerg Infect Dis 2003;9:196203.[ISI][Medline]
- Kaye KS, Sands K, Donahue JG, et al. Preoperative drug dispensing as predictor of surgical site infection. Emerg Infect Dis 2001;7:5765.[ISI][Medline]
- Leveille SG, Gray S, LaCroix AZ, et al. Physical inactivity and smoking increase risk for serious infections in older women. J Am Geriatr Soc 2000;48:15828.[ISI][Medline]
- Tacconelli E, DAgata EM, Karchmer AW. Epidemiological comparison of true methicillin-resistant and methicillin-susceptible coagulase-negative staphylococcal bacteremia at hospital admission. Clin Infect Dis 2003;37:6449.[CrossRef][ISI][Medline]
- Lodise TP Jr, McKinnon PS, Rybak M. Prediction model to identify patients with Staphylococcus aureus bacteremia at risk for methicillin resistance. Infect Control Hosp Epidemiol 2003;24:65561.[ISI][Medline]
- Harris AD, Karchmer TB, Carmeli Y, et al. Methodological principles of case-control studies that analyzed risk factors for antibiotic resistance: a systematic review. Clin Infect Dis 2001;32:105561.[CrossRef][ISI][Medline]
- Zhang JX, Iwashyna TJ, Christakis NA. The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims. Med Care 1999;37:112839.[CrossRef][ISI][Medline]
- Lamont EB, Lauderdale DS. Low risk of hip fracture among elderly breast cancer survivors. Ann Epidemiol 2003;13:698703.[CrossRef][ISI][Medline]
- Lamont EB, Christakis NA, Lauderdale DS. Favorable cardiac risk among elderly breast carcinoma survivors. Cancer 2003;98:210.[CrossRef][ISI][Medline]
- Iwashyna TJ, Lamont EB. Effectiveness of adjuvant fluorouracil in clinical practice: a population-based cohort study of elderly patients with stage III. colon cancer. J Clin Oncol 2002;20:39928.[Abstract/Free Full Text]
- Wang PS, Walker A, Tsuang M, et al. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol 2000;53:5718.[CrossRef][ISI][Medline]