1 Department of Epidemiology, School of Public Health, University of North Carolina, Chapel Hill, NC.
2 Collaborative Studies Coordinating Center, Department of Biostatistics, School of Public Health, University of North Carolina, Chapel Hill, NC.
3 Division of Epidemiology and Clinical Applications, National Heart, Lung, and Blood Institute, Bethesda, MD.
4 Department of Epidemiology, School of Public Health, State University of New York, Albany, NY.
5 Department of Epidemiology and Health Services Evaluation, Faculty of Health Sciences, Ben Gurion University of the Negev, Beersheva, Israel.
6 Department of Pediatrics, School of Medicine, University of Mississippi, Jackson, MS.
7 Division of Epidemiology, School of Public Health, University of Minnesota, Minneapolis, MN.
Received for publication February 11, 2004; accepted for publication May 14, 2004.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
coronary disease; diagnosis; hospital records; myocardial infarction; population surveillance; validation studies [publication type]
Abbreviations: Abbreviations: AMI, acute myocardial infarction; ARIC, Atherosclerosis Risk in Communities; CI, confidence interval; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The hospital discharge diagnosis is a critical part of the medical record. Physicians may rely on hospital discharge diagnoses from a patients previous hospitalizations to guide future treatment decisions. Hospital administrators use final discharge diagnoses for billing purposes and quality improvement monitoring, and national surveys use rates of hospital discharge diagnoses to monitor health-care utilization and disease burden. Hospital discharge diagnoses are also critical in research efforts aimed at evaluating trends in disease incidence in the population.
In 2000, there were over 1,200,000 hospital discharges for coronary heart disease in the United Statesa 17.7 percent increase since 1979 (1). Assessment of trends in coronary heart disease morbidity is greatly facilitated by numeric coding of hospital discharge diagnoses using the guidelines of the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (2). Periodic evaluation of such a system is important to ensure that coronary heart disease is being monitored appropriately, that the system is operating efficiently, that the information it is providing is useful for public health practice, and that the best use of public health resources is being promoted (3).
Although the International Classification of Diseases system provides a theoretical worldwide standard for categorizing both fatal and nonfatal coronary heart disease events, the validity of some code groups is known to vary widely (317). The ICD-9-CM code for nonfatal acute myocardial infarction (AMI), code 410, has been reported to modestly overestimate the number of true AMI hospitalizations (12).
Variations in the validity of hospital discharge diagnoses can complicate the assessment of AMI trends. However, validity over time and differences between US geographic regions, ethnic groups, and men and women are not well documented. Recent advancements in technology (e.g., the use of cardiac troponin, a more sensitive and specific bio-marker) in the diagnosis of AMI make reliance on discharge codes for evaluation of incident AMI trends dubious without independent and standardized validation. Because discharge diagnosis codes form the basis of retrospective community hospital surveillance studies, epidemiologists need to be aware of the inaccuracies in diagnosis documentation that might be intrinsic to this practice and how this might affect the interpretation of disease trends (3, 13).
The goal of the current analysis was to evaluate trends in the sensitivity, positive predictive value, false-positive rate, and comparability ratio of hospital discharge diagnosis codes for AMI and to identify factors that might be associated with their accuracy in four geographically and ethnically diverse communities in the United States.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Annual lists of hospital discharges meeting the studys age (3574 years) and residence criteria were obtained for each of 31 hospitals serving the four ARIC communities. Sampling criteria were used to select cases for investigation and thorough abstraction of medical records. Eligibility for sampling was based on gender, date of discharge, and discharge diagnosis code. Target codes included a range of primary or secondary diagnosis codes (ICD-9-CM codes 402, 410414, 427, 428, and 518.4). Sampled discharge diagnosis codes, their descriptions, and sampling probabilities are shown in table 1. In 1994, the sampling probabilities were adjusted to reduce the variance of event rate estimations while stabilizing the overall number of records requiring abstraction. Trained staff abstracted eligible medical records for information on presenting symptoms, the presence of chest pain, medical history, and levels of cardiac biomarkers during the first 4 days after the event. Copies of up to three electrocardiograms were made and sent to the University of Minnesota ECG Reading Center for classification according to the Minnesota Code (20). A computerized algorithm was applied to data on symptoms, cardiac biomarkers, and electrocardiographic evidence to determine the ARIC myocardial infarction classification (figure 1). Data on cardiac troponin measurements were collected starting in 1996. As with the other biomarkers, troponin levels two times the upper limit of normal (hospital specific normal range) were considered abnormal. Levels above the upper limit of normal but less than two times the upper limit were considered equivocal.
|
|
Because the ARIC classification system allows for different levels of certainty of AMI diagnosis, we computed validity measures using two rubrics. First, we compared the combination ARIC classification category of definite or probable AMI with the presence or absence of discharge ICD-9-CM code 410. Second, we compared the ARIC classification of definite-only AMI with the presence or absence of discharge ICD-9-CM code 410.
Each hospital was classified as to its status as a teaching hospital. A teaching hospital was defined as an institution that had full-time residents in internal medicine. In 2000, there were 31 hospitals serving the four ARIC communities. Of these, 14 (45 percent) met the criterion for a teaching hospital. Hospital type for events involving transfers and linked events was classified on the basis of the teaching status of the hospital from the most recent discharge.
Statistical methods
For the purpose of computing the sensitivity, positive predictive value, false-positive rate, and comparability ratio of hospital discharge diagnoses, an events diagnostic classification based on the ARIC Study criteria was designated the gold standard. This gold standard classification was compared with the presence or absence of a hospital discharge code for AMI (ICD-9-CM code 410). Discharge diagnosis codes were treated as hierarchical. For example, a 410 code present as either a primary discharge diagnosis or a secondary discharge diagnosis, regardless of the presence of other eligibility codes, was sufficient for classification as having a 410 code. Validity measures for ICD-9-CM code 410 were defined on the basis of the 2 x 2 table shown in table 2 and included sensitivity (a/c1), positive predictive value (a/r1), false-positive rate (b/c2), and comparability ratio (r1/c1). The comparability ratio was the ratio of the number of AMIs involving hospitalization ("hospitalized AMIs") as defined by the presence of ICD-9-CM code 410 to the number of validated AMIs as determined by the ARIC diagnostic protocol. Sensitivity and the positive predictive value were computed in the standard manner (21). The comparability ratio reflects the overall effect of misclassification as to whether ICD-9-CM codes or validated cases are used. The comparability ratio is especially useful in this context, where specificity is only estimated and the number of true myocardial infarctions missed because of sampling is likely to be very small. Weighted estimates were used for the numerator and denominators of these four ratio measures. Variances for the ratio estimators defining sensitivity were estimated by means of the delta method (22). The test for a difference in sensitivity between groups used as a test statistic the difference between sensitivities divided by an estimated standard error, the square root of the sum of the variances of the group-specific sensitivity estimates. The distribution of values for this test statistic is approximately standard normal in large populations.
|
Combining data for all years, we computed male:female odds ratios for receiving an ARIC diagnosis of AMI among all events with a discharge ICD-9-CM code of 410, using multivariate logistic regression analysis weighted for the inverse of the sampling fraction, with age (continuous variable), community (four levels), calendar year (14 levels), race (Black, White), and teaching status (teaching, nonteaching) included as independent variables. Similarly, we constructed adjusted White:Black and teaching:nonteaching odds ratios. We evaluated trends in validity measures by fitting a linear regression model to the annual estimates and testing whether the slopes were statistically significantly different from zero.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Of all hospital discharges with sampled codes, 13.8 percent of the discharges (n = 21,297) pertained to definite or probable AMI by the ARIC criteria (table 3). Among events meeting ARIC criteria for definite or probable AMI, 66 percent of the patients (n = 14,102) were male and 79 percent (n = 16,916) were White. Thirty-five percent (n = 7,495) of the discharges involving definite or probable AMI events were from hospitals classified as teaching sites. Among men, 16 percent of hospital discharges involved definite or probable AMI by the ARIC criteria, as compared with 11 percent of discharges among women. The proportions of eligible discharges meeting ARIC criteria for AMI were very similar between Blacks and Whites (14 percent), across communities (1216 percent), and between teaching and nonteaching hospitals (13 percent and 14 percent, respectively).
|
The sensitivity, positive predictive value, false-positive rate, and comparability ratio for the ability of ICD-9-CM code 410 to identify AMI are shown in table 4. The sensitivity of code 410 to correctly classify definite or probable AMI in men (sensitivity = 0.65, 95 percent confidence interval (CI): 0.63, 0.66) was statistically significantly greater than that found for women (sensitivity = 0.60, 95 percent CI: 0.58, 0.62) (p = 0.0003). The difference between Whites and Blacks in the sensitivity of code 410 was over three times that of the gender difference (for Whites, sensitivity = 0.67, 95 percent CI: 0.65, 0.68; for Blacks, sensitivity = 0.50, 95 percent CI: 0.47, 0.53) (p < 0.0001).
|
The ethnic difference in the false-negative discharges was related to a greater frequency of hypertensive heart disease and congestive heart failure codes encompassing AMI among Blacks in comparison with Whites. Of events among Blacks that met ARIC criteria for definite or probable AMI yet did not have an ICD-9-CM discharge code of 410414, 11 percent of the patients were discharged with a hypertensive heart disease code, as compared with 3 percent among Whites. Fifty-four percent of events among Blacks meeting ARIC criteria for AMI yet without a discharge code of 410414 had a congestive heart failure code, as compared with 36 percent among Whites. For Whites, the predominate discharge diagnosis among cases meeting ARIC criteria for AMI without an ICD-9-CM code of 410414 was cardiac dysrhythmia, including cardiac arrest (ICD-9-CM code 427); 60 percent of White patients carried this discharge code, as compared with 33 percent among Blacks.
The difference in sensitivity between Blacks and Whites was most pronounced in Forsyth County, where the sensitivity among Blacks was 0.46 (95 percent CI: 0.43, 0.50) as compared with 0.61 (95 percent CI: 0.59, 0.63) among Whites (p < 0.0001). In Jackson, the difference between ethnic groups in sensitivity was smaller (for Blacks, sensitivity = 0.53, 95 percent CI: 0.49, 0.57; for Whites, sensitivity = 0.59, 95 percent CI: 0.56, 0.63) yet statistically significant (p = 0.02). The sensitivity of ICD-9-CM code 410 was highest in Washington County and Minneapolis hospitals. There was no difference in sensitivity estimates between teaching and nonteaching hospitals (p = 0.31). Similar patterns of sensitivity were seen when data were restricted to definite AMI only (table 4).
The positive predictive value of an ICD-9-CM 410 code for definite plus probable AMI varied little across the strata examined (table 4). Across communities, positive predictive values were similar, ranging from a high of 0.80 among Whites in Forsyth County to a low of 0.71 in Washington County. As expected, the magnitudes of positive predictive values were lower when validation was restricted to the less inclusive definite-only AMI classification; however, the patterns observed across strata were similar to the pattern seen with definite plus probable events.
The false-positive rate differed little between men and women. There was also little difference in the false-positive rate between Blacks and Whites, between communities, between ethnic groups within communities, or between hospital types. The patterns of false-positive rates across strata were similar regardless of whether validation was based on definite AMI only or included probable AMI events.
The comparability ratiothe ratio of cases identified by ICD-9-CM code 410 to the number of validated caseswas similar for men and women. However, this ratio was greater among Whites (0.88) than among Blacks (0.69). As was seen for sensitivity, the ethnic difference was greater in Forsyth County (0.64 for Blacks and 0.76 for Whites) than in Jackson (0.72 for Blacks and 0.75 for Whites).
Adjusted estimates of validity
We also investigated the independent association of gender, ethnicity, and hospital type with the odds that a discharge ICD-9-CM code of 410 would meet ARIC criteria for definite or probable AMI (table 5). The crude male:female odds ratio for meeting the criteria for definite or probable AMI given an ICD-9-CM code of 410 was 1.24 (95 percent CI: 1.16, 1.32). However, adjustment for age, ethnicity, hospital type, community, and calendar year removed any association with gender (odds ratio = 1.06). The greater odds of meeting AMI criteria among Whites as compared with Blacks remained statistically significant after adjustment (odds ratio = 1.24, 95 percent CI: 1.13, 1.38). Similarly, the odds that a discharge met ARIC criteria for definite or probable AMI given an ICD-9-CM code of 410 were greater for patients discharged from nonteaching hospitals as compared with teaching hospitals.
|
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
These results have several important implications. First, they underscore the importance of independent validation of hospital discharges for the purpose of monitoring the occurrence of hospitalized AMI in community settings. Although the validity measures remained stable over time, the ethnic and geographic differences indicate that monitoring hospital discharges for ICD-9-CM code 410 alone is not sufficient to accurately assess the disease burden of hospitalized AMI across communities. Estimates of ethnic disparity in the disease burden of hospitalized coronary heart disease that are based on unvalidated hospital discharge surveys may be difficult to interpret.
There are several possible explanations for the ethnic difference we observed. One explanation may relate to the differential use of diagnostic procedures between ethnic groups that we have documented in these communities (23). Another possible reason is that for Black patients, physicians are probably concerned with diagnosing and treating the effects of congestive heart failure and hypertensive heart disease and less concerned with the small electrocardiographic or biomarker changes that may be present to an extent sufficient for the ARIC algorithm to classify these cases as AMI. From a clinical perspective this is certainly appropriate, but it suggests that epidemiologic definitions of AMI need to account for comorbidity, especially for conditions with strong secular trends in hospital admission such as congestive heart failure and diabetes.
Understanding time trends in validity
Putting our findings, especially those on time trends, in context with other studies is complicated by variations in methods. Methodological differences between studies conducted in the past decade make it difficult to infer temporal trends, and to our knowledge ours is the first study to report detailed trends in validity measures throughout the 1990s from a single study using consistent methods. Most published studies have reported sensitivities for ICD-9-CM code 410 higher than those we report, with estimates from studies conducted in recent years generally being greater (47, 9, 12, 13, 16, 17, 2427). Reported data from the World Health Organization MONICA Project (37 populations in 21 countries) for the combined years 19841994 found sensitivities for International Classification of Diseases (Eighth or Ninth Revision) code 410 ranging from 56 percent to 100 percent, with an average of 91 percent (13). Although they did not report gender-specific results, Mahonen et al. (8) found that in Finland for the combined years 19831990, sensitivity ranged from 53 percent to 92 percent across categories defined by diagnostic codes and geographic areas. Pladevall et al. (12) reported data from the Corpus Christi Heart Project on 5,329 events involving hospitalization in 19881990, showing that the sensitivity of ICD-9-CM code 410 in detecting validated definite AMI was 80 percent. Reasons for the difference in sensitivity observed in the Corpus Christi Heart Project and the ARIC Study are probably related to the use of different target ICD-9-CM code groups. ARIC surveillance did not investigate ICD-9-CM code 429 (ill-defined heart disease), code 440 (atherosclerosis), or code 786.5 (chest pain, unspecified). Fewer than 1 percent of events in these additional code groups were validated as definite AMI in the Corpus Christi Heart Project (12). Other validation studies varied in the non-410 codes used in determining sensitivity. Some investigators restricted non-AMI codes to 411414 (4, 24), while others used 412414 only (9) or considered all codes other than 410 as non-AMI codes (5). Our validity estimates were also lower than the estimate found in a recent study of Ontario, Canada, hospitalizations occurring between 1996 and 2000 (16). In that study, Austin et al. (16) reported a specificity of 93 percent, a sensitivity of 89 percent, and a positive predictive value of 89 percent. The gold standard used in the Ontario study was the coronary care unit discharge diagnosis assigned prospectively by the caregiver at the patients bedside, as opposed to the retrospective diagnostic algorithm used in our study. In addition, we used diagnosis codes which are established after discharge to be those chiefly responsible for the admission of the patient to the hospital for care; the Ontario analysis used the "most responsible diagnosis," which emphasizes the most significant condition of the patient that caused his/her stay in the hospital.
The declining trend in the sensitivity and positive predictive value of ICD-9-CM code 410 among teaching hospitals we observed is probably a reflection of changes in diagnostic practices and acute-care treatment strategies occurring in the late 1990s. Our adjusted models showing a lower percentage of events with ICD-9-CM code 410 discharges validated as AMI in teaching hospitals versus nonteaching hospitals agree with a study carried out in the 1980s (17) that found positive predictive values among teaching centers to be dramatically lower in teaching hospitals (58 percent) than in nonteaching hospitals (91 percent).
Although measurements of troponin in the ARIC communities increased from 6 percent of discharges in 1996 to 92 percent of discharges in 2000, the trends in validity measures remained stable during this period. This is somewhat counterintuitive, since the positive predictive value might be expected to decline with the use of more sensitive biomark-ers that would identify smaller infarcts from a clinical perspective yet not result in electrocardiographic changes and/or symptoms sufficient to result in an algorithm-based diagnosis. The ARIC diagnostic algorithm used two times the upper limit of normal in defining abnormal biomarker evidence. The use of other cutpoints may have an effect on estimates of recent trends in discharge code validity. The impact of troponin in classifying more events in the context of clinical practice is varied (28). The true increase attributed to troponin in communities is difficult to ascertain, because some hospitals discontinued the use of other biomarkers, thus complicating direct comparisons of event determination in the pretroponin era and the posttroponin era. These findings of validity in the era of troponin measurement need to be confirmed in other study communities.
Strengths and limitations
The long-term, continuous, and multicenter nature of ARIC community surveillance is a strength of the study, allowing for investigation of validity by gender, ethnicity, geography, and time using a single method. Our study, however, was limited in its inability to evaluate the validity of discharge codes among patients aged 75 years or older. Because the elderly constitute an increasing proportion of persons with acute myocardial infarction, future studies should address the question of whether the validity of discharge codes tracks across older age groups.
Summary
We found that although the validity of ICD-9-CM hospital discharge code 410 to identify AMI was stable from 1987 through 2000, differences between Blacks and Whites and across geographic locations support the investment in validation efforts in ongoing surveillance studies. Sole reliance on ICD-9-CM codes to enumerate myocardial infarction events may be a particular problem in assessing disease burden and trends among Black populations, because of the greater co-occurrence of hypertensive heart disease and congestive heart failure. The measures of validity reported here may help public health researchers interpret the wealth of data available from national hospital discharge surveys, as well as region- or institution-specific hospital discharge databases.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|