a Department of Public Health, Chung Shan Medical and Dental College, Taichung, Taiwan.
b Department of Family Medicine, Chung Shan Medical and Dental College Hospital, Taichung, Taiwan.
Reprint requests to: Dr Ming-Chih Chou, Department of Family Medicine, Chung Shan Medical and Dental College Hospital, 23, Section 1, Taichung Kang Road, Taichung 403, Taiwan. E-mail: robertlu{at}ms1.hinet.net
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Method A systematic sample of 5621 death certificates was reviewed. The underlying cause of death (UCD) selected by the reviewer for each death certificate was compared with that selected by the original coder. The UCD was selected according to ACME (Automated Classification of Medical Entities) Decision Tables.
|
Conclusion This study showed various levels of agreement for different diseases between the reviewer and the original coders in selection of the UCD. Owing to the compensatory effect of errors, the national mortality statistics were not affected significantly. The national administration should undertake routine internal studies to control the quality of UCD coding practices.
Keywords Cause of death, coding, ICD-9, death certificates, mortality, Taiwan
Accepted 15 July 1999
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Despite the detailed and specific rules provided in the ICD for coding death certificate data, coding practices still vary substantially among countries.411 Some studies have focused on the coding problems related to specific diseases in particular counties, such as hypertension,12 rheumatoid arthritis,13,14 myocardial infarction,15 Down syndrome16 and short gestation.17 Few studies have discussed the types of coding errors and the reasons they occur,4,6,14 and only one study has provided corrected mortality rates after adjusting for coding discrepancies.6
The objectives of this study were to assess the accuracy of cause-of-death coding in Taiwan and to determine the extent to which coders follow the selection rules of coding set out in the ICD. We also looked at which diseases were overcoded and which were undercoded, and the effects of miscoding on mortality statistics in Taiwan.
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In 1994, 112 238 death certificates were issued in Taiwan. We chose to use a 5% sample for this study because we determined that this would yield sufficient numbers (i.e. >5) in each cross-tabulation cell, especially in the analysis by ICD chapter. Death certificates were systematically sampled (1 of every 20). Because there was little possibility of cyclic characteristics in this series of numbers, we did not use a randomly selected starting point. Thus, the first author reviewed 5621 death certificates and assigned a UCD for each, with strict adherence to the International Classification of Diseases, 9th Revision (ICD-9) selection rules (Appendix). Because the interpretation of Selection Rule 3 and Rule 7 is highly controversial, ACME (Automated Classification of Medical Entities) Decision Tables18 were used to determine whether they were applied properly. The ACME Decision Tables
were also used to determine whether the causal sequence of causes of death was highly improbable or not.
Before the formal review, 200 death certificates randomly selected from the 5621 certificates were reviewed by the first author on two occasions, one month apart. The UCD were not concordant between the two reviews for only three of the 200 death certificates. The reasons for the discrepancies were inconsistent interpretation of Rule 3 for septic shock and sepsis. The interpretations of these situations were unified for the formal review.
The UCD selected by the reviewer were then compared with the UCD selected by the original coders. The UCD selected by the reviewer was viewed as the gold standard to calculate the agreement rate, false-positive rate, false-negative rate, and kappa values.19 The methods of calculating different indicators of agreement are illustrated as follows:
Chi-square analysis was used to test the differences in agreement rates according to various characteristics (sex and age of the deceased, number of diagnoses per death certificate, different coders). We further classified the disagreements into different types according to the reason for disagreements. The death rates estimated according to the reviewer and the original coders were compared to evaluate the effects of coding errors on mortality statistics. Age-adjusted mortality rate ratios (reviewer/original coder) and 95% CI20 were calculated to determine the extent of overcoding (mortality rate ratio <1) and undercoding (mortality rate ratio >1) for main causes of death.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Factors associated with the level of agreement
Overall, the agreement rates between original coders and reviewer according to ICD-9 three-digit and two-digit categories were 80.9% and 83.9%, respectively. The agreement rates decreased as the age of the deceased increased, and as the number of diagnoses entered on the death certificate increased (Table 1). According to the two-digit categories, the agreement rates varied from 82.4% to 86.6% between the reviewer and each of the five coders. The variation increased (78.284.2%) if the three-digit category was used.
|
|
The agreement was good for malignant neoplasms (kappa = 0.94) and injuries and poisoning (kappa = 0.97), and poor for nephrotic diseases (kappa = 0.74), hypertension-related diseases (kappa = 0.74), and cerebral infarction (kappa = 0.77) by main causes of death (Table 3). The false-positive rate represents the degree of overcoding of a disease by the original coder as judged the reviewer. The false-negative rate, on the other hand, expresses the degree of undercoding of a disease by the original coder in comparison with the reviewer. Liver cancer (20%), cerebral infarction (29%), nephrotic diseases (34%), and hypertension-related diseases (29%) had high false-positive rates, while diabetes (1.8%) and liver cirrhosis (1.1%) had high false-negative rates.
|
Effects of disagreements on mortality statistics
The age-adjusted death rates (estimated according to the reviewer) and the death rate ratios (reviewer/original coders) were calculated for men (Table 4) and women (Table 5
). The mortality rate ratio (MRR) is the net effect of false-positive and false-negative coding. Only for male liver cancer (MRR = 0.82, 95% CI : 0.670.99) and male nephrotic diseases (MRR = 0.69, 95% CI : 0.470.99) did the mortality rate estimated by the reviewer differ significantly from the mortality rate estimated by the original coder.
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The European Economic Community (EEC) studies showed great variations in coding practices among different countries.69 The investigators sent sets of case histories to samples of certifying physicians of different countries and the completed certificates were coded by national coding offices and by a WHO reference centre (as gold standard). The coding agreement rates between countries ranged from 60% to 92% in cases of COPD,6 80% to 94% in cases of cancer,7 and 29% to 98% in cases of diabetes.9 Our study showed poor coding agreement for liver cancer, cerebral infarction, nephrotic diseases, hypertension-related diseases, diabetes, and liver cirrhosis. There were several sources of disagreement, including choosing different codes, inappropriate judgement of causal relationship, and incorrect interpretation of Selection Rule 3 and Modification Rules.
Types and mechanisms of disagreement
The first type of disagreement involved the nomenclature of cause of death. Percy and Muir6 identified several sources of problems for coding death certificates mentioning cancer, including multiple sites of cancer, coexistent heart diseases and other cancers, interpretation of the coding rules, and translation problems. The translation problem is serious in Taiwan, because most medical doctors use English medical terms in clinical practice, while the death certificates are completed in Chinese. Different doctors might use different translations for the same English diagnosis. Because there is no Chinese version of the ICD-9 Index, the coders might also choose different codes. In fact, this was the most frequent type of disagreement between the reviewer and coders in our study (42.8%).
Another problem related to translation is that there are some controversial terms with diverse connotations. Although the term neoplasm can denote either a malignant or a benign tumour depending on how it is further qualified, some certifiers interpret it as meaning a malignant tumour. The reviewer coded liver tumour as ICD-9 code 235 (neoplasm of uncertain behaviour of digestive and respiratory systems); however some original coders coded the same diagnosis as 155 (malignant neoplasm of liver and intrahepatic bile ducts), which caused the high false-positive rate for liver cancer.
The second type of disagreement was inappropriate judgement of causal relationships (41.5%). There was considerable divergence among the coders themselves and between some of the coders and the reviewer on the extent to which they were prepared to accept the statement of causation made by the certifying doctor. This divergence of approach is exemplified by one of the disputed death certificates, which read: I (a) Cerebral infarction (b) Diabetes mellitus (c) Hypertension. II Gastric cancer with metastases. The first coder might accept the certifier's statement of causation among different diseases, and selected hypertension as the UCD, according to the General Rule. The second coder might also accept the certifier's statement of causation, but used Rule 7 and selected cerebral infarction as the UCD. The third coder might judge that diabetes was highly improbable as being due to hypertension, so this coder rejected the certifier's statement of causation and selected diabetes as the UCD according to Rule 1. The fourth coder might maintain that the metastasizing gastric cancer was the most serious condition, so this coder incorrectly interpreted Rule 3 and selected gastric cancer as the UCD. Thus, it appears that the judgement of the acceptance or rejection of the certifier's statement of causation is a key source of disagreement.
A study from Sweden showed that the coder's judgement and interpretation of the medical certificate play a decisive role in the data-collecting process for rheumatoid arthritis (RA) mortality statistics.13 In some situations the coders in the National Central Bureau of Statistics of Sweden had a preference for choosing RA as the UCD.14 As shown in Table 2, 46 cases in which the UCD selected by the reviewer was an endocrine disease (diabetes in most cases), the original coder determined a circulatory disease (most of them were CVA) to be the UCD. In these cases, the CVA was put on the first line and at the same time diabetes was on the last used line in Part I (of the cause-of-death section in the death certificate); in this situation, coders in Taiwan appear to prefer choosing CVA as the UCD.
A similar situation existed when liver cirrhosis and liver cancer were put on the same first line, with liver cirrhosis mentioned first: the reviewer chose liver cirrhosis as the UCD according to Rule 1, while most of the coders in Taiwan chose liver cancer. The coders' preferences for these diseases were according to the principle of severity, and this is the reason why diabetes and liver cirrhosis were undercoded. The false-negative rate for the two diagnoses were among the highest: 1.80% for diabetes and 1.09% for liver cirrhosis, respectively.
The third type of disagreement was incorrect interpretation of Rule 3 and the Modification Rules. As the EURODIAB study10,11 revealed, diabetes is a disease with certification and coding problems. Deciding whether diabetes is the cause of the death or a contributing factor is often difficult. Thus, diabetes may be undercoded in some situations and overcoded in others. If diabetes is put in Part II (of cause-of-death section in the death certificate) by the certifier, some coders in Taiwan might incorrectly use Rule 3 to select diabetes as the UCD. This is why there were 23 cases in which the reviewer selected CVD as the UCD while the original coder determined diabetes (in Part II) to be the UCD.
The interpretation of Rule 3 is always controversial. For example, in 1984 the Office of Population Censuses and Surveys introduced a revised interpretation of the WHO's Rule 3 in England and Wales.23 As a result, there was a large decrease in the number of reported deaths from causes such as pneumonia, while deaths from causes often mentioned in Part II increased in number.24 The automated cause coding system was implemented in England and Wales at the beginning of 1993, and the mortality rate from pneumonia reversed to the level of 1984.25 The change was caused by the fact that Rule 3 is interpreted differently in the US, where the automated system was designed.
Another example of the controversy in the interpretation of Rule 3 is the experience in Sweden, where the coders of the Central Bureau of Statistics routinely queried the death certificates with pneumonia reported as the UCD. The Bureau conducted a retrospective study to abstract the medical records of decedents with pneumonia selected as the UCD on the death certificate. The study found that in about one-quarter of cases a condition mentioned in Part II was indeed the UCD; in another quarter of the cases the pneumonia was due to a serious disease that had not been mentioned on the certificate; and in the remaining half of the cases the pneumonia itself was judged to be the true originating cause of death.26
Effects of disagreements on mortality statistics
Though the false-positive rates and false-negative rates were high for some diseases, the net effect of coding disagreements on most specific mortality rates was not significant in our study. This is possibly because of the compensatory effect of errors', which was first proposed 40 years ago.22 For example, diabetes may be undercoded in some situations and overcoded in others, so the net effect was not significant in our study (MRR for diabetes was 0.96 for males and 1.02 for females).
The overcoding of cancer of the liver (MRR 0.82 for males and 0.75 for females by the original coders) can be further illustrated by comparing the incidence rates and mortality rates of this disease between Taiwan and the US. The incidence of hepatocellular carcinoma (HCC) in the US in 1994 was 4.0 per 100 000 for males and 1.4 per 100 000 for females; because the survival of HCC patients is poor, the mortality and incidence were virtually identical.27 Nevertheless, the mortality rate of HCC in Taiwan in 1993 (36.9 per 10 000 for males and 12.2 per 100 000 for females) was one-third higher than the incidence rate (25.6 per 100 000 for males and 8.7 per 100 000 for females),28 reflecting the overcoding of this disease in Taiwan.
The accuracy of coding for the general disease categories (e.g. the 10 leading causes of death) in Taiwan appears to be adequate, in that the ranking of these causes was not changed after review. Nevertheless, for diseases with small numbers of deaths (e.g. pregnancy-related deaths) the coding disagreements did have great impacts on mortality rates.
Limitations
Although we did not attempt to assess the accuracy of the diagnosis reported on the death certificates, errors in the way the doctor converts basic information about the patient into a diagnosis and completes the death certificate should also be noted as potential sources of error.2935 Sources of error include the amount of information in cause-of-death diagnosis, the competence of the physician in deciding the main cause-of-death chain, and the wording in death certificate completion.
This study followed the same method used by previous investigations,411 using the UCD selected by the reviewer as the gold standard for assessing the accuracy of coding. However, because selecting the UCD requires difficult interpretations of various selection rules, the process cannot be entirely objective. Nevertheless, in this study we not only showed the extent of the disagreements, but also the types and possible reasons for the disagreements. This information is critical for improving the cause-of-death coding system.
Implications for improvements
As we expected, the selection of a single UCD became more difficult in those who were older and had more morbid conditions. Some scholars have recommended the coding and publication of statistics on multiple causes of death to correct the problem of disagreement in selection of the UCD.3639
The Head of the WHO Collaborating Center for Classification of Disease in Beijing has translated the ICD-10 index into Chinese. Nonetheless, the Chinese characters and medical terms used in Taiwan are quite different from those used in mainland China. Almost all physicians in Taiwan use English for writing medical records, and nosologists in hospitals use the English index of the ICD for coding. The Taiwan Department of Health is still evaluating the need for translation of the ICD-10 index into Taiwanese.
The agreement rates between the reviewer and each of the five coders were quite low compared with other studies.1115 The Office of Statistics of the Provincial Government of Taiwan has now implemented routine double coding of samples of the death certificates completed by the five coders, to evaluate and improve the quality of the coding practices. Some of the coding problems discussed above could be resolved by using the ACME (Automated Classification of Medical Entities) computer program.18,25 The US National Center for Health Statistics sponsored a conference on an International Collaborative Effort on Automating Mortality Statistics, which was held in November 1996. A mortality coding forum on ICD-10 selection rules has been established on the Internet by Statistics Sweden (http://www.pubcare.uu.se/nordwho/verksam/mortfor.htm), to allow the coders in different countries to share their experiences and discuss coding-related questions. It is scheduled that ACME will be implemented when the ICD-10 is adopted in Taiwan.
In conclusion, the accuracy of cause-of-death coding in Taiwan appears to vary according to the type of disease. Types of disagreements between reviewer and the original coder included disagreement in nomenclature, inappropriate judgement of causal relationships, and incorrect interpretation of Selection Rule 3 and Modification Rules. Because of the compensatory effect of errors, the national mortality statistics were not affected significantly.
![]() |
Appendix |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Rule 1. If there is a reported sequence terminating in the condition first entered on the certificate, select the underlying cause of this sequence. If there is more than one such sequence, select the underlying cause of the first-mentioned sequence.
Rule 2. If there is no reported sequence terminating in the condition first entered on the certificate, select this first mentioned condition.
Rule 3. If the condition selected by the General Rule or Rule 1 or 2 can be considered a direct sequel of another reported condition, whether in Part I or Part II, select this primary condition.
Modification of the underlying cause of death selected by the above rules
Rule 4. Senility.
Rule 5. Ill-defined conditions.
Rule 6. Trivial conditions.
Rule 7. Linkage.
Rule 8. Specificity.
Rule 9. Early and late stages of disease.
Rule 10. Late effects.
Rule 11. Old pneumonia, influenza and maternal conditions.
Rule 12. Errors and accidents.
![]() |
Acknowledgments |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2
Start RD, Bury JB, Strachan AG, Cross SS, Underwood JCE. Evaluating the reliability of causes of death in published clinical research. Br Med J 1997;314:271.
3 World Health Organization. Manual of the International Statistical Classification of Diseases, Injuries, and Cause of Death, Vol 1. Geneva: WHO, 1977.
4 Anonymous. The accuracy and comparability of death statistics. WHO Chronicle 1967;21:1117.[ISI][Medline]
5 Percy C, Dolman A. Comparison of the coding of death certificates related to cancer in seven countries. Public Health Rep 1978;93:33550.[ISI][Medline]
6 Percy C, Muir C. The international comparability of cancer mortality data. Am J Epidemiol 1989;129:93446.[Abstract]
7 Kelson MC, Heller R. The effect of death certification and coding practices on observed differences in respiratory disease mortality in 8 EEC countries. Rev Epidemiol Sante Publique 1983;31:42332.[ISI][Medline]
8 Kelson M, Farebrother M. The effect of inaccuracies in death certification and coding practices in the European Economic Community (EEC) on international cancer mortality statistics. Int J Epidemiol 1987;16:41114.[Abstract]
9 Mackenbach JP, Van Duyne WMJ, Kelson MC. Certification and coding of two underlying causes of death in the Netherlands and other countries of the European Community. J Epidemiol Community Health 1987;41:15660.[Abstract]
10 Jougla E, Papoz L, Balkau B, Maguin P, Hatton F, the EURODIAB Subarea C Study Group. Death certificate coding practices related to diabetes in European CountriesThe EURODIAB Subarea C study. Int J Epidemiol 1992;21:34351.[Abstract]
11 Balkau B, Jougla E, Papoz L, the EURODIAB Subarea C Study Group. European study of the certification and coding of causes of death of six clinical case histories of diabetic patients. Int J Epidemiol 1993; 22:11626.[Abstract]
12 Curb JD, Babcock C, Pressel S, Tung B, Remington RD, Hawkins CM. Nosological coding of cause of death. Am J Epidemiol 1983;118: 12228.[Abstract]
13 Lindahl BIB. The reliability of Swedish mortality statistics for rheumatoid arthritis. Scand J Rheumatol 1984;13:28996.[ISI][Medline]
14 Lindahl BIB. In what sense is the rheumatoid arthritis the principal cause of death? A study of the National Statistics Office's way of reasoning based on 1224 death certificates. J Chron Dis 1985;38:96372.[ISI][Medline]
15 Guibert RL, Wigle DT, Williams JI. Decline of acute myocardial infarction death rates not due to cause of death coding. Can J Public Health 1989;80:41822.[ISI][Medline]
16 Baird PA, Sadovnick AD. Underlying causes of death in Down syndrome: accuracy of British Columbia death certificate data. Can J Public Health 1990;81:45661.[ISI][Medline]
17 Carver JD, McDermott RJ, Jacobson HN et al. Infant mortality statistics do not adequately reflect the impact of short gestation. Pediatrics 1993;92:22932.[Abstract]
18 National Center for Health Statistics. Instruction Manual Part 2C: ICD-9 ACME Decision Tables for Classifying Underlying Causes of Death. Hyattsville: US Department of Health and Human Services, Public Health Services, 1987.
19 Fleiss JL. Statistical Methods for Rates and Proportions. New York: Wiley, 1981, pp.21225.
20 Ahlbom A. Biostatistics for Epidemiologists. Boca Raton: Lewis, 1993, pp.6466.
21 World Health Organization. World Health Statistics 1993. Geneva: WHO, 1994.
22 James G, Patton RE, Heslin AS. Accuracy of cause-of-death statements on death certificates. Public Health Rep 1955;70:3951.
23 Bain MRS, Chalmers JWT, Brewster DH. Routinely collected data in national and regional databasesan under-used resource. J Public Health Med 1997;19:41318.[Abstract]
24 Martyn CN, Pippard EC. Usefulness of mortality data in determining the geography and time trends of dementia. J Epidemiol Community Health 1988;42:13437.[Abstract]
25 Rooney C, Devis T. Mortality trends by cause of death in England and Wales 198094: the impact of introducing automated cause coding and related changes in 1993. Pop Trends 1997;86:2935.
26 Johansson LA. Need for clarification of the ICD-10 mortality coding rules. Paper presented in the Meeting of Heads of WHO Collaborating Centers for the Classification of Diseases; 1996 October 1521; Tokyo, Japan.
27 London WT, McGlynn KA. Liver cancer. In Schottenfeld D, Fraumeni JF (eds). Cancer Epidemiology and Prevention, 2nd edn. New York: Oxford University Press, 1996.
28 Department of Health. Cancer Registry Annual Report, Republic of China, 1993. Taipei: Department of Health, Taiwan, Republic of China, 1996.
29 Leadbeatter S. Semantics of death certification. J Royal Coll Phys London 1986;20:12932.[ISI][Medline]
30 Zumwalt RE, Ritter MR. Incorrect death certificationan invitation to obfuscation. Postgrad Med 1987;81:24554.[Medline]
31 Jordan JM, Bass MJ. Errors in death certificate completion in a teaching hospital. Clin Invest Med 1993;16:24955.[ISI][Medline]
32 Messite J, Stellman SD. Accuracy of death certificate completionthe need for formalized physician training. JAMA 1996;275:79496.[Abstract]
33 Maudsley G, Williams EMI. Inaccuracy in death certificationwhere are we now? J Public Health Med 1996;18:5966.[Abstract]
34 Magrane BP, Gilliland MGF, King DE. Certification of death by family physicians. Am Fam Physician 1997;51:143338.
35 Peach HG, Brumley DJ. Death certification by doctors in non-metropolitan Victoria. Aust Fam Physician 1998;27:17882.[Medline]
36 Israel RA, Rosenberg HM, Curtin LR. Analytical potential for multiple cause-of-death data. Am J Epidemiol 1986;124:16179.[ISI][Medline]
37 White MC, Selvin S, Merrill DW. A study of multiple causes of death in California: 1955 and 1980. J Clin Epidemiol 1989;42:35565.[ISI][Medline]
38 Crew DE, Stamler J, Dyer A. Conditions other than underlying cause of death listed on death certificates provide additional useful information for epidemiological research. Epidemiology 1991;2:27175.[Medline]
39 Westerling R. Small-area variation in multiple causes of death in Swedena comparison with underlying causes of death. Int J Epidemiol 1995;24:55258.[Abstract]