1 Centre for Ageing and Public Health, London School of Hygiene and Tropical Medicine, University of London, London, United Kingdom.
2 Section of Care of the Elderly, Faculty of Medicine, Imperial College of Science, Technology, and Medicine, University of London, London, United Kingdom.
Received for publication February 3, 2004; accepted for publication May 14, 2004.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
aged; cognition; mass screening; mortality; sensitivity and specificity
Abbreviations: Abbreviations: CDT, Clock-Drawing Test; CI, confidence interval; HR, hazard ratio; MMSE, Mini-Mental State Examination.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The Mini-Mental State Examination (MMSE) is a well-validated measure of cognitive function (3) and has often been used to measure cognitive impairment in epidemiologic and clinical surveys. It consists of 20 items in seven categories. As a population screening tool for cognitive impairment in the community setting, the Clock-Drawing Test (CDT) warrants consideration as a quicker, simpler, and possibly more cost-effective screening instrument for use in primary care. The reliability and validity of the CDT has been reported (412), but the majority of these studies were based on relatively small sample sizes in hospital- or clinic-based settings. Data on the validity of the CDT in the older general population are needed. Therefore, we conducted a study with the following objectives: 1) to assess the validity of the CDT in comparison with the MMSE as a population screening tool for cognitive impairment and to determine whether validity varies by gender, age group, or method of administration and 2) to examine the association between cognitive function assessed by the CDT and mortality.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
The CDT and the MMSE
Patients interviewed face-to-face by a layperson or nurse for the CDT were given a blank sheet of paper, a pencil, and the following instruction: "Draw a large clock face and put all the numbers in." After this had been done, patients were asked to draw in the clock hands to indicate a time of 10 past 11. For postal administration, patients were provided with a blank sheet of paper in the questionnaire and the instructions given above.
A nurse scored all questionnaires regardless of whether questionnaire administration was postal or by layperson or nurse. Because nurses with no special psychological knowledge rated the CDT in this study, a simple scoring system was used (15). The nurse evaluated three items: "correctly drawn clock shape," "all numbers in the correct position," and "hands of the clock set to the correct time." A score of 1 was assigned for each of these items if the nurse judged that the drawing, the assignment of numbers, and the time were correct; thus, the score could range from 0 (all items incorrect) to 3 (all items correct). If information on any of these three items was missing, the patient was excluded from the analysis. In addition, the nurses were trained to make a judgment as to whether there were abnormalities in the CDT that were not captured by the simple pass/fail scoring described above. For instance, if the drawing was a very disorganized, bizarre, or otherwise abnormal representation of a clock, the nurse recorded the presence of bizarre drawings, and these were included in the final score with a score of 0 if present and 1 otherwise. Therefore, final possible scores ranged between 0 (the worst) and 4 (the best).
Since there were no predefined cutoff points to indicate cognitive impairment, we compared different threshold scores on the CDT with MMSE scores as part of this study. For the MMSE, two cutoff points were used: 23 or less for mild cognitive impairment (16) and 17 or less for moderate-to-severe impairment (17).
Covariates
Through the brief and detailed assessments in the trial, a broad range of patient information was collected. To assess the association between cognition and mortality, we considered the following potential confounders.
Demographic and social variables
Age at brief assessment was categorized into three 5-year age groups (7579, 8084, and 8589 years) and a fourth group of persons aged 90 years or more. Home ownership as a marker of the socioeconomic status of subjects was categorized as "owner occupier" or "renter." Persons living in sheltered or residential placements were regarded as having missing data, since there was no information on their previous residence.
Lifestyle variables
Smoking status was categorized as "nonsmoker," "ex-smoker," and "current smoker." Alcohol drinking was divided into four categories: "never drinker," "ex-drinker," "06 units per week," and "7 units per week." In the United Kingdom, a unit of alcohol is defined as the amount of ethanol contained in half a pint (284 ml) of beer, a single glass (125 ml) of table wine, or a single measure (25 ml) of spirits (18). A unit is equivalent to approximately 10 ml or 8 g of ethanol.
Health status indicators
Reported current or recent history of heart attack, stroke, Parkinsons disease, diabetes mellitus, or cancer was categorized as "yes" or "no." This information was determined by asking patients whether they had ever been told of these conditions by physicians. Self-assessed general health was categorized into "poor/fair," "good," and "very good/excellent." The presence of depression was determined through the use of the Geriatric Depression Scale (19), and subjects were classified as either "normal" (score of 05) or "depressed" (score of 6). Number of falls was categorized as "none," "one," and "two or more."
Statistical analysis
In all analyses, we took account of the stratified cluster sampling by general practice in the estimation of standard errors. All statistical analyses were performed using Stata 7 software (Stata Corporation, College Station, Texas).
Since the participants in the targeted arm with detailed assessments were not a representative sample, only people in the universal arm who responded to both the brief assessment (which included the CDT) and the detailed assessment (which included the MMSE) were entered into the analyses. The sensitivity, specificity, positive predictive value, and negative predictive value of the CDT in comparison with the MMSE, using different CDT cutoff points, were calculated for the whole sample and by gender, age group, and method of administration.
Survival time was defined as time from the date of baseline assessment (between 1995 and 1999) to the date of death, the date of emigration, or December 31, 2001, whichever was earlier. The Kaplan-Meier method was used to plot the unadjusted survivor function by level of baseline cognitive function as assessed by the CDT. To quantify the impact of cognitive function on mortality, we used Cox proportional hazards regression. Adjustment for potential confounders was accomplished via a series of nested models. The initial step included only age group, gender, and home ownership. The next step added lifestyle variables (model 1). The health status indicators described above were then added (model 2). Wald tests were used to determine the statistical significance of variables, to check linearity for categorical variables (by giving consecutive integers to each category), and to evaluate interactions. It was confirmed that all variables included in model 2 were significant predictors of mortality and that there was no evidence of interaction between variables. On the basis of the test for linearity, age group, smoking category, self-assessed general health, and number of falls were included in the model in a linear manner. Because subjects with cognitive impairment might have had preceding diseases that caused both impairment and death, we repeated the analyses after excluding deaths occurring within the first year of follow-up. The proportional hazards assumption was graphically confirmed using an Aalen-Nelson plot of cumulative rate on a log scale (20).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Brief assessments were done by postal questionnaire (19 practices), lay interview (17 practices), and nurse interview (17 practices) for 5,934 subjects (44 percent of the 13,557), 4,254 subjects (31 percent), and 3,369 subjects (25 percent), respectively.
Validity of the CDT
CDT scores were distributed as shown in table 1. If the cutoff point was set at a score of 2 or less, the prevalence of cognitive impairment was 13.6 percent. The prevalence of moderate/severe cognitive impairment by the MMSE was 2.9 percent, and the prevalence of at least mild cognitive impairment was 17.3 percent. Sensitivity, specificity, positive predictive value, and negative predictive value according to different MMSE cutoff points are shown in table 2, where a cutoff point of 2 or less was used for the CDT score. The CDT scores showed higher sensitivity and lower specificity for more severe cognitive impairment than for milder scores. The positive predictive value was lower and the negative predictive value was higher for a cutoff point of 17 or less, reflecting the lower prevalence of moderate/severe cognitive impairment in comparison with milder impairment. Sensitivity tended to be higher in females than in males, and specificity was lower. Among subjects aged 8589 years and 90 years, higher sensitivities and lower specificities were observed when subjects were compared with those in other age groups. Regardless of cutoff points for the MMSE, the postal questionnaire had a lower sensitivity than face-to-face methods. In contrast, lay interviews and nurse interviews provided acceptable sensitivity and specificity if the cutoff point for the MMSE was set at 17 or less, as shown by the receiver operating characteristic curves graphed by administration method (figure 2). For nurse administration, sensitivity and specificity versus the MMSE using a cutoff point of 17 or less were 76.5 percent and 87.1 percent, respectively. If a cutoff point of 1 or less for the CDT was used, corresponding sensitivity and specificity were 46.3 percent and 96.2 percent, showing much lower sensitivity. For a CDT cutoff point of 3 or less, corresponding sensitivity and specificity were 92.7 percent and 68.1 percent, which means that lower specificity would produce many more false-positive cases.
|
|
|
|
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Since mild cognitive impairment may be a sign of early dementia, screening for mild impairment would become more important as new medicines for early treatment became available. However, the CDT alone did not appear to be a good screening tool for mild cognitive impairment, and a two-step strategy (the CDT followed by the MMSE in persons scoring positive) might lead to improved validity. Since there was no clinical measure of dementia for use as a gold standard in this study, it was impossible to assess the validity of the combination of the CDT and the MMSE for detecting early dementia. However, we were able to investigate the effects of both scores on mortality. Hazard ratios adjusted for demographic, social, and lifestyle variables and health status indicators were higher for the combined MMSE and CDT (HR = 2.22, 95 percent CI: 1.85, 2.66) than for the CDT alone (HR = 1.55, 95 percent CI: 1.42, 1.70) and the MMSE alone (HR = 1.74, 95 percent CI: 1.26, 2.39).
The postal questionnaire showed lower sensitivity than the nurse interview, though gender and age distributions were comparable among administration groups reflecting random allocation. It is unlikely that this reflected any bias due to the fact that the nurse both scored the CDT and administered the test, because similar results were observed for administration by the lay interviewer, which was also scored by the nurse. Response bias might also have affected the estimates of validity by method of administration. Of subjects who participated in the brief assessments, 9.6 percent of the postal administration group did not respond to the CDT, as compared with 4.5 percent for nurse administration and 3.9 percent for lay administration. Women, older people, nonhomeowners, and depressed subjects were less likely to complete the CDT. These factors were associated with worse MMSE scores. If cognitively impaired patients, who could be easily detected by both the CDT and the MMSE, were more likely to refuse the CDT, sensitivity would be underestimated. Indeed, the prevalence of subjects being assessed as cognitively impaired by the CDT was lower for postal administration (10.2 percent, 95 percent CI: 8.5, 12.2) than for lay administration (16.8 percent, 95 percent CI: 12.8, 21.7) and nurse administration (15.5 percent, 95 percent CI: 13.3, 17.9), which suggests that differences in response rates might partly account for the lower sensitivity for postal administration. Additionally, for postal administration, family members might have assisted the elderly person, and this would have resulted in more false-negatives. This suggests that, for the CDT as a population screening tool, face-to-face administration may be preferable to postal administration. The validity also varied by age group; this variation was independent of gender or administration methods (data not shown). The reason for this interesting variation is unclear, and additional studies using the same scoring and cutoff points in other populations are needed.
In contrast to almost all previous studies, the standard used in this study was the MMSE rather than clinical diagnosis of dementia, because an extensive clinical evaluation of over 13,000 elderly patients was impractical. Although the MMSE is a tool that is commonly used to detect cognitive impairment, it is not a diagnostic test. Cognitive function detected by the CDT might be different from that detected by the MMSE (5). Some studies found only moderate correlations between the CDT and the MMSE in clinical settings (7, 22). Evaluation of the validity of the CDT against a clinical diagnosis of dementia as a gold standard is still needed. The educational level and cultural background of participants may influence the validity of the CDT (9). There were very few older participants from minority ethnic groups in our study, reflecting the very low proportion of older people of minority ethnicity currently in the British population. We did not ask about educational level. Housing tenure, which might be considered a weak proxy measure of educational attainment, was associated with the CDT score in unadjusted analyses and also after adjustment for other confounders. However, the validity of the CDT varied little by housing tenure. For nurse administration, sensitivity and specificity were 75 percent and 90 percent, respectively, for people who owned their homes and 71 percent and 84 percent, respectively, for those in rental accommodations.
We did not consider the visual acuity of the participants in evaluating CDT performance. Exclusion of those with binocular presenting acuity less than 6/18 (the conventional definition of vision impairment) reduced sensitivity and increased specificity slightly; however, this was entirely due to the effect of age. People with poor visual acuity were older, and the CDT was less sensitive and specific among older people.
The role of the CDT as a screening tool has been evaluated in previous studies (6, 9, 2325). In a review article, Shulman (9) reported that most studies achieved sensitivities and specificities of approximately 85 percent and concluded that the CDT, in conjunction with other widely used tests such as the MMSE, could provide a significant advance in the early detection of dementia. In contrast, Powlishta et al. (24) concluded from their study that the CDT did not appear to be a useful screening tool for detecting very mild dementia. Other authors have concluded that the CDT should not be used alone as a dementia screening test because of its overall inadequate performance (25, 26). However, most previous studies were based on relatively small sample sizes or were undertaken in a clinical setting, and their results may not be applicable to a larger community population. Paganini-Hill et al. (27) conducted a large postal questionnaire survey including the CDT among 5,000 community-dwelling elderly persons and concluded that the CDT might help predict cognitive frailty and future disability. However, the validity of the CDT was not examined in their study.
There are various scoring systems for the CDT and no universally accepted standards. More complex scoring systems require greater rating skill and increase training time. In our study, because nurses with no special training in the detailed interpretation of the test were to be used, we adopted a simple scoring method previously used in the CAMCOG battery (the cognitive portion of the Cambridge Examination for Mental Disorders of the Elderly) (15, 28) that we considered suitable for mass screening of the elderly. Nurses were trained to recognize drawings as bizarre/disorganized using examples or a general impression, as was also done in other studies (4, 7, 8). Recently, Scanlan et al. (29) found that a simple binary rating (normal or abnormal) made by naive raters was surprisingly effective in detecting dementia.
The association of MMSE scores with survival has previously been reported (3033). This study provided the first opportunity to assess the predictive validity of the CDT with regard to mortality. The result revealed that cognitive impairment assessed by the CDT was associated with higher mortality across all levels of cognitive function. Though the mechanism for the association between cognitive impairment and mortality has not been clear, there is a hypothesis that this relation could be explained by the preceding condition of patients. Although inclusion of health indicators such as current/past diseases and self-assessed general health in the model did reduce this association, it still remained significant. Moreover, exclusion of subjects who died within 1 year of the baseline assessment did not alter the result. Cognitive impairment seemed to be an independent risk factor for mortality in this study population, though residual confounding or confounding by other factors not included in this study should be taken into consideration.
Cognitive impairment was associated with risk of death from cerebrovascular disease, respiratory disease, and malignant neoplasms. The increase in cerebrovascular disease mortality suggests that a proportion of cognitive impairment in this study population was vascular in origin. Increased mortality from respiratory disease was consistent with the findings of another study (34). Although the pathway is unclear, death due to respiratory disease has already been found to be a common cause of death among persons with Alzheimers disease (35). In contrast to the case with cerebrovascular and respiratory diseases, the association with cancer death was not observed across all levels of cognitive function. Therefore, this could be a chance finding.
This analysis was based on 64 percent of eligible subjects in the universal arm. Because there were missing values for some covariates, multivariate analyses for the association between the CDT and mortality were restricted to a subset of 11,694 subjects. Unless the association between cognition and mortality was different among subjects who were excluded from the analysis, this would not have introduced selection bias. One drawback of the present study was that data on all possible confounders except age and gender were self-reported. Since cognitively impaired subjects could have memory disturbances, recall among those subjects may have been less reliable. If subjects with cognitive impairment were less likely to recall a history of stroke, the prevalence of reported stroke would have been underestimated among subjects with cognitive impairment; this would have led to overestimation of the hazard ratio.
We found that the CDT is a quick and easily completed face-to-face screening tool for detecting moderate/severe cognitive impairment in a community elderly population. The CDT had acceptable validity in comparison with the MMSE using simple scoring by a nurse without special knowledge. However, the standard used in this study was not clinical diagnosis. Considering the many false-positive cases generated by the CDT, we caution against using the CDT as a stand-alone instrument to screen for cognitive impairment.
![]() |
ACKNOWLEDGMENTS |
---|
The authors thank the nurses, general practitioners, and other staff in the participating medical practices and Judith Nickson at the University of Cambridge for training nurses in the administration of the Clock-Drawing Test and the Mini-Mental State Examination.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|