a Writing committee: Michael E Dewey, Christine J Parker and the Analysis Group of the MRC-CFA Study.
Reprint requests to: Dr ME Dewey, Trent Institute for Health Services Research, Medical School, Queen's Medical Centre, Nottingham NG7 2UH, UK. E-mail: michael.dewey{at}nottingham.ac.uk
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Objective To compare information given by elderly people on their past and current health and family history of illness with similar information obtained from a relative, friend or carer.
Design Screening and assessment stages of a prevalence study.
Setting Three urban and two rural areas of England and Wales.
Participants A subsample weighted by age and cognitive status of random population samples of people 65 years, living in their own home or in a residential or nursing home, interviewed between 1991 and 1994. A relative, friend or carer identified by each elderly person to provide proxy information.
Interview Computerized schedules including items on demographic details, cognitive function, lifetime illnesses, current health problems and family history of illness.
Results The rate of proxy don't know responses and the agreement between the elderly person and their proxy were calculated for each item, both for the overall sample and for subgroups based on characteristics of the respondent and of the proxy and on the relationship between them. Higher don't know rates were found to be particularly associated with more distant relationships, questions on family history, a shorter length of time known and a lack of co-residence. Agreement was strongly related to the nature of the question and less to co-residence, with other factors such as relationship having much smaller effects.
Conclusions Proxy information on past and current health problems can be almost complete and in good agreement with self-report, particularly where the proxy lives with the respondent. On family history of illness, history of head injury or boxing and current sleep problems, proxy information is likely to be less complete and show poor agreement. A proxy who is not a close relative is likely to give less complete information but agreement will not be substantially lower.
Keywords Elderly, health problems, self report and proxy information
Accepted 5 April 1997
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
There has been a growing recognition in the epidemiological literature of the need to evaluate proxy information, and two issues which have been highlighted are those of completeness and accuracy.19 Both have important implications for study results, and previous investigations have found them to be related to the question being asked, the method of data collection, and characteristics of the informant and the index person such as the relationship between them. However, few of the studies carried out to date have had sufficiently large sample sizes to do more than point towards possible factors influencing the value of proxy information.
Data collected by the Medical Research Council Cognitive Function and Ageing Study (MRC-CFAS) provide an opportunity to compare self-report with information from a relative, friend or carer for nearly 2000 pairs. The questions concern lifetime illnesses, current health problems and family history of illness, and the sample size is large enough to enable comparisons to be made between the two sources of information both overall and also by subgroups based on various characteristics of the respondent and the proxy. The focus of the present paper is mainly a descriptive presentation of the data. For clarity, the person on whom the health data was sought will be referred to as the proband throughout and the relative, friend or carer providing proxy information as the informant.
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The proband and informant interviews
The whole sample received a face-to-face screening interview containing items on socio-demographic variables, cognitive function (including the Mini Mental State Examination11 and the GMS AGECAT12), activities of daily living, family history of illness, and lifetime health problems and events including a suggested risk set for dementia.13 These interviews were carried out between 1991 and 1994 by centrally trained interviewers using laptop computers which automatically provided appropriate routing through the questionnaire. For example, failure to respond correctly to orientation items at the beginning of the interview caused much of the questionnaire to be skipped and only selected memory and cognition items to be displayed. Because of this the items on past and current health were not asked of the most cognitively frail. The software also selected for more detailed study a subsample of just over 20% of those screened, weighted on the basis of their age and their cognitive status as measured by MMSE and AGECAT. This subsample was to provide the basis for estimates of prevalence and incidence of dementia in the main study and also to provide cases and controls for risk factor analysis. For this subsample, in addition to a more detailed assessment interview, including items required for the diagnosis of dementia and depression, the interviewer identified a relative, friend or carer to complete an informant interview, the History and Aetiology Schedule. Interviewers were instructed that if proband and informant lived together every effort should be made to interview each in private, and where a face-to-face informant interview was not possible one was conducted by telephone. The median time interval between the screening and informant interviews was 87 days.
Analysis
Out of a total of 2201 probands with an informant interview, 145 have been excluded from the analysis because the interviewer rated either the information from the proband or from the informant as unreliable. For a further 158 none of the health items in the proband interview was answered, for example because the proband's cognitive state prevented completion of the interview. This leaves a total of 1898 proband-informant pairs with data believed to be of reasonable quality from both sources, and these form the basis of the analyses presented here.
The interview schedules were derived from existing established schedules and do not contain directly comparable questions to proband and informant on every health item. Some conditions have been excluded from analysis for this reason; for others a comparison has been made even though wordings to proband and informant are somewhat different. Details of all the interview items considered in this paper are given in the Appendix.
Some variables from the informant interview have been combined to enable them to be compared with variables in the proband interview. Asthma, arthritis and bronchitis are each a combination of two informant variables, one for whether the proband suffered from the condition in the past and one for whether they currently suffer from it. Information on heart attack is taken from three variables: whether the proband has had any heart trouble either in the past or currently, and if so, whether a heart attack was diagnosed. Head injury here refers to traumatic injury with loss of consciousness and is computed from two informant variables, one on any serious accident affecting the head and one on whether a period of unconsciousness followed. Three variables on different sleeping problems have been combined (getting to sleep, wakefulness during the night, waking before normal time). Two questions elicit information on whether the proband has ever drunk alcohol: whether they currently drink it and if not, whether they have ever done so. The only questions from the proband interview to be combined are on family history of dementia, where there are separate variables on senility or dementia and on Alzheimer's disease.
For some purposes, those items relating to the proband's lifetime illnesses have been considered separately, and the items on other factors have been italicized in the Tables for ease of reference.
For sub-group analysis the different relationships between proband and informant have been divided into five categories: spouse, sibling, child, friend and other. The latter is a disparate group in which the majority are non-first-degree relatives such as niece, grandchild or relation by marriage, and the remainder are matrons or wardens of homes, care workers or neighbours: sample numbers necessitate these being aggregated. Probands have also been divided into the cognitively well (MMSE score >21 and AGECAT organic syndrome level <3) and the cognitively impaired.
To measure agreement between probands and informants on each question and to enable comparisons to be made between questions and between subgroups of informants, three measures have been computed: kappa, positive agreement and negative agreement. These are the measures most frequently reported by investigators in this area and will allow comparison with other studies. Of the possible methods of calculating positive and negative agreement the one chosen is:
![]() |
Because of the longitudinal nature of the CFA Study, versions of the research database are released at intervals: the results reported here are based on version 3 released in October 1995.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Table 1 shows the characteristics of the sample, both overall and by category of relationship. Spouse informants were typically older and much more likely to live with the proband. This group contains the highest proportions of male informants and of interviews conducted with the proband present, and also the lowest proportion of cognitively impaired probands. Sibling informants also tended to be older: all had known the proband for more than 25 years but frequency of face-to-face contact was typically lower. Children were a younger group with contact being predominantly daily to weekly: the highest proportion of interviews by telephone was in this group. Informants who were friends were more likely to be female and typically had as frequent contact as children but had known the proband for a shorter time. The other informant group was similar to friends in degree of contact but was younger and had the highest proportion of cognitively impaired probands. We have no data on the cognitive status of informants, except that any overt impairment would have led the interviewer either to seek an alternative informant, or to rate the informant data as unreliable (which would have led to that pair being excluded from this analysis).
|
Table 2 shows the magnitude of both these types of non-response. For each question, the number of pairs with missing data is shown as a percentage of the total number of pairs. The number of DK responses is shown as a percentage of total valid responses for each question. Items are listed in ascending order of their overall DK rate and the Table also shows a breakdown of DK rates for the five relationship groups, with the lowest rate for each item being highlighted.
|
Table 2 also shows substantial differences in DK rates between the five categories of relationship. For lifetime illnesses, spouses and first-degree relatives typically have very low rates: many are zero and none are above 1% except for thyroid disease and high blood pressure, where siblings especially have a higher rate. In general, friends and others have much higher rates for these health questions, with friends having rates over 9% for bronchitis, meningitis/encephalitis and high blood pressure. Stroke and heart attack have low rates for all categories, as do the questions on hearing impairment and memory problems. Siblings surprisingly were more likely to give a DK response to the question on head injuries than were either spouses or children. The highest rates appear in the questions on boxing history and family history of illness: for these three the rates for siblings are zero, for spouses and children somewhat higher, and for friends and others strikingly high between 14.5% and 21.4%. For sleep problems, siblings and friends have the highest DK rates. The last row of Table 2
summarizes the 19 questions reported and confirms the overall lower DK rates in spouses and first-degree relatives.
The effects of various proband and informant characteristics on DK rates have been examined: of these, proband age and proband sex showed no noticeable pattern, and the results for proband cognitive status, informant sex, informant age, length of time known and co-residence are shown in Table 3. Again, items are ordered by their overall DK rate, and the lower DK rate in each comparison of subgroups is highlighted. It can be seen that informants for cognitively impaired probands tend to have higher DK rates as do female informants and younger informants: these differences are fairly consistent but not large. The two factors of time known and co-residence show much larger differences, with informants who have known the proband for 25 years or less and informants who are not co-resident with the proband having consistently and substantially higher DK rates. Among the informants who did not live with the proband there is a less consistent pattern, with those with face-to-face contact at least weekly having a tendency for slightly lower DK rates than those with less regular contact (not tabulated).
Table 4 shows the effect on DK rates of interviewing the informant face-to-face, either with or without the proband present, or by telephone. For the majority of questions the DK rate where the proband was present is zero, with somewhat higher rates for other face-to-face interviews and slightly higher still for interviews conducted by telephone.
|
|
|
|
The 11 questions on lifetime illness generally show good agreement as measured by kappa, with diabetes having almost perfect agreement, six questions having substantial agreement and four having moderate agreement as defined by Landis and Koch.14 One of the highest kappa values is for thyroid disease (0.78): informants and probands were also asked whether the problem was under- or over-activity, and agreement at this level of detail is also substantial (0.68). The four lifetime illness questions with relatively low kappa values are arthritis, bronchitis, meningitis/encephalitis and pernicious anaemia: in the latter two the low kappa probably reflects very low population prevalences rather than low agreement. There is generally lower agreement on the other eight questions: none show substantial agreement and only two (hearing impairment and history of boxing) show moderate agreement. On boxing, both probands and informants were asked whether this was 18 or >18 years: at this level agreement is somewhat lower (2 x 2 kappa 0.47, 3 x 3 kappa 0.40). For a typical kappa of 0.64 with a prevalence of around 30%, the 95% CI is 0.600.68.
It might have been expected that there would be some relationship between the level of DK response to a question and the level of agreement of those giving a definite answer to it, but the results do not bear this out (Spearman's rank correlation, coefficient 0.2).
The columns tabulating P pos and P neg show generally very high values for negative agreement and lower values for positive agreement, although there is a clear effect of population prevalence: meningitis/encephalitis and pernicious anaemia with very low prevalences have unusually low positive agreement and the question on whether the proband has ever drunk alcohol, with prevalence over 70%, shows positive agreement much higher than negative. For head injury, positive agreement is only 0.33 and for family psychiatric history 0.23.
Unlike the finding for DK rates, there is no clear pattern of agreement between the five relationship categories. For many of the questions kappa values are very similar in all categories, with spouses and first-degree relatives not showing higher values than friends and others. Asthma, angina, high blood pressure and heart attack have slightly higher agreement in spouses with only small differences between the other four categories. Siblings have low agreement on bronchitis and on sleep problems and relatively high agreement on head injury, which is the only question where friends and others have substantially lower agreement than spouses and first-degree relatives.
Various characteristics of the probands and informants have been examined to see whether they have any bearing on agreement as measured by kappa. Of proband characteristics, sex has no consistent effect on agreement (results not tabulated). There is no consistent overall pattern for proband age or cognitive status, although taking only the nine lifetime illness questions there is a tendency for younger probands and also for cognitively well probands to have higher agreement. These results are shown in Table 8 together with the effect of informant characteristics on agreement. Neither the sex of the informant nor whether they have known the proband for more or less than 25 years has a consistent bearing on agreement (not tabulated). Older informants tend to have higher agreement than younger ones, particularly on the lifetime illness questions. Informants who live with the proband have higher agreement than those who do not on every question except bronchitis: of the informants who are not co-resident there is no difference between those who see the proband at least weekly and those who see them less frequently, and these two groups have been aggregated in the Table.
|
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The second issue, that of accuracy of proxy information, is more difficult to evaluate. One of the main problems is finding a suitable measure of agreement between proband and informant. Previous studies have generally treated both sources of information as prone to measurement error and computed an index of reliability between the two. For categorical data, this has usually been one or more of: percentage agreement, positive and negative agreement, and kappa. The problems associated with these measures have been discussed at length by otherse.g.15,16 but no single measure has yet been suggested which has none of the drawbacks of the above. In the present study, percentage agreement has not been presented because of its failure to account for agreement by chance, and the main measure used for comparison has been kappa. It is acknowledged that this is far from perfect, particularly where the population prevalence is very low. The alternative to using an index of reliability is to take the proband's response as the standard against which to measure the informant's response, and compute an index of bias such as that proposed by Magaziner et al.3 Without an independent source of information such as medical records it is not possible to say whether this approach is justified. The current finding that younger and cognitively well probands tend to have higher agreement with their informant, together with the extent of over-reporting by informants, particularly those for the cognitively impaired, may suggest that the proband's information should not be taken as infallible.
Previous studies have found agreement to vary considerably with the topic under investigation. Past and current medical conditions have been reported to show quite good agreement, although proxies have tended to under-report relative to the proband, particularly for history of head trauma.3,59 We have found more instances of over-reporting by the informant, most notably in family history of psychiatric problems. Well-defined chronic illnesses have been reported to show higher agreement than those which are short-term and not life-threatening.3,9 The present findings, with the highest agreement for diabetes and thyroid disease, tend to a similar conclusion and also concur with those studiese.g.7 which have found poor agreement for questions involving family history. The previous finding of substantial informant under-reporting of head trauma7 is confirmed, with its important implications for case-control studies of risk factors for dementia.
Perhaps a surprising finding of the present study is the lack of large differences in agreement between the different proband/ informant relationship groups. For most of the conditions studied, there appears to be no reason to consider the information given by spouses or first-degree relatives as more accurate than that given by the more distant informants. Head injury, with low agreement overall, is the only condition on which friends and others show markedly lower agreement than closer relatives. The only characteristic of informants which appears to be consistently associated with higher agreement is co-residence with the proband. Previous findings have differed as to whether the sex of the informant affects agreement, possibly because different types of information about the proband were being collected.2,6 The present study has found no influence of informant sex on agreement for the health-orientated questions under examination here. Cognitive impairment of a degree which does not prevent the proband answering the question appears to have a limited effect on agreement.
For most of the health questions investigated here, face-to-face informant interviews at which the proband was present showed the lowest DK rates, the lowest probable rates and the highest agreement, suggesting that some degree of correction or reminder by the proband may have been taking place. This effect is sometimes used deliberately in the so-called assisted interview. Arranging to interview proband and respondent separately can be very difficult when the two are co-resident and the interview is being conducted in their home. However, this finding shows the importance of achieving this if the researcher wishes to avoid contamination of one source by the other.
The present paper has not included any multivariate analyses of DK response or agreement: the relative contributions of such factors as relationship, frequency of contact and time known, together with any interactions between them, have therefore not been estimated, but are the subject of the companion paper presented in this edition. In a practical setting the investigator is faced with choosing the informant most likely to provide complete and accurate information and this will usually be done on the basis of relationship, for which we have presented detailed analyses. The best proxy may of course differ according to the nature of the question being asked, with, for example, siblings giving the most complete information on the proband's early life and spouses being more informative on current health. Questions on proxy information still to be answered include the extent to which proband and proxy responses accord with more objective sources of information and whether probands for whom no proxy is available are different in some way. For some studies it would also be useful to know whether the value of proxy information declines after the death of the proband.
To summarize: completeness of proxy information has been found to be strongly influenced by the nature of the question being asked and by the relationship of the informant to the proband, with length of time known and co-residence also showing an effect. Agreement as measured by kappa has also been found to depend heavily on the question being asked and to a much lesser extent on whether the informant lives with the proband, but little on the informant-proband relationship. The results from this study would suggest that if a close relative cannot be found as a proxy, the choice of a more distant informant is likely to produce significantly higher DK rates on some questions but will not lead to radically lower agreement for those informants prepared to give a definite answer.
![]() |
Appendix |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
![]() |
Acknowledgments |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Magaziner J, Simonsick EM, Kashner TM, Hebel JR. Patient-proxy response comparability on measures of patient health and functional status. J Clin Epidemiol 1988;41:106574.[ISI][Medline]
3 Magaziner J, Hebel JR, Warren JW. The use of proxy responses for aged patients in long-term care settings. Compr Gerontol B 1987;1:11821.[Medline]
4 Rubenstein LZ, Schairer C, Wieland GD, Kane R. Systematic biases in functional status assessment of elderly adults: effects of different data sources. J Gerontol 1984;39:68691.[ISI][Medline]
5 Halabi S, Zurayk H, Awaida R, Saab B. Reliability and validity of self and proxy reporting of morbidity data: a case study from Beirut, Lebanon. Int J Epidemiol 1992;21:60712.[Abstract]
6 Herrmann N. Retrospective information from questionnaires I. Comparability of primary respondents and their next-of-kin. Am J Epidemiol 1985;121:93747.[Abstract]
7 Korten AE, Jorm AF, Henderson AS, McCusker E, Creasey H. Control-informant agreement on exposure history in case-control studies of Alzheimer's disease. Int J Epidemiol 1992;21:112131.[Abstract]
8 Rocca WA, Fratiglioni L, Bracco L, Pedone D, Groppi C, Schoenberg BS. The use of surrogate respondents to obtain questionnaire data in case-control studies of neurologic diseases. J Chron Dis 1986;39: 90712.[ISI][Medline]
9 Heymann A, Wilkinson WE, Stafford JA, Helms MJ, Sigmon AH, Weinberg T. Alzheimer's disease: a study of epidemiological aspects. Ann Neurol 1984;15:33541.[ISI][Medline]
10 The Medical Research Council Cognitive Function and Ageing Study (MRC CFAS). Cognitive function and dementia in six areas of England and Wales: the distribution of MMSE and prevalence of GMS organicity level in the MRC CFA Study. Psychol Med 1998;28:31935.[ISI][Medline]
11 Folstein MF, Folstein SE, McHugh PR. Mini Mental State: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:18998.[ISI][Medline]
12 Copeland JRM, Dewey ME, Griffiths Jones HM. A computerised psychiatric diagnosis system and case nomenclature for elderly subjects: GMS and AGECAT. Psychol Med 1986;16:8999.[ISI][Medline]
13 Van Duijn CM, Stijnen T, Hofman A. Risk factors for Alzheimer's disease: overview of the EURODEM collaborative re-analysis of case-control studies. Int J Epidemiol 1993;22:83847.[Abstract]
14 Landis JR, Koch GC. The measurement of observer agreement for categorical data. Biometrics 1977;33:15974.[ISI][Medline]
15 Fleiss JL. Measuring agreement between judges on the presence or absence of a trait. Biometrics 1975;31:65159.[ISI][Medline]
16 Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990;43:55158.[ISI][Medline]