a Writing committee: Michael E Dewey, Christine J Parker and the Analysis Group of the MRC CFA Study.
b M Dewey, Trent Institute for Health Services Research, Medical School, University Hospital, Nottingham NG7 2UH, UK. E-mail: Michael.Dewey{at}nottingham.ac.uk
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Material In three urban and two rural areas of England and Wales a sub-sample, weighted by age and cognitive status, was drawn from random population samples of people aged 65, not permanently living in hospital, and interviewed between 1991 and 1994, plus for each of these a relative, friend or carer.
Methods The relation between the reports of probands and informants on cognitive function, lifetime illnesses, current health problems, and family history of illness was analysed using multiple regression of agreement, measured using Cohen's kappa, and logistic regression of bias and of proportion of not known responses. Potential explanatory variables used were: centre, proband cognitive function, proband sex, informant sex, proband age, difference in age (generation), relationship, frequency of contact, period known, modality of informant interview.
Results Overall agreement per interview measured by kappa between informant and proband was lower for older probands, probands who were cognitively impaired, and those who saw one another less often; each effect adjusted for all the others using multiple regression. Informants tended to over-report relative to probands if the proband was cognitively impaired, male, they themselves were female, of a younger generation, and co-resident; each effect adjusted for all the others using logistic regression. Interviews were more likely to be missing at least one item if the informant was not the spouse, had known the proband for less time, and was not co-resident; each effect adjusted for all the others using logistic regression.
Conclusion There is reasonable accord between probands and informants which is enhanced by close contact, long period of knowledge, and a female informant. Degree of kinship is not critical. Informant interviews may be conducted by telephone rather than face-to-face without serious loss of quality.
Keywords Elderly, health problems, multivariate analysis, self-report and proxy information
Accepted 21 December 1999
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
To avoid confusion in the remainder of the paper we use the term proband to refer to the respondent. We prefer this usage as both proband and informant respond.
In a large epidemiological study we have collected self-reported information on a variety of health problems from probands and informants. A review of the literature, descriptive results on the agreement between probands and informants, and on the rate of informant inability to provide data have already been published.10 In this paper we present multivariate analyses of agreement, bias, and inability to respond.
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
A stratified random sub-sample was drawn within each centre for further interviewing. It is this sub-sample, of approximately 500 in each centre, which is used in this paper. The original sample was stratified by age, and the sub-sample was further stratified by age and cognitive status. The interview covered demographic information, cognitive function, activities of daily living, family history of illness, and lifetime health and health-related events. In total 2619 people were interviewed. In addition they were also asked to provide the name of an informant. Some people were unwilling or unable to provide an informant, and sometimes the nominated informant refused. In all this happened for 418 (16%) of the probands.
Of 2201 probands with an informant interview we exclude 145 because the interviewer rated the information, either from the proband or informant, as unreliable. For a further 158 none of the health items was answered by the proband, predominantly because the proband's cognitive state did not permit this. The remaining 1898 have already been discussed and univariate analyses have been presented:10 they form the basis for the multivariate analyses presented here. This paper is based on release 3 of the dataset.
The areas covered by the 18 items which we consider here are: past or present illness, 12 items (angina, arthritis, asthma, bronchitis, diabetes, head injury, heart attack, high blood pressure, meningitis or encephalitis, pernicious anaemia, stroke, and thyroid disorder); present problems, three items (hearing difficulty, memory difficulty, and sleep difficulty); and other health matters, three items (ever used alcohol, family history of dementia, and family history of psychiatric illness). Details of the actual questions used and any necessary recoding have already been presented.10 Note that 19 questions are considered there, but we have excluded history of boxing from the present analyses as it was only asked for male probands.
Analytical methods
In the multivariate models we distinguish between explanatory variables which we regard as part of the minimal model and other explanatory variables. The minimal model includes all the variables which we feel have to be included irrespective of their predictive power. In particular the minimal model includes the variables used to stratify at earlier stages of the design. By contrast the other variables are only included if there is interest in their inclusion, or they add predictive power. The minimal model includes: centreCambridge (reference category), Gwynedd, Newcastle, Nottingham, and Oxford; proband cognitive functionunimpaired (reference category) and impaired (Mini-Mental State Examination12 score 21 or AGECAT13,14 organic syndrome level
3); proband sexmale (reference category) and female; informant sexmale (reference category) and female; proband age
69 (reference category) 7074, 7579, 8084,
85.
In addition we include as appropriate some of: difference in age (generation)informant >15 years younger (reference category), <15 years younger and older; relationshipspouse (reference category), sibling, child, friend, other; frequency of contactlives with (reference category), daily, more than weekly, weekly, less often than weekly; period knownalmost all proband's life (reference category), >25 years, 1025 years, 59 years, 4 years; modality of informant interview face-to-face (reference category), and telephone. For some of the Tables we have collapsed over the levels of some of the explanatory variables to simplify the presentation. We have only done this where the collapsed levels were adjacent, and gave rise to similar effects.
As is obvious from their meanings these variables have a complex pattern of inter-relationship. Most obviously spouses are of different sexes, and usually co-resident and of the same generation, children are of the younger generation and cannot have known the proband for all the proband's life.
We present further details of the analysis, results, and some discussion grouped together for each of three measures: agreement, bias, and not known responses. This is followed by an overall discussion.
![]() |
The Analyses |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We take each of the 1898 pairs of proband and informant who gave adequate data on the 18 responses. For each pair we calculate an overall based on those 18 judgements. Then we model
using linear regression with
as outcome, and the chosen explanatory variables. We considered transforming
but decided against it. Although
can be interpreted as a proportion, and so could be subjected to any of the usual transformations for proportions, it can take on negative values which rules out this possibility. Under some circumstances it can be seen as an intra-class correlation coefficient and the Fisher hyperbolic arctangent transformation could be used, but we have not done this.
There is some missing information here, and we have chosen, arbitrarily, to exclude from this analysis any pair with fewer than four variables with available data from both proband and informant. In addition there are a few missing values for some explanatory variables. This leaves a final total of 1879 pairs for the analysis.
Results
Figure 1 shows the results of the multiple regression of
on the explanatory variables.
|
The remaining three are included either because they account for substantial amounts of variability, or because they are intrinsically interesting. We have included interview type because we regard it as of practical importance, given cost pressures in research, to know whether face-to-face interviews are necessary.
Perhaps surprisingly the variable for relationship did not seem to be an important predictor given the other variables already in the model. We have therefore excluded it from the final model. Similarly the variable reflecting how long the proband and informant had known one another also failed to predict.
An examination of the residuals and theoretical knowledge of the distribution of gave some concern, As mentioned above there is an absence of any attractive transformation so we checked the model by also fitting a logistic regression to percentage agreement. This gave a model similar to that shown here for
, and in the interests of space we omit it and only present the model for
.
Discussion
What we are considering here is agreement. It is not necessarily the case that the proband is correct, and so close agreement is not necessarily a good thing.
A pair of proband and informant in the reference category for all factors has the quite respectable estimated of 0.66, and if either the proband or informant were female this is a couple of percentage points higher. Nonetheless we can see that the oldest old probands and people not co-resident with the informant have lower agreement by a few points. The slightly anomalous result for informants who only see the proband monthly or less frequently has no easy explanation.
Centre differences will all be discussed together in the final Discussion.
As expected cognitively impaired probands have lower agreement. Of course severely impaired probands would not have answered these health questions, and so have already been excluded.
When the proband and informant are of a different generation (that is when the informant is younger) this does not account for much variability.
The pairs where informants were interviewed by telephone lead to less agreement, but not by much.
Bias
Purpose of analysis and method
The analysis of agreement gives us some information about the factors which affect interview performance, but there are other aspects we can examine. One of these is bias. If the proband and informant disagree we can analyse whether this is more likely to be the proband claiming a health problem which the informant claims is absent, or whether the informant claims it is present while the proband claims it is absent. We define bias as
![]() |
This analysis can only take place for those pairs where there is disagreement, and so involves only 1686 of the pairs.
Results
Figure 2 shows the results of the logistic regression of bias on the explanatory variables. In interpreting this note that bias is scored in the direction that high values indicate a propensity for informants to claim something is present, when probands claim it is absent.
|
Discussion
In the absence of a gold standard we cannot tell who is correct, and when we summarize this as over-reporting we do not imply that probands are correct. Overall there is mild over-reporting, if there was no bias then the estimated P would be 0.5 instead of the actual 0.53. However, as can be seen several factor levels lead to less predicted over-reporting: female probands, same generation, and not co-resident. Some others lead to predicted further over-reporting: proband cognitively impaired, female informants, and older probands.
Centre effects are again postponed to the final Discussion.
Once again, contrary to our expectations, relationship was not an influence, although it is of course confounded with how often the pair saw one another which did enter the model. There is no real effect of interview modality.
Not known
Purpose of analysis and method
An important feature of interview quality is the rate of item non-response. What factors lead to informant interviews being incomplete?
We initially intended to analyse number of not known responses per interview, but they are infrequent, and so it has been collapsed to a binary variable: interviews with all 18 known versus those with any not known, and a logistic regression used as analytical method.
Results
Figure 3 shows the results of the logistic regression of not known on the explanatory variables.
|
As well as the first five from the minimal model this includes three other explanatory variables.
Discussion
The major effects here are clearly for relationship and period known. As might be expected friends are much more likely to say they do not know. Siblings and other (a miscellaneous category composed of distant relatives and professional carers) were slightly more likely to say they did not know. Similarly, informants with relatively short knowledge of the proband are much more likely to say they do not know. Over and above these effects informants who were not co-resident gave more not known responses.
By comparison with these most other effects are small. Informants on cognitively impaired probands are more likely to not know. Older probands slightly less often gave rise to not known responses. Neither sex of proband nor of informant was important.
![]() |
Overall Discussion and Conclusions |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The specific results for the centres are only of interest to the present study, but the pattern of the different outcomes is of interest because it shows that high levels of agreement are not bought at the expense of high levels of incompleteness. Gwynedd had a higher rate of not known, but lower agreement rather than higher. Newcastle and Cambridge were similar, whereas Nottingham had relatively lower agreement coupled with under-reporting, Oxford had average agreement but some over-reporting.
We had initially expected relationship to be an important predictor, but for two of the three outcomes this is not true as other variables are more important. So relationship, although having no effect on agreement or on bias, does heavily influence completeness. It is clear that the effect of relationship, if any, is better accounted for in terms of variables like length of relationship, and frequency of contact.
Seeing the proband less frequently has an effect throughout leading to lower agreement, under-reporting, and less completeness. How long the relationship has lasted by contrast only affects completeness.
Although telephone interviews do differ from face-to-face in the expected direction of poorer agreement the effect is not large, and with this sample size of 1800 we think that the use of telephone interviews as a cost-saving substitute for face-to-face remains a viable option.
Female probands and informants give better agreement, but female probands lead to under-reporting, female informants to over-reporting.
Older probands lead to poorer agreement, more over-recording, and less incompleteness. We have no explanation of why these effects (which are consistent over the age groups, although not very strong) persist after multivariate adjustment.
Mildly cognitively impaired probands lead to less agreement, over-reporting and incompleteness. The loss of agreement is perhaps predictable, and the over-reporting might optimistically be attributed to the proband under-reporting because of memory problems. This does not account for the incompleteness though, and this remains worrying as it might introduce bias into comparative studies where informant histories are taken for both dementia cases and well controls.
To summarize, our results conclude that in general there is reasonable accord between probands and informants. When there is a choice of informants our results point to straightforward conclusions: choose someone who is in close contact, and has known the proband for a long time, exact degree of relationship is not so important, but female informants may be preferred.
One further practical conclusion is that informant interviews may be conducted by telephone rather than face-to-face without serious loss of quality, and in view of the cost savings may be preferred.
![]() |
Acknowledgments |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Magaziner J, Simonsick EM, Kashner TM, Hebel JR. Patient-proxy response comparability on measures of patient health and functional status. J Clin Epidemiol 1988;41:106574.[ISI][Medline]
3 Magaziner J, Hebel JR, Warren JW. The use of proxy responses for aged patients in long-term care settings. Compr Gerontol B 1987; 1:118121.[Medline]
4 Rubenstein LZ, Schairer C, Wieland GD, Kane R. Systematic biases in functional status assessment of elderly adults: effects of different data sources. J Gerontol 1984;39:68691.[ISI][Medline]
5 Halabi S, Zurayk H, Awaida R, Saab B. Reliability and validity of self and proxy reporting of morbidity data: a case study from Beirut, Lebanon. Int J Epidemiol 1992;21:60712.[Abstract]
6 Herrmann N. Retrospective information from questionnaires I. Comparability of primary respondents and their next-of-kin. Am J Epidemiol 1985;121:93747.[Abstract]
7 Korten AE, Jorm AF, Henderson AS, McCusker E, Creasey H. Control-informant agreement on exposure history in case-control studies of Alzheimer's disease. Int J Epidemiol 1992;21:112131.[Abstract]
8 Rocca WA, Fratiglioni L, Bracco L, Pedone D, Groppi C, Scoenberg BS. The use of surrogate respondents to obtain questionnaire data in case-control studies of neurologic diseases. J Chron Dis 1986;39:90712.[ISI][Medline]
9 Heymann A, Wilkinson WE, Stafford JA, Helms MJ, Sigmon AH, Weinberg T. Alzheimer's disease: a study of epidemiological aspects. Ann Neurol 1984;15:33541.[ISI][Medline]
10
MRC CFA Study. Survey research into health problems of elderly people: a comparison of self-report with proxy information. Int J Epidemiol 2000;29:684697.
11 The Medical Research Council Cognitive Function and Ageing Study (MRC CFAS). Cognitive function and dementia in six areas of England and Wales: the distribution of MMSE and prevalence of GMS organicity level in the MRC CFA Study. Psychol Med 1998;28:31935.[ISI][Medline]
12 Folstein MF, Folstein SE, McHugh PR. Mini-mental state: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:18998.[ISI][Medline]
13 Copeland JRM, Dewey ME, Griffiths-Jones HM. A computerized psychiatric diagnostic system and case nomenclature for elderly subjects: GMS and AGECAT. Psychol Med 1986;16:8999.[ISI][Medline]
14 Dewey ME, Copeland JRM. Computerized psychiatric-diagnosis in the elderlyAGECAT. J Microcomp Applic 1986;9:13540.