From the Netherlands Institute of Mental Health and Addiction, Utrecht, the Netherlands.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
data collection; longitudinal studies; patient dropouts; psychopathology
Abbreviations: CI, confidence interval; CIDI, Composite International Diagnostic Interview; DSM-III-R, Diagnostic and Statistical Manual of Mental Disorders: DSM-III-R, third edition, revised; ECA, Epidemiologic Catchment Area; NEMESIS, Netherlands Mental Health Survey and Incidence Study; OR, odds ratio.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Factors associated with attrition in longitudinal surveys have been investigated in a number of studies (3, 7
), but little data are available on the loss of respondents in general population studies on mental health. Only a few such large-scale longitudinal studies exist, the best known of which are the Epidemiologic Catchment Area (ECA) Program in the United States (8
), the Stirling County study in Canada (9
), and the Lundby Study in Sweden (10
). Only the ECA study extensively reported on the relation between psychopathology and attrition (8
). Since mental health research is now on the increase, there is a growing need to identify its characteristic sources of sample attrition.
To understand as clearly as possible the differences between respondents who continue to participate and those lost to follow-up, we need to distinguish between different types of attrition. The most prominent groups of respondents lost to follow-up are those who refuse further participation, those who cannot be located at follow-up, and those who become incapacitated by illness or have died. Psychopathology may be associated with all three types of attrition. In studies of clinical samples, psychopathology is associated with higher rates of mortality; attitudes toward the survey process and social interaction in general may be associated with some forms of psychopathology, which may affect the rate of refusal to participate; and psychopathology may be associated with residential mobility (8).
In the United States, the Epidemiologic Follow-Up Study of the National Health and Nutrition Examination Survey I (8-year follow-up among 2,981 subjects) has found distinctly different predictors for the three types of attrition (3). Psychiatric diagnoses were not a focus of the study, but a Center for Epidemiologic Studies Depression Scale was administered. Multivariate analysis revealed that participants with a high score, as well as those who were young, single, and smoked tobacco, were less likely to be located for follow-up. Educational level was the only factor associated with refusal; those at the lowest level were more likely to refuse participation. Males, elderly people, single people, the unemployed, people with high blood pressure, and smokers were more likely to die during the follow-up interval. Thus, depression was linked to failure to locate but not to refusal or mortality.
The Journal has published findings from the ECA, a 1-year follow-up study among 10,167 subjects, on the effects of psychopathology on attrition (8). Respondents not located at follow-up were more likely to be male, young, moderately educated, and unmarried than those who were reinterviewed successfully. Refusal to participate was associated with higher age and lower educational level. Attrition due to morbidity or mortality was not assessed. After adjustment for demographics, psychopathology was found to have a weak-to-moderate effect on attrition, with a stronger effect on failure to locate than on refusal. Failure to locate was associated with the presence, measured at baseline, of 12-month panic disorder, major depression, drug abuse/dependence, alcohol abuse/dependence, and antisocial personality disorder. Major depression was the only disorder associated with refusal but in the reverse direction than the one expected. One of the five ECA data collection sites, Baltimore, Maryland, also investigated attrition in a 15-year follow-up (11). Failure to locate and mortality were strongly influenced by psychopathology, but refusal was not.
Similar to the paper by Eaton et al. (8), the present article describes the sociodemographic and psychiatric determinants of attrition in the second wave (1-year follow-up) of the Netherlands Mental Health Survey and Incidence Study (NEMESIS), a longitudinal survey investigating psychopathology in the general population. In addition to failure to locate and refusal to participate, we investigated morbidity and mortality as a source of attrition. Item nonresponse was not dealt with, since it was negligible as a result of the computer-assisted interviewing.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
To optimize the response rate and to compensate for possible seasonal influences, we spread the initial data collection phase over the entire period from February through December 1996. No adjustments had to be made to the procedure in the course of the fieldwork; hence, no additional respondents were drawn from specific groups.
A total of 7,076 persons were interviewed in the first wave. Depending on the method of calculation, the response rate was 64.2 percent (of the households eligible for interviewing) or 69.7 percent (of the adults eligible for interviewing) (12). No proxy information was collected. Persons who declined to take part in the full interview (despite our offer of a 50-guilder (20-dollar) gift certificate to these subjects only) were asked to furnish several key data (age, gender) and to complete the General Health Questionnaire-12, a screener for current mental health problems (13
). Some 43.6 percent of the refusers agreed to do so. These nonrespondents were found to have a slightly lower average questionnaire score (that is, to be in better mental health) than the respondents (1.16 vs. 1.22); they also had a lower average age, and there was a higher percentage of women. Compared with the Dutch population (according to the Netherlands Central Bureau of Statistics), survey participants are fairly representative of the population in terms of gender, civil status, and degree of urbanization of place of residence (12
). Only the group aged 1824 years was significantly underrepresented.
Second wave
All persons who had taken part in the first interview were approached for the follow-up. As in the first wave, the interviewers made a minimum of 10 telephone calls or visits at various times of the day and week to establish contact. A tracing process involving mail, telephone calls, field tracing, and municipality records was used to locate the original sample. At the end of the first interview, a change of address card was left behind that could be mailed to us in case a respondent moved. Interviewers recorded the reasons that respondents failed to continue participation.
The fieldwork in the second wave took place from February 1997 through January 1998. The mean interval between the first and second interviews was 379 days (standard deviation, 35), slightly longer than the intended 365 days.
Variables
The dependent variables were the three types of attrition: failure to locate respondents, refusal to participate, and morbidity or mortality. We used two types of prognostic variables, measured at baseline, that might potentially predict attrition: sociodemographic characteristics and the presence of psychopathology at baseline or at any time during the 12 months before the first interview. The demographic variables were gender, age, educational level, degree of urbanization (rural, municipalities with fewer than 500 addresses per square kilometer; urban, municipalities with 500 or more addresses per square kilometer), cohabitation status (irrespective of children), employment status, country of birth, and presence of one or more conditions from a list of 31 somatic disorders treated or monitored by a physician in the 12 months prior to baseline (for example, asthma, lung emphysema, arthritis, rheumatism, heart disease, heart attack, stomach or intestinal ulcer, diabetes).
The diagnoses of psychiatric disorders were based on the Diagnostic and Statistical Manual of Mental Disorders: DSM-III-R, third edition, revised (DSM-III-R) (14). The instrument used to determine the diagnoses was the computerized version of the Composite International Diagnostic Interview (CIDI), version 1.1 (15
). The CIDI is a structured interview developed by the World Health Organization (16
, 17
) on the basis of the Diagnostic Interview Schedule and the Present State Examination. It was designed for use by trained interviewers who are not clinicians. CIDI version 1.1 has two diagnostic programs to compute diagnoses according to the criteria and definitions of either DSM-III-R or the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision. The CIDI is now being used worldwide, and World Health Organization research has found high interrater reliability (18
, 19
) and high test-retest reliability (20
22
). The following DSM-III-R diagnoses were recorded in the NEMESIS data set: mood disorders (depression, dysthymia, bipolar disorder), anxiety disorders (panic disorder, agoraphobia, social phobia, simple phobia, obsessive-compulsive disorder, generalized anxiety disorder), psychoactive substance use disorders (alcohol or drug abuse and dependence, including sedatives, hypnotics, and anxiolytics), eating disorders, schizophrenia, and other nonaffective psychotic disorders.
Statistical analysis
We first carried out bivariate and multivariate logistic regression analyses to obtain the odds ratios and their 95 percent confidence intervals that reflected the associations between the demographic characteristics and the three types of attrition. Respondents who were reinterviewed successfully served as the comparison category.
We then used logistic regression to investigate to what extent psychopathology in the 12 months prior to the first interview predicted the three types of attrition. Because the prevalence of psychopathology varies strongly by demographic characteristics (23, 24
), these analyses were adjusted for demographics. The odds ratios were calculated in a series of models into which the presence of each disorder or each category of disorders was introduced, along with the demographic variables. In addition, these analyses also were carried out by using lifetime disorders measured at baseline. Since we focused on associations, we made no use of sample weights that generalize to the general population, except when reporting prevalences of disorders.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Sociodemographic characteristics and attrition
We first focused on the relation between follow-up status and certain sociodemographic characteristics. Table 2 presents the demographic characteristics of the respondents who stayed in the sample and of those in the three groups lost to follow-up. We used this distribution to calculate the crude odds ratios for attrition, comparing each nonresponse group with the response group (table 3). Table 3 also shows the multivariate odds ratios for the three types of attrition relative to the demographic variables.
|
|
There was a trend toward an increased risk for attrition due to refusal with increasing age (although the odds for those in the oldest age category were similar to the odds for those in the youngest age category) and lower educational level, but the first trend was significant only in the bivariate analysis. In the bivariate analysis, attrition due to refusal was also significantly related to employment status; people not in paid employment were more likely to refuse to cooperate. In the multivariate analysis, respondents not born in the Netherlands had a 28 percent greater odds of attrition due to refusal than respondents born in the Netherlands, and subjects living in urban areas had a 20 percent greater odds than subjects living in rural areas; however, these associations were not statistically significant at the conventional alpha = 0.05 level. Gender, cohabitation status, and presence of somatic disorders were not related to refusal.
Age, education, employment status, and presence of somatic disorders predicted attrition due to morbidity or mortality in both the bivariate and multivariate analyses. Older subjects and subjects with a lower educational level had an increased risk for attrition due to illness or death. Subjects not in paid employment had higher odds than subjects in paid employment. Subjects with one or more somatic disorders showed a greater likelihood of attrition due to illness or death than those without. In the multivariate analysis, female respondents had a 37 percent lower odds of attrition due to morbidity or mortality than males, but the 95 percent confidence interval overlapped the null hypothesis. Urbanization, cohabitation status, and country of birth were not associated significantly with this type of attrition.
Psychopathology and attrition
Because prevalence rates of psychiatric disorders are known to vary by demographics, we adjusted for gender, age, education, degree of urbanization, cohabitation status, employment status, country of birth, and presence of somatic disorders while investigating the relation between mental disorders and attrition. The odds ratios were calculated in a series of models; each model contained the presence of a separate disorder or a category of disorders and the demographic variables.
For nearly every disorder diagnosed in the 12 months preceding the first interview, we found that the odds ratio of attrition due to failure to locate was greater than 1.0 when compared with absence of the disorder (table 4). Exceptions were panic disorder and, surprisingly, drug abuse. The associations of failure to locate with agoraphobia and with alcohol abuse, and the categories of mood disorders, substance use disorders, and eating disorders, were found to be statistically significant. In addition, the presence of one or more DSM-III-R diagnoses was associated with failure to locate.
|
For all disorders, we found that morbidity/mortality was evident more often among subjects diagnosed with a disorder than among subjects not diagnosed with that disorder. Morbidity/mortality was significantly associated with the presence of dysthymia, agoraphobia, simple phobia, and obsessive-compulsive disorder. Some of the confidence intervals for this type of attrition were wide because of the small number of respondents lost to illness or death (n = 72) and the low prevalences of psychiatric disorders. The general category of anxiety disorders and the presence of one or more CIDI/DSM-III-R disorders also predicted attrition due to morbidity or mortality.
These analyses also were carried out by using the presence of lifetime disorders as predictors of attrition (not shown in table). Failure to locate was predicted significantly only by agoraphobia (odds ratio (OR) = 1.94, 95 percent confidence interval (CI): 1.06, 3.54), the category of eating disorders (OR = 4.63, 95 percent CI: 2.00, 10.72), and the presence of one or more disorders (OR = 1.40, 95 percent CI: 1.05, 1.87). Simple phobia still predicted a significantly smaller likelihood of refusal (OR = 0.70, 95 percent CI: 0.54, 0.90), but refusal also was seen less often in subjects with lifetime major depression (OR = 0.77, 95 percent CI: 0.62, 0.95), dysthymia (OR = 0.72, 95 percent CI: 0.53, 0.97), drug abuse (OR = 0.42, 95 percent CI: 0.18, 0.98), and the main category of mood disorders (OR = 0.76, 95 percent CI: 0.62, 0.92). For morbidity/mortality, the only significant predictors were simple phobia (OR = 2.73, 95 percent CI: 1.55, 4.82) and the category of anxiety disorders (OR = 1.97, 95 percent CI: 1.18, 3.28).
Number of psychiatric disorders and attrition
We next investigated whether attrition was related to the number of psychiatric disorders a participant had had in the 12 months before the first interview. Again, we adjusted for demographic variables. Five comorbidity categories were used: none, one, two, three, and four or more disorders. Having no disorder was the reference category (table 5). The odds for respondents with one or with four or more disorders were high (OR = 2.09 for both categories), while respondents with two or with three disorders had relatively low odds (OR = 1.27 and OR = 1.29, respectively). No consistent pattern for refusal was seen in relation to number of disorders. Attrition due to morbidity or mortality was more likely for those with two or more disorders, and a trend was suggested toward an increased odds of this type of attrition with a higher number of disorders. When we used lifetime disorders in the analyses, similar results were found (not shown in table), with the exception of morbidity/mortality, which was more likely for only those with four or more disorders (OR = 2.80, 95 percent CI: 1.28, 6.12).
|
When lifetime disorders were used in these analyses, social phobia (OR = 1.28, 95 percent CI: 1.04, 1.57) as well as drug abuse (OR = 0.51, 95 percent CI: 0.28, 0.93) were associated with attrition. None of the categories of disorders, nor the presence of at least one disorder, was associated with attrition.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In agreement with Eaton et al. (8), we found that psychopathology, adjusted for demographic factors, has no more than a weak-to-moderate effect on attrition and that psychopathology is more strongly related to failure to locate respondents than to refusal. Only one disorder in the 12 months before baseline was found to be related significantly to refusal in either studysimple phobia in NEMESIS and depression in the ECA (8
)but, in both cases, the disorder was linked to a reduced likelihood of refusal. In neither study did the number of disorders influence refusal at follow-up. In addition, Badawi et al. (11
) and Farmer et al. (3
) found that psychopathology and that depressive symptomatology, respectively, were linked to failure to locate but not to refusal. In an overview of many studies on a range of issues, Ribisl et al. (1
) similarly concluded that participants who are not located at follow-up introduce more systematic bias in the dependent variable measured at the second wave compared with participants who refuse to be reinterviewed. We showed that subjects with a lifetime history of major depression, dysthymia, simple phobia, or drug abuse were even more willing to stay in this study on psychopathology than subjects without such a history.
A few differences were found between the ECA (8) and NEMESIS in relation to the specific 12-month disorders associated with not-located respondents. The former study found that panic disorder and major depression were linked to this source of attrition. The latter found that agoraphobia and the category of mood disorders (although not major depression by itself) were implicated. These differences may have to do with different samples and different sample sizes of the two surveys. Eaton et al. (8
) also found an association between failure to locate and drug abuse/dependence, alcohol abuse/dependence, and antisocial personality disorder. Our study examined (alcohol and drug) abuse and dependence separately, and only alcohol abuse was found to be related. Antisocial personality was not included in our study.
In NEMESIS, we were also able to study attrition due to morbidity/mortality, albeit among limited numbers of participants. It was significantly linked to dysthymia, agoraphobia, simple phobia, and obsessive-compulsive disorder and to the category of anxiety disorders.
The results of our study should be interpreted within the context of its limitations. First, in analyses such as these, the determinants of attrition are identified in terms of baseline data, while the dependent variable attrition is measured in the second wave of the study. Demographic data are unlikely to change much between the two assessments, but psychopathology may be subject to greater variation (even though many disorders are chronic). Second, we used DSM-III-R psychopathology as a predictor of attrition. Mental health problems below the clinical level might be associated with attrition as well. Third, because we analyzed numerous independent variables and three outcome variables, some associations might be significant by chance.
How much bias causes the attrition associated with psychopathology to affect the measurements at the second wave, such as prevalence and incidence rates? These prevalences and incidences differ only slightly from those controlled for attrition due to psychopathology at baseline. For example, in the general population, the 12-month prevalence of one or more diagnoses at the second wave, weighted for demographics, was 15.3 percent. When we also weighted for the presence of at least one disorder in the 12 months before the first wave, this prevalence was 15.5 percent.
Methodological research is important to minimize attrition in future longitudinal studies (26). As the present analysis has shown, epidemiologic research on mental health should pay extra attention to participants likely to be more difficult to trace in follow-up waves. If researchers keep in touch with the groups in question, they may succeed in tracking down more of the difficult-to-reach respondents, which will enhance the validity of the survey outcomes.
![]() |
ACKNOWLEDGMENTS |
---|
The authors thank Dr. J. Ormel (University of Groningen), Dr. W. van den Brink (University of Amsterdam), Dr. H. Verkleij (RIVM), and S. Laitinen-Krispijn (Trimbos Institute) for their comments on previous versions of this manuscript.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|