Addiction Research Institute (IVO), Heemraadssingel 194, 3021 DM Rotterdam, The Netherlands
Received 11 July 2001; in revised form 30 October 2001; accepted 12 November 2001
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Non-response leads to a smaller final sample size and, therefore, to a loss of accuracy in population estimates. However, if the non-response is not related to the research variable of interest, taking larger samples can compensate for this loss. Conversely, if non-response is directly related to the research topic, errors may occur, which can seriously distort the survey results. This non-response bias occurs when a significant number of people in the survey sample fail to respond and have relevant characteristics that differ from those who do respond (Dillman, 2000). In such cases, the non-response is selective.
A widely used method to correct for a non-response bias is corrective weighting of the survey data by use of socio-demographic variables (Lemmens et al., 1988; Bongers et al., 1997
). However, it has been shown that this method does not correct the bias sufficiently, because the inherent assumption that respondents and non-respondents within the same socio-demographic category are also equal on the outcome variable seems untenable (Bradburn, 1992
; Presser and Traugott, 1992
; Van Goor and Stuiver, 1995
).
In the field of alcohol research, non-response is frequently considered as a causal factor of the discrepancy between the survey-based estimates and the official (taxes-) based estimates of alcohol consumption. Pernanen (1974) has reported that many survey subpopulations with a higher proportion of heavy drinkers tend to show higher non-response rates; this can cause underreporting of the survey data on alcohol consumption, compared to the official sales data on alcohol consumption. Garretsen (1983) showed an underreporting of both the frequency and the quantity of alcohol consumption. The underestimation varied according to the subpopulations; for example, underreporting of alcohol consumption was higher among women than men, but there was no support for a higher non-response among the problem drinkers. Knibbe (1982) found some evidence for a higher percentage of abstainers and excessive drinkers among the non-respondents whereas Lemmens et al. (1988) found no support for the hypothesis of higher alcohol consumption among non-respondents, compared with respondents. In the same study, Lemmens and colleagues did show that female non-respondents generally drank less and had higher abstention rates. Furthermore, although occasional heavy alcohol use in the previous 6 months was more frequent among male non-respondents than among male respondents, the reverse was true for frequent heavy alcohol use. A follow-up of non-respondents of the (American) National Household Survey of Drug Abuse showed no differences between non-respondents and respondents in the prevalence of alcohol consumers (Caspar, 1992). Gmel (2000) found no differences between non-respondents and respondents in higher alcohol consumption; although the mean consumption and the percentage of heavy drinkers were higher in non-respondents, these differences were not significant. Wild et al. (2001) showed that non-responders to a follow-up questionnaire on alcohol use and beliefs about drinking reported consuming five or more standard drinks once per week or more at a slightly greater rate (19.1%) than did respondents (14.3%).
A limitation of most follow-up studies among non-respondents is the relatively low response rates of these secondary surveys. In the follow-up of non-respondents, generally there are many initial non-respondents who cannot be reached or who refuse again (Sosdian and Sharp, 1980; Jansen and Hak, 1999
). It is often assumed that excessive drinkers and/or problem drinkers are difficult to reach because of their lifestyle characteristics (e.g. homelessness, seldom at home, not answering their mail). It is also assumed that excessive drinkers and/or problem drinkers, if reached, are more likely to refuse to participate in a survey than light and moderate drinkers.
Various models and theories have been developed to identify and explain non-response. Dillman (2000) applied the social exchange theory as a basis for survey design methods aiming at maximizing the response rate. According to this theory, actions of individuals, in this case responding or not responding, can be predicted on the basis of three elements: rewards, costs and trust. Rewards are what one expects to gain from a particular activity, e.g. social validation, appreciation, liking to dO', or tangible rewards, such as money. In conducting a survey, a phrase such as we very much appreciate your help has a reward value for many people. Liking to dO' is also a powerful determinant of behaviour; most people enjoy participating in a survey if the topic is of interest to them. Costs are what one gives up or spends to obtain the rewards, e.g. inconvenience, embarrassment, feeling subordinated, anxiety, great physical or mental effort, or invasion of privacy. A simple act such as inclusion of a (paid) envelope for a mailed questionnaire increases the response rate, whereas questions that may cause embarrassment or anxiety lower the response rate. The third element in the social exchange theory is trust. This is the expectation that in the long run the rewards of doing something will outweigh the costs, e.g. participants may see a legitimate authority or a token of appreciation in advance as a form of trust.
On a more concrete level, Groves et al. (1992) reported various (reward/cost) factors which can influence survey participation. They distinguished factors at four levels: i.e. societal-level factors; attributes of the survey design; attributes of the interviewer and respondentinterviewer interaction; and characteristics of the sample person. Examples of societal-level factors are the survey-taking climate, the legitimacy of societal institutions or social cohesion. Attributes of the survey design can be the mode of the survey, the length of the questionnaire or the survey topic. Relevant personal characteristics are age, gender, income and health status of the sample person. These latter characteristics are also important for the interviewer. Factors related to respondentinterviewer interaction can include strategies of the interviewer to persuade the respondents or expectations of the interviewer and the interviewee.
These theories are tools for understanding the decision to participate (or not) in a survey, but do not specifically address the question of non-response bias. Most factors mentioned by Groves et al. (1992) may cause bias only in an indirect way, whereas non-response bias is strongest when respondents select themselves in relation to the perceived topic of the survey. Therefore, here the focus is on topic as the main explanatory factor of non-response in our earlier survey on alcohol consumption. The two main aspects of the topic with regard to responsiveness are reported to be salience and social desirability. Heberlein and Baumgartner (1978) showed that salience (interested or not interested in the topic) has a strong influence on the response rate. When the topic of the survey is salient to the respondent, the costs of responding may be reduced. Martin (1994) verified that people's interest in the survey topic can have considerable impact on response rates: persons were almost twice as likely to participate if the topic was of high interest. Similarly, Dillman and Carley-Baxter (2000) showed that salience is a significant determinant of response rate. Others have shown that people who feel threatened by a topic behave socially less desirably or feel embarrassed/ashamed with respect to the survey topic and thus respond less (Gannon et al., 1971; Green, 1991
). For example, socially acceptable behaviour, such as exercise and good nutrition, may be frequently overreported, whereas undesirable behaviour, such as smoking and drinking, may be underreported (Warnecke et al., 1997
; Bongers, 1998
). Van Goor and Stuiver (1995) showed a pattern of overrepresentation of non-response in both extreme categories of his outcome variable, i.e. the effectiveness of governmental organizations in policy making with regard to trailer camps. In his explanation of this pattern, the factor topic plays an important role in that the very effective organizations lost interest in the topic of effectiveness and the less effective organizations did not respond because they felt threatened by the survey because of their poor performance. This latter study gave rise to a two-tailed pattern of non-response bias. We hypothesize that this pattern can also be applied to surveys on alcohol consumption. Two hypotheses are formulated: (1) non-response is high at the lowest end of the research variable, in our case the abstainers; (2) non-response is also high at the upper end of the research variable, in our case the frequent excessive drinkers. These hypotheses rely on the same reasoning as used by Van Goor and Stuiver (1995). Abstainers do not respond because they have no experience with alcohol and therefore may not be interested in it and/or may not see the relevance of their response. Frequent excessive drinkers do not respond because this group may perceive their amount of consumption as socially undesirable.
In the present study, the two hypotheses were tested by conducting a follow-up investigation of non-respondents to a mailed questionnaire on alcohol consumption, in order to compare them with respondents.
![]() |
SUBJECTS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Information about the primary respondents' alcohol use was derived from their answers to two alcohol questions in the mailed questionnaire, which were completed by all of them. All primary non-respondents were approached by means of house visits without prior notice. A maximum of five attempts was made to reach the right person at home. During the fieldwork, 22 of the 177 primary non-respondents appeared to have moved house or were absent for a long period; these persons were deemed ineligible. Moreover, it proved impossible to contact another 26 primary non-respondents (they were not at home or did not open the door). Finally, having lost 48/177 persons to follow-up, 129 primary non-respondents were contacted during the house visits; this yielded a contact rate of 83% [129/(177 22)]. Of the several questions asked of the primary non-respondents, two were about alcohol use. Of the 129 primary non-respondents, 102 answered at least one alcohol question and 80 of these 102 secondary respondents answered both alcohol questions; this yielded a net response rate of 52% [80/(177 22)].
Both the primary respondents and secondary respondents were asked the same two alcohol questions.
Measures
Alcohol consumption was measured by the following two items: Do you ever drink alcoholic beverages? and Do you ever drink six or more units of alcoholic beverages in one day? The frequency of the alcohol consumption was assessed in the first instance by a 9-point scale (every day, more than 3 times a week, 2 or 3 times a week, once a week, 2 or 3 times a month, once a month, less than once a month, never drinking 6 units in one day and never drinking alcohol at all). Table 1
presents the classification used for alcohol consumption. This classification is based on a combination of those used by Garretsen (1983) and Wild (2001). The socio-demographic measures used for the analyses included: gender, age and nationality. Age was classified as a categorical variable (1625, 2635, 3669 years). Nationality was divided into two categories: Dutch only and non-Dutch only.
|
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Some other limitations of the study should be addressed. First, because the secondary non-response rate was ~52%, our conclusions rely on the assumption that the secondary non-response has the same effects on the estimates of alcohol consumption as the primary non-response. Another limitation concerns the representativeness of the sample. Only 25 postal areas in the city centre were included to obtain the sample for the follow-up study and the age group 3669 years was underrepresented in this second sample. Nevertheless, there were no significant differences in gender and nationality between the primary survey sample and the secondary survey sample. The study methods also differed. In the primary survey, a mailed questionnaire was used, whereas the follow-up of non-respondents consisted of face-to-face interviews. It has been suggested that, due to greater anonymity, mail surveys encourage fuller report of sensitive topics than face-to-face interviews (De Leeuw, 1992; Gmel, 2000
; Kraus and Augustin, 2001
), whereas others found higher reported alcohol consumption in personal interviews (Cutler et al., 1988
; Rehm and Arminger, 1996
). Because of these conflicting reports, it is difficult to assess the impact of the different methods used in our alcohol surveys.
Despite these limitations, the results of this study reveal a serious non-response bias in our primary survey, which is directly related to the topic of the survey, namely alcohol consumption. This bias cannot be corrected by weighting data on the basis of socio-demographic variables, because, within our subgroups, the same bias exists. Therefore, the aim should be to minimize non-response in a survey by, for example, developing more appealing survey materials. Furthermore, this study confirms the need for a thorough non-response follow-up study to evaluate non-response biases, as also emphasized by Hill and Roberts (1997) in the field of lifestyle surveys. Because of the time-consuming effort required to contact primary non-respondents, it is advisable to restrict the follow-up study to a small subsample, thus allowing more attention to be given to the individual subject.
![]() |
FOOTNOTES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Bongers, I. M. B., Lemmens P. H. H. M., Oers, H. A. M. van and Tan, F. E. S. (1997) Methode ter correctie voor vertekening van surveyresultaten ten gevolge van indirecte selectieve non-respons. Tijdschrift voor Sociale Gezondheidszorg 75, 122128.
Bradburn, N. M. (1992) A response to the nonresponse problem. Public Opinion Quarterly 56, 391397.[ISI]
Caspar, R. (1992) Follow-up of non-respondents in 1990. In Survey Measurement of Drug Use, Turner, C. F., Lessler, J. T. and Gfroerer, J. C. eds, pp. 155173. National Institute on Drug Abuse, Rockville.
Cutler, S. F., Wallace, P. G. and Haines, A. P. (1988) Assessing alcohol consumption in general practice patients a comparison between questionnaire and interview. Alcohol 23, 441450.
De Heer, W. (1999) International response trends: results of an international survey. Journal of Official Statistics 15, 129142.
De Leeuw, E. D. (1992) Data quality in mail, telephone, and face-to-face surveys. Dissertation, Vrije Universiteit Amsterdam, Amsterdam.
Dillman, D. A. (2000) Introduction to tailored design. In Mail and Internet Surveys: The Tailored Design Method, Dillman, D. A. ed., pp. 331. Wiley, New York.
Dillman, D. A. and Carley-Baxter, L. R. (2000) Structural determinants of mail survey response rates over a 12 year period, 19881999. Proceedings of the section on survey method. The American Statistical Association, Alexandria, VA. Published on web page http://survey.sesrc.wsu.edu/dillman
Gannon, M. J., Northern, J. C. and Carrol, S. J., Jr (1971) Characteristics of nonrespondents among workers. Journal of Applied Psychology 55, 586588.[ISI]
Garretsen, H. F. L. (1983) Probleemdrinken: prevalentiebepaling, beïnvloedende factoren en preventiemogelijkheden. Dissertation, Swets and Zeitlinger B.V., Lisse.
Gmel, G. (2000) The effect of mode of data collection and of non-response on reported alcohol consumption: a split-sample study in Switzerland. Addiction 95, 123134.[ISI][Medline]
Green, K. E. (1991) Reluctant respondents: differences between early, late and nonresponders to a mail survey. Journal of Experimental Education 56, 268276.
Groves, R. M., Cialdini, R. B. and Couper, M. P. (1992) Understanding the decision to participate in a survey. Public Opinion Quarterly 56, 475495.[Abstract]
Heberlein, T. A. and Baumgartner, R. M. (1978) Factors affecting response rates to mailed questionnaires: a quantitative analysis of the published literature. American Sociological Review 43, 447462.[ISI]
Hill, A. and Roberts, J. (1997) Non-response bias in a lifestyle survey. Journal of Public Health in Medicine 19, 203207.
Jansen, H. and Hak, T. (1999) Nonresponse to mail surveys in a lower-class urban area a two stage exploration of access failure and refusal. Bulletin de Méthodologie Sociologique 62, 527.
Knibbe, R. A. (1982) Probleemdrinken in Limburg. Rijkuniversiteit Limburg, Maastricht.
Kraus, L. and Augustin, R. (2001) Measuring alcohol consumption and alcohol-related problems: comparison of responses from self-administered and telephone interviews. Addiction 96, 459471.[ISI][Medline]
Lemmens, P. H. H. M., Tan, E. S. and Knibbe, R. A. (1988) Bias due to non-response in a Dutch survey on alcohol consumption. British Journal of Addiction 83, 10691077.[ISI][Medline]
Martin, C. L. (1994) The impact of topic interest on mail survey response behaviour. Journal of Market Research Society 36, 327338.
Pernanen, K. (1974) Validity of survey data on alcohol use. In Research Advances in Alcohol and Drug Problems, Vol. 1, Gibbens, R., Israël, Y., Kalant, H., Popham, R., Schmidt, W. and Smart, R. G. eds, pp. 355374. Wiley, New York.
Presser, S. and Traugott, M. (1992) Little white lies and social science models: correlated response errors in a panel study of voting. Public Opinion Quarterly 56, 7786.[Abstract]
Rehm, J. and Arminger, G. (1996) Alcohol consumption in Switzerland 198793: adjusting for differential effects of assessment techniques on the analysis of trends. Addiction 91, 13351344.[ISI][Medline]
Sosdian, C. P. and Sharp, L. M. (1980) Non-response in mail surveys: access failure or respondent resistance. Public Opinion Quarterly 44, 396402.[Abstract]
Van Goor, H. and Stuiver, B. (1995) Succes en falen van beleid en non-respons: een empirisch onderzoek naar het terugzenden van schriftelijke enquêtes door Nederlandse gemeenten. Sociologische Gids 42, 388406.
Warnecke, R. B., Johnson, T. P., Chávez, N., Sudman, S., O'Rourke, D. P., Lacey, L. and Horm, J. (1997) Improving question wording in surveys of culturally diverse populations. Annals of Epidemiology 7, 334342.[ISI][Medline]
Wild, T. C. (2001) Effects of personal drinking on perceived norms for alcohol consumption, approval of others' drinking, and criteria for defining alcohol problems. Paper presented at the 27th Annual Alcohol Epidemiology Symposium of the Kettil Bruun Society for Social and Epidemiological Research on Alcohol, 28 May1 June, 2001, Toronto, Canada.
Wild, T. C., Cunningham, J. and Adlaf, E. (2001) Nonresponse in a follow-up to a representative telephone survey of adult drinkers. Journal of Studies on Alcohol 62, 257261.[ISI][Medline]