Contribution of Follow-up of Nonresponders to Prevalence and Risk Estimates: A Norwegian Respiratory Health Survey

Jan Brøgger1,, Per Bakke1, Geir E. Eide2,3 and Amund Gulsvik1

1 Department of Thoracic Medicine, Institute of Medicine, University of Bergen, Bergen, Norway.
2 Centre for Clinical Research, Haukeland University Hospital, Bergen, Norway.
3 Section for Medical Statistics, University of Bergen, Bergen, Norway.

Received for publication May 16, 2002; accepted for publication September 30, 2002.


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Achieving a high response rate is often expensive and time consuming. Does an extensive survey effort change prevalence estimates and exposure-disease relations? In 1998–1999, the authors conducted a population-based respiratory health survey in two Norwegian counties (Oslo, n = 20,000; Hordaland, n = 5,000) of a random sample of the adult population aged 15–70 years. A postal questionnaire was mailed, with as many as two reminder letters. A 25% random sample of postal nonresponders was contacted by ordinary or cellular telephone. Cumulative response rates after the first mailing, first reminder, second reminder, and telephone follow-up were 42.7%, 60.7%, 68.3%, and 79.9%, respectively. Compared with initial postal responders, responders at later stages were younger, and more were male. Late responders had a lower educational level, were more often smokers, and were occupationally exposed to dusts or gases more frequently. After the authors adjusted for these factors, late responders were found to have less asthma, attacks of breathlessness, and hay fever. Hardly any changes in prevalences and odds ratios were noted when initial responders were compared with all responders. The additional contribution of sending reminder letters and conducting a telephone follow-up to prevalence estimates and the exposure-disease relation was small. A single letter would have produced nearly the same prevalence estimates and risk factor associations.

asthma; data collection; epidemiologic methods; health care; interviews; postal service; questionnaires; signs and symptoms, respiratory


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Nonresponse bias is a potential problem in most population surveys (1, 2). Definitive investigation of nonresponse bias would require a 100 percent response rate, which is costly and nearly impossible to achieve. The "willingness to respond" approach has been used instead (35). Early responders or responders who are easier to reach are compared with later responders or responders who are more difficult to reach. In essence, survey effort is used as a proxy for nonresponse bias. This approach has been applied to prevalence estimates in cross-sectional population surveys on a wide range of topics (620). We are aware of only two studies that have looked at whether exposure-disease relations are in fact modified by a more intensive survey effort, and these studies did not have high statistical power (2023). Additionally, approaching those nonresponders who do not have an ordinary telephone through their cellular phones may be a new way of improving response rates. To our knowledge, a systematic review of nonresponse bias in respiratory epidemiology has not been performed before.

To investigate the effect of increasing survey efforts on results, we performed a large-scale, population-based survey on respiratory health by using a postal questionnaire with a telephone and cellular phone follow-up. We also conducted a systematic literature review of studies of nonresponse bias in respiratory epidemiology.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
This cross-sectional study of the adult population was conducted in two regions of Norway, the capital county of Oslo and the county of Hordaland. The sampling frame was inhabitants in the Central Population Register aged 15–70 years on December 31, 1997. Random samples of 20,000 (Oslo) and 5,000 (Hordaland) inhabitants were selected.

Fieldwork
A postal questionnaire was mailed to each subject. After 3 weeks and 8 weeks, nonresponders were sent a reminder letter. Responders were classified as initial responders and responders to the first and second reminder letters.

A telephone follow-up was initiated for a 25 percent random sample of postal nonresponders. Telephone numbers for the follow-up sample were located by using a direct mail firm (DM Huset, Oslo, Norway), requiring a match on name, date of birth, and address. Six months after the first mailing, computer-assisted telephone interviews were performed by trained operators. Nine months after the study started, nonresponders without a telephone directory entry but with a cellular phone were located from the customer databases of the cellular phone providers. A similar cellular telephone interview was conducted.

Questionnaires were mailed, responses were processed, and telephone interviews were performed by the Interview Office of Statistics Norway.

Questionnaire
The questions regarding symptoms were taken from a modified translation of the British Medical Research Council on Chronic Bronchitis (24, 25). A one-sheet questionnaire designed for self-completion was used; it contained 45 questions to be answered by all subjects on 18 respiratory symptoms, seven respiratory and cardiac diagnoses, and 20 risk factors. Smokers were defined as daily smokers at the time of the study. Former smokers were persons who had smoked daily but had stopped (26). The study was approved by the Norwegian Data Inspectorate.

Statistical methods
Responders were categorized as initial responders, responders to the first reminder letter, responders to the second reminder letter, and responders to the telephone-based follow-up. These categories represented "survey effort." Prevalences and cumulative prevalences of symptoms or diagnoses by survey effort were computed, with 95 percent confidence intervals. A two-sided p value of less than 5 percent was taken as statistically significant. Equality between groups was investigated by using analysis of variance, ordered logistic regression, or logistic regression, as applicable. All statistical tests were of heterogeneity, not trend. To control for possible confounding of prevalence estimates, multiple logistic regression was performed (27). Age was entered as a cubic spline with knots at 15, 30, 50, and 70 years (28). Smoking habits was modeled as pack-years. Pack-years was modeled as a linear term for current smokers only. The adequacy of doing so was checked by using generalized additive models (29) with scatter-plot smoothing terms with 4 degrees of freedom for pack-years and age. Changes in the following risk factors with survey effort were assessed with interactions: gender, occupational exposure to dusts or gases, and pack-years. This analysis also adjusted for age, county, and educational level. A joint test that any interaction was significant was performed, and those significant at p < 0.10 were considered further. In addition, the odds ratios of the risk factors for all responders were computed. A missing response was defined as a selection of "don’t know" or no response to a given question. Subjects for whom answers were missing in an analysis were excluded from that analysis. All calculations accounted for the sampling in the follow-up by using weighting and robust standard errors. Adjusted Wald tests were used to decide on the significance of terms. All data were analyzed by using Stata 7.0 software (30).

Literature search
A systematic literature search was performed to find studies of nonresponse bias in respiratory epidemiology. The following databases were searched in early 2002: MEDLINE (Bethesda, Maryland), CINAHL (Glendale, California), and the Cochrane databases through Ovid (New York, New York; http://www.ovid.com/). We also searched the database Science Citation Index Expanded (http://wos.isiglobalnet.com/) by using the phrase (("nonresponse" or "response" or "nonresponder" or "responder") and ("epidemiology" or "health surveys") and (pulm* or respirat* or lung*)). Studies that cited Drane et al. (31, 32) were identified by searching the citation database Science Citation Index Expanded. Matching references were retrieved and were screened on the basis of title and abstract. Only those studies of the general population in a defined geographic area on respiratory symptoms and/or diagnoses with at least a 10-year age span in the interval 15–70 years were included. Reference lists of retrieved articles were also searched.

For simplicity purposes and to enable a conservative comparison of response rates across studies, response rates were recomputed as the ratio of the number of answers to the number of subjects who had not yet answered at that stage, regardless of whether they had received the letter, had a telephone, were dead, or had moved. Some numbers were calculated from percentages or read from graphs. The total response rate was recalculated with weighting for possible subsampling in the follow-up, when available.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
The response rates for the first mailing, the first reminder letter, and the second reminder letter were 42.7 percent, 31.4 percent, and 19.5 percent, respectively. There were 7,917 postal nonresponders, of whom 864 had moved and had an unknown address, 271 had actively refused, and 88 were excluded for other reasons. Of the remaining 6,694 nonresponders, 1,669 were randomly selected for follow-up. Telephone numbers were identified for 1,176 (70.5 percent). Of the 493 persons for whom no telephone number was found, a cellular phone number was available for 175 (35.5 percent). The response rate for the ordinary telephone interview (n = 660) was 56.1 percent, while that for the cellular phone interview (n = 61) was 34.9 percent.

The cumulative response rates after the first mailing, the first reminder letter, the second reminder letter, and the telephone follow-up were 42.7 percent, 60.7 percent, 68.3 percent, and 79.9 percent, respectively. The respective response rates in Oslo were 41.8 percent, 30.1 percent, 18.5 percent, and 34.6 percent; those in Hordaland were higher at 46.4 percent, 36.8 percent, 24.0 percent, and 46.6 percent for each successive phase of follow-up.

Willingness to respond was associated with age, gender, educational level, smoking habits, and occupational exposure to dusts or gases univariately (table 1). Men responded later than women. Age and educational level decreased with increasing contact effort. Current smoking and occupational exposure were more prevalent and pack-years increased with increasing contact effort.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Age, gender, educational level, smoking habits, and occupational exposure, by intensity of follow-up, of subjects in the Oslo-Hordaland respiratory disease survey, Norway, 1998–1999
 
The mean age of the sample was 40.0 years compared with 41.9 years for initial responders only and 41.0 years for all responders. The percentage of women in the sample was 50.0 percent. Women constituted 54.3 percent of the initial responders and 52.1 percent of all responders. Changes in prevalence by the end of the survey compared with prevalence for only the immediate responders were small (<2.5 percentage points), except that the prevalence of current smokers increased by 3.8 percentage points (from 30.1 percent to 33.9 percent). The mean number of pack-years of smoking increased by 0.3. Confidence intervals were only slightly narrower in the final estimate.

The crude prevalences of symptoms and diagnoses by survey effort are shown in table 2. All symptoms were associated with survey effort except for attacks of breathlessness, dyspnea when walking two stairs, and dyspnea at rest. There was no uniform tendency toward an increase or decrease in prevalence with survey effort. Note in particular the large change for morning cough in responders to the telephone follow-up. No association for diagnoses was found.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Prevalence of self-reported symptoms or diagnoses, by intensity of follow-up, among subjects in the Oslo-Hordaland respiratory disease survey, Norway, 1998–1999
 
After adjustment for age, gender, county, educational level, pack-years, and occupational exposure, there was no longer any significant change in chronic cough and wheezing, whereas asthma was now associated with survey effort (table 3). No uniform tendency toward a decrease or increase in the odds ratios with survey effort was evident. Changes in the prevalence of symptoms or diagnoses by the end of the survey compared with the prevalence of only the immediate responders were small (<1 percentage point). The changes appeared to be stronger for the telephone responders.


View this table:
[in this window]
[in a new window]
 
TABLE 3. Adjusted* odds ratios of symptoms or diagnoses, by intensity of follow-up, among subjects in the Oslo-Hordaland respiratory disease survey,{dagger} Norway, 1998–1999
 
The effect of risk factors on symptoms might vary with survey effort. Figure 1 shows the association of risk factors with survey effort of those symptoms or diagnoses for which the p value of the test of a change in one or more risk factors with survey effort was less than 0.10. Risk factors evaluated were gender, occupational exposure to dusts or gases, and pack-years. Of 16 symptoms or diagnoses, only chronic cough (joint-test p = 0.06) and ever wheezing (joint-test p = 0.10) reached the threshold. For chronic cough, the effect of gender was found to vary by survey effort (p = 0.05), probably driven by the large effect of gender among telephone responders. For wheezing, the effect of pack-years was significantly different (p = 0.03), probably because of the lesser effect of pack-years among telephone responders. These changes had little impact, since changes in odds ratios by the end of the survey compared with the odds ratios of only the immediate responders were small.



View larger version (19K):
[in this window]
[in a new window]
 
FIGURE 1. Adjusted odds ratios of risk factors for symptoms, by survey effort, in the 1998–1999 Oslo-Hordaland, Norway, respiratory disease survey of adults aged 15–70 years (horizontal lines are 95% confidence intervals) and all responders. Risk factors considered for chronic cough and wheezing were gender (women vs. men), occupational exposure to dusts or gases (any vs. none), and pack-years of smoking (per 10-year increase). Adjusted for age, educational level, and county. Shown are only those symptoms or diagnoses for which the p value of the joint test of any interaction was less than 0.1.

 
Analyses were rerun separately for Oslo and Hordaland. Analyses of unadjusted and adjusted prevalences, and associations with risk factors, by county showed the same trends and magnitudes of associations.

Cost and time were substantial for the follow-up. The second reminder letter increased the cost of the survey by 25 percent. The telephone follow-up increased the cost of the survey by an additional 14 percent and netted 741 responses, while data collection time increased from 4 months to 12 months.

The literature search netted 305 hits in the electronic databases, of which 11 were included (22, 23, 3341) (table 4).


View this table:
[in this window]
[in a new window]
 
TABLE 4. Results of systematic review of the initial sample size, number of responders, and response rates (%),* by intensity of follow-up, in studies of nonresponse bias in respiratory health surveys of the general adult population{dagger}
 

    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
We have shown an association between survey effort regarding prevalences and exposure-disease relations, but changes were small when we compared all responders with only immediate responders. The effect of gender on chronic cough varied with survey effort, and the effect of pack-years on wheezing varied with survey effort. Through a telephone-based follow-up, we managed to increase the response rate from 68.3 percent to 79.9 percent. Although we found little evidence of nonresponse bias by survey effort, average age improved toward the sample mean of 40.0 years from the initial responders only (41.9 years) through all responders (41.0 years).

Previous surveys using exactly the same postal survey methods had response rates of 89 percent in Oslo in 1972 (42) and 90 percent in Hordaland in 1985 (23). This decrease in response rates may reflect a temporal trend (43), although solid long-term data on response rates indicate that this trend varies with country and the circumstances of the survey (44). There may be reasons other than a simple decline in willingness to respond (4). The introductory text was longer because of regulatory and ethical concerns, and the questionnaire itself was slightly longer. The previous surveys were executed locally and were mailed in a University of Bergen envelope. The present survey was performed with the aid of Statistics Norway and was sent in a Statistics Norway envelope.

Oslo had a response rate 4–12 percent lower than that for Hordaland at each stage of survey follow-up. The survey methods were exactly the same in both counties. A number of sociodemographic differences between Oslo and Hordaland might explain the result. Oslo is the capital of Norway and is the undisputed economic and political center. It is a wholly urban area of 500,000 persons situated in a greater metropolitan area of about 1 million inhabitants (out of 4.5 million inhabitants in Norway as a whole). Hordaland is the center of coastal western Norway, a mixed urban and rural area with the city of Bergen (250,000 inhabitants) at its center (45). Gross income is 20 percent higher in Oslo than in Bergen (45).

Oslo has a large population of foreigners (foreign born or born to one or both foreign parents), constituting 19 percent of the population, whereas the figure is only 6.4 percent in Bergen (46). In the 2000 Norwegian census, the response rate was as low as 56 percent among those from Asia, Africa, and Latin America. When we reran the analyses separately by study area, the effects were of the same magnitude and direction.

We identified a telephone number for only 70 percent of the follow-up sample, whereas telephone coverage is universal in Norway. This figure was probably due to our conservative telephone book look-up algorithm. However, we then found a cellular telephone number for as many as 35 percent of those for whom we could not identify an ordinary telephone number.

For both the adjusted and unadjusted estimates, it appeared that the changes among telephone responders were driving most of the associations. Telephone responders had a lower risk of symptoms. The use of a telephone interview instead of a postal questionnaire may explain this finding. However, in an earlier study (47), we found few differences between the postal and the telephone survey mode. A previous Norwegian study found that postal nonresponders were more likely to be smokers but had fewer symptoms (23). Telephone responders are probably less symptomatic and hence probably perceive little benefit from participating in a postal questionnaire. When questioned directly on the telephone, resistance to response may be overcome. Our findings support this theory.

For differences in prevalence, our study had substantial power. The tests of changes in risk factors with survey effort would have smaller power. There was also some concern about multiple statistical tests, especially for the interactions of risk factors with survey effort. However, we tested a limited number of plausible interactions in a hierarchic fashion designed to reduce the number of false positives (48). There may have been differences in exposure-disease associations for other symptoms, but our data could not provide enough precision to pinpoint them. Differences for other risk factors may also have been possible. These we did not test, to avoid multiple comparisons.

The systematic literature search found 11 respiratory epidemiology articles that contained some data on nonresponse bias (22, 23, 3341). The response rates to the first postal questionnaire varied from less than 22 percent to 69 percent. A number of reasons for this variation are possible. Monso et al. (41) had a very low response rate, which, even after one reminder, was only 22 percent. These authors used a 1991 census sample frame, and it appears that the study was initiated in or about 1996. The study by Howard et al. (33) was conducted in a bilingual Hispanic population and used a commercial household directory that was not as up-to-date. Clearly, a prerequisite for a high response rate is a good-quality, up-to-date sampling frame. Even so, the remaining initial response rates varied from less than 41 percent to 69 percent. All the Nordic studies (23, 37, 38, 40) except ours had initial postal response rates of 60 percent or more. These studies were all based on mandatory universal population registries. Both Chinn et al. (34) and de Marco et al. (39) used a population-based register, but ones that were used for health care purposes, so quality may have been less than optimal.

All studies that reported on smoking found that smokers were late responders (table 4) (22, 23, 3739, 41). The relation of intensity of follow-up with symptoms and diagnoses was not universal; one study showed no relation (22), others found an increased prevalence of symptoms with intensity of follow-up (23, 37, 38), and others found a decreased prevalence of symptoms with follow-up (36, 39). None of these studies adjusted for potential confounders. However, follow-up had little effect on the final prevalence.

Only two studies are known to have investigated exposure-disease associations by survey effort (22, 23). One found a difference in an exposure-disease relation that caused occupational exposure to be no longer significant in a stepwise regression (23), but both reported little effect on final exposure-disease relations (22, 23).

The total sample size was moderate to large in most of these studies, but the number of respondents in follow-ups was relatively small (table 4). For example, the mean number of respondents in the telephone responder group was 389. If the prevalence of a symptom was 10 percent in a group with 389 respondents, the 95 percent confidence interval would be 7.2 percent to 13.4 percent. If this prevalence was compared with the prevalence in a larger group of initial responders, this confidence interval might obscure an important difference. In our study, the impact of follow-up on confidence intervals was small. Smaller studies may find it necessary to include a follow-up on the basis of study power alone. Most etiologic studies in respiratory epidemiology require adjustment for confounding, and doing so increases the required sample size. This adjustment is especially important for continuous variables for which sparseness of data is likely to be a problem.

Drane (31) proposed a linear or exponential extrapolation of trends in prevalence by reminder letter (32), which is attractive in its simplicity. This method has seen some use (34, 36, 39) but no thorough validation. There is little theoretical or empirical justification for selection of a linear or exponential trend. A what-if scenario with plausible alternatives might be just as reasonable. One plausible alternative is that the latest responders are most "equal" to the nonresponders and that prevalences for the nonresponders should be "set" to the prevalence of these latest responders (5). We did not explore this alternative, since neither of these techniques is sophisticated enough to cater to the multivariate situation, for example, when exposure-disease estimates are estimated and are controlled for confounding (49).

A cautious conclusion is that, so far, no studies have found important nonresponse bias in respiratory epidemiology health surveys that used this method, in either prevalence estimates or exposure-disease associations. A number of studies in other fields of epidemiology have used the same "wave" or willingness-to-respond approach (6, 820). These studies cover a wide range of geographic areas, different populations, and a variety of subjects. Most have found trends in the prevalence of outcomes or risk factors by willingness to respond, but the effect on final estimates has been variable. Some have found strong differences (7, 18, 19), whereas most others have found only small differences (6, 817, 20, 22, 23). The effect on exposure-disease associations (e.g., odds ratios) has rarely been studied (20).

Surprisingly little is known regarding the consistent refusers, the residual nonresponders, which probably reflects the difficulty and cost involved in obtaining information on these persons. In our study, we had no information on the residual 20 percent of nonresponders, except for age, gender, and geographic location. In an extreme scenario, both prevalences and associations with risk factors could be quite different on a population level. If the trends observed for the telephone responder group persisted or increased, results on a population level might be somewhat different. Our own literature search (5055) and various PubMed searches (National Library of Medicine, Bethesda, Maryland) revealed a single study (12) with a very high response rate (e.g., >90 percent). In a later commentary, Sheikh took a skeptical view of nonresponse (56), although he offered few solid references.

A limited amount of data is available from some studies that have compared nonresponders and responders regarding information external to the study. This use of external information to track nonresponse bias has shown that nonresponders are likely to under- and overreport previous illnesses (57, 58), have higher admission rates to the hospital (20, 59), and are more likely to have a history of psychiatric illness (60). They may also have higher mortality (20, 6164) and differences in health care utilization (65)—possibly related to smoking (66). A number of mostly nonmedical studies are discussed by Wentland and Smith (51).

There is potentially a large nonresponse bias in most epidemiologic studies, or indeed in any survey. Outcomes are usually uncommon (e.g., <10 percent prevalence), and response rates are rarely above 90 percent. With a 90 percent response rate, a 5 percent prevalence among the responders may reflect a true prevalence of 4.5–14.5 percent in the whole population, given that the nonresponders have either a zero or a 100 percent prevalence of the outcome. This scenario may seem extreme but is probably the reason that response rates receive much attention, even given the fact that good examples of clear nonresponse bias are not abundant. In fact, nonresponse rates are sometimes used as the only indicator to judge the quality of a survey, which may be an oversimplification (67).

There are two possible interpretations of the present study. On the one hand, one may choose to use a larger sample size up front and a less-intense follow-up survey effort if it has already been established that nonresponse bias is not likely to be a large problem in the same population with the same survey topic. It would save time and money to use a large sample size and eliminate telephone follow-up to achieve a 50–60 percent response rate rather than to select a smaller sample size and aim for a response rate of 70–80 percent with follow-up. The larger survey would achieve about the same total number of responses and hence study power but in a shorter time and at a lower cost. In particular, the cellular telephone survey netted very few additional responses and required a substantial amount of time and work.

On the other hand, there is a large potential for nonresponse bias in all surveys, as noted above. A study may be greeted with skepticism if its response rate is as low as 43 percent, as it was in the present study. However, our study does show that pushing response rates from 60 percent to 80 percent would have been unlikely to materially affect conclusions since the additional reluctant responders were not very different from early responders. This finding is supported to some extent in smaller studies on other survey topics (6, 8, 9, 13, 15). If this finding can be replicated in other populations and for other survey topics, it could represent important savings of money and manpower.

In this study, the additional contribution of sending reminder letters and conducting a telephone follow-up to prevalence estimates and exposure-disease relations in respiratory epidemiology was small. In this population, a single letter would have been enough. The possibility that this finding can be generalized to other populations should be studied.


    ACKNOWLEDGMENTS
 
This study was supported by the Norwegian Research Council.

The assistance of the Interview Office of Statistics Norway is acknowledged. Karl Mørkved conceived of the idea of using cellular telephones.


    NOTES
 
Correspondence to Dr. Jan Brøgger, Department of Thoracic Medicine, Institute of Medicine, University of Bergen, N-5021 Bergen, Norway (e-mail: jan.brogger{at}med.uib.no). Back


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Groves RM. Temporal change in response rates. Survey errors and survey costs. New York, NY: John Wiley & Sons, 1989:145–55.
  2. Kessler RC, Little RJ, Groves RM. Advances in strategies for minimizing and adjusting for survey nonresponse. Epidemiol Rev 1995;17:192–204.[ISI][Medline]
  3. Groves RM. Probing the causes of nonresponse and efforts to reduce nonresponse. Survey errors and survey costs. New York, NY: John Wiley & Sons, 1989:185–294.
  4. Fox RJ, Crask MR, Kim J. Mail survey response rate: a meta-analysis of selected techniques for inducing response. Public Opin Q 1998;52:467–91.[CrossRef]
  5. Brenner H. Alternative approaches for estimating prevalence in epidemiologic surveys with two waves of respondents. Am J Epidemiol 1995;142:1236–45.[Abstract]
  6. Criqui MH, Barrett-Connor E, Austin M. Differences between respondents and non-respondents in a population-based cardiovascular disease study. Am J Epidemiol 1978;108:367–72.[Abstract]
  7. Parker C, Dewey M. Assessing research outcomes by postal questionnaire with telephone follow-up. TOTAL Study Group. Trial of Occupational Therapy and Leisure. Int J Epidemiol 2000;29:1065–9.[Abstract/Free Full Text]
  8. Holt VL, Martin DP, LoGerfo JP. Correlates and effect of non-response in a postpartum survey of obstetrical care quality. J Clin Epidemiol 1997;50:1117–22.[CrossRef][ISI][Medline]
  9. Boersma F, Eefsting JA, van den Brink W, et al. Characteristics of non-responders and the impact of non-response on prevalence estimates of dementia. Int J Epidemiol 1997;26:1055–62.[Abstract]
  10. Etter JF, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol 1997;50:1123–8.[CrossRef][ISI][Medline]
  11. Papageorgiou AC, Croft PR, Ferry S, et al. Estimating the prevalence of low back pain in the general population. Evidence from the South Manchester Back Pain Survey. Spine 1995;20:1889–94.[ISI][Medline]
  12. Sheikh K, Mattingly S. Investigating non-response bias in mail surveys. J Epidemiol Community Health 1981;35:293–6.[Abstract]
  13. Talley NJ, Weaver AL, Zinsmeister AR, et al. Onset and disappearance of gastrointestinal symptoms and functional gastrointestinal disorders. Am J Epidemiol 1992;136:165–77.[Abstract]
  14. Cote P, Cassidy JD, Carroll L. The Saskatchewan Health and Back Pain Survey. The prevalence of neck pain and related disability in Saskatchewan adults. Spine 1998;23:1689–98.[CrossRef][ISI][Medline]
  15. Gasquet I, Falissard B, Ravaud P. Impact of reminders and method of questionnaire distribution on patient response to mail-back satisfaction survey. J Clin Epidemiol 2001;54:1174–80.[CrossRef][ISI][Medline]
  16. Hebert R, Bravo G, Korner-Bitensky N, et al. Refusal and information bias associated with postal questionnaires and face-to-face interviews in very elderly subjects. J Clin Epidemiol 1996;49:373–81.[CrossRef][ISI][Medline]
  17. McCarthy GM, MacDonald JK. Nonresponse bias in a national study of dentists’ infection control practices and attitudes related to HIV. Community Dent Oral Epidemiol 1997;25:319–23.[ISI][Medline]
  18. Blanker MH, Groeneveld FP, Prins A, et al. Strong effects of definition and nonresponse bias on prevalence rates of clinical benign prostatic hyperplasia: the Krimpen study of male urogenital tract problems and general health status. BJU Int 2000;85:665–71.[CrossRef][ISI][Medline]
  19. Hill A, Roberts J, Ewings P, et al. Non-response bias in a lifestyle survey. J Public Health Med 1997;19:203–7.[Abstract]
  20. Paganini-Hill A, Hsu G, Chao A, et al. Comparison of early and late respondents to a postal health survey questionnaire. Epidemiology 1993;4:375–9.[ISI][Medline]
  21. Kreiger N, Nishri ED. The effect of nonresponse on estimation of relative risk in a case-control study. Ann Epidemiol 1997;7:194–9.[CrossRef][ISI][Medline]
  22. Siemiatycki J, Campbell S. Nonresponse bias and early versus all responders in mail and telephone surveys. Am J Epidemiol 1984;120:291–301.[Abstract]
  23. Bakke P, Gulsvik A, Lilleeng P, et al. Postal survey on airborne occupational exposure and respiratory disorders in Norway: causes and consequences of non-response. J Epidemiol Community Health 1990;44:316–20.[Abstract]
  24. Gulsvik A. Prevalence and manifestations of obstructive lung disease in the city of Oslo. Scand J Respir Dis 1979;60:286–96.[ISI][Medline]
  25. Brøgger J, Bakke P, Gulsvik A. Comparison of two respiratory symptoms questionnaires. Int J Tuberc Lung Dis 2000;4:83–90.[ISI][Medline]
  26. Bakke P, Gulsvik A, Eide G, et al. Smoking habits and lifetime occupational exposure to gases and dusts, including asbestos and quartz, in a Norwegian community. Scand J Work Environ Health 1990;16:195–202.[ISI][Medline]
  27. Kleinbaum DG. Logistic regression: a self-learning text. New York, NY: Springer-Verlag, 1999.
  28. Greenland S. Dose-response and trend analysis in epidemiology: alternatives to categorical analysis. Epidemiology 1995;6:356–65.[ISI][Medline]
  29. Venables WN, Ripley BD. Smooth regression. Modern applied statistics with S-PLUS. New York, NY: Springer-Verlag, 1999:281–302.
  30. Stata Corporation. Stata statistical software, version 7.0. College Station, TX: Stata Corporation, 2001.
  31. Drane JW. Imputing nonresponses to mail-back questionnaires. Am J Epidemiol 1991;134:908–12.[Abstract]
  32. Drane JW, Richter D, Stoskopf C. Improved imputation of non-responses to mailback questionnaires. Stat Med 1993;12:283–8.[ISI][Medline]
  33. Howard CA, Samet JM, Buechley RW, et al. Survey research in New Mexico Hispanics: some methodological issues. Am J Epidemiol 1983;117:27–34.[Abstract]
  34. Chinn S, Zanolin E, Lai E, et al. Adjustment of reported prevalence of respiratory symptoms for non-response in a multi-centre health survey. Int J Epidemiol 1995;24:603–11.[Abstract]
  35. Crane J, Lewis S, Slater T, et al. The self reported prevalence of asthma symptoms amongst adult New Zealanders. N Z Med J 1994;107:417–21.[ISI][Medline]
  36. Abramson M, Kutin J, Czarny D, et al. The prevalence of asthma and respiratory symptoms among young adults: is it increasing in Australia? J Asthma 1996;33:189–96.[ISI][Medline]
  37. Ronmark E, Lundqvist A, Lundback G, et al. Non-responders to a postal questionnaire on respiratory symptoms and diseases. Eur J Epidemiol 1999;15:293–9.[CrossRef][ISI][Medline]
  38. Bjornsson E, Plaschke P, Norrman E, et al. Symptoms related to asthma and chronic bronchitis in three areas of Sweden. Eur Respir J 1994;7:2146–53.[Abstract/Free Full Text]
  39. de Marco R, Verlato G, Zanolin E, et al. Nonresponse bias in EC Respiratory Health Survey in Italy. Eur Respir J 1994;7:2139–45.[Abstract/Free Full Text]
  40. Hedman J, Kaprio J, Poussa T, et al. Prevalence of asthma, aspirin intolerance, nasal polyposis and chronic obstructive pulmonary disease in a population-based study. Int J Epidemiol 1999;28:717–22.[Abstract]
  41. Monso E, Munoz-Rino F, Izquierdo J, et al. Occupational asthma in the community: risk factors in a western Mediterranean population. Arch Environ Health 1998;53:93–8.[ISI][Medline]
  42. Gulsvik A. Prevalence of respiratory symptoms in the city of Oslo. Scand J Respir Dis 1979;60:275–85.[ISI][Medline]
  43. Hartge P. Raising response rates: getting to yes. (Editorial). Epidemiology 1999;10:105–7.[ISI][Medline]
  44. Heer W. International response trends: results of an international survey. J Official Stat 1999;15:129–42.
  45. Statistics Norway. August 23, 2002. (http://www.ssb.no/).
  46. Lie B. Innvandring og innvandrere 2002. Statistics Norway. July 21, 2002. (In Norwegian). (http://www.ssb.no/emner/02/sa_innvand/sa50/sa50.pdf).
  47. Brogger J, Bakke P, Eide GE, et al. Comparison of telephone and postal survey modes on respiratory symptoms and risk factors. Am J Epidemiol 2002;155:572–6.[Abstract/Free Full Text]
  48. Kleinbaum DG. Modeling strategy for assessing interaction and confounding. Logistic regression: a self-learning text. New York, NY: Springer-Verlag, 1999:195–6.
  49. Rubin DB. Multiple imputation for nonresponse in surveys. New York, NY: John Wiley & Sons, 1987.
  50. Groves RM. Survey errors and survey costs. New York, NY: John Wiley & Sons, 1989.
  51. Wentland EJ, Smith KW. Survey responses: an evaluation of their validity. Chicago, IL: Academic Press, 1993.
  52. Gail MH, Benichou J. Encyclopedia of epidemiologic methods. Chichester, United Kingdom: Wiley, 2000.
  53. Rothman KJ, Greenland S, eds. Modern epidemiology. 2nd ed. Philadelphia, PA: Lippincott-Raven, 1998.
  54. Journal of Official Statistics. Statistics Sweden. August 23, 2002. (http://www.jos.nu/).
  55. International Conference on Survey Nonresponse 1999. Joint Program in Survey Methodology. August 23, 2002. (http://www.jpsm.umd.edu/icsn/).
  56. Sheikh K. Late response vs. non-response to mail questionnaire. Ann Epidemiol 1998;8:75–6.[CrossRef][ISI][Medline]
  57. O’Toole BI, Battistutta D, Long A, et al. A comparison of costs and data quality of three health survey methods: mail, telephone and personal home interview. Am J Epidemiol 1986;124:317–28.[Abstract]
  58. O’Mahony PG, Dobson R, Rodgers H, et al. Validation of a population screening questionnaire to assess prevalence of stroke. Stroke 1995;26:1334–7.[Abstract/Free Full Text]
  59. Vestbo J, Rasmussen FV. Baseline characteristics are not sufficient indicators of non-response bias follow up studies. J Epidemiol Community Health 1992;46:617–19.[Abstract]
  60. Hansen V, Jacobsen BK, Arnesen E. Prevalence of serious psychiatric morbidity in attenders and nonattenders to a health survey of a general population: the Tromso Health Study. Am J Epidemiol 2001;154:891–4.[Abstract/Free Full Text]
  61. Ives DG, Traven ND, Kuller LH, et al. Selection bias and nonresponse to health promotion in older adults. Epidemiology 1994;5:456–61.[ISI][Medline]
  62. Rosengren A, Wilhelmsen L, Berglund G, et al. Non-participants in a general population study of men, with special reference to social and alcoholic problems. Acta Med Scand 1987;221:243–51.[ISI][Medline]
  63. Heilbrun LK, Nomura A, Stemmermann GN. The effects of nonresponse in a prospective study of cancer. Am J Epidemiol 1982;116:353–63.[Abstract]
  64. Bisgard KM, Folsom AR, Hong CP, et al. Mortality and cancer rates in nonrespondents to a prospective study of older women: 5-year follow-up. Am J Epidemiol 1994;139:990–1000.[Abstract]
  65. Reijneveld SA, Stronks K. The impact of response bias on estimates of health care utilization in a metropolitan area: the use of administrative data. Int J Epidemiol 1999;28:1134–40.[Abstract]
  66. Macera CA, Jackson KL. Patterns of non-response to a mail survey. J Clin Epidemiol 1990;43:1427–30.[ISI][Medline]
  67. Groves RM. Nonresponse in sample surveys. Survey errors and survey costs. New York, NY: John Wiley & Sons, 1989:133–84.