Characteristics of Telephone Survey Respondents According to Willingness to Participate

Lynda F. Voigt1,2,, Thomas D. Koepsell2 and Janet R. Daling1,2

1 Fred Hutchinson Cancer Research Center, Seattle, WA.
2 Department of Epidemiology, School of Public Health and Community Medicine, University of Washington, Seattle, WA.

Received for publication January 28, 2002; accepted for publication August 13, 2002.


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
When conducting epidemiologic case-control studies, some investigators include only controls who can be interviewed within a certain time after contact and/or do not recontact potential participants who initially refuse, whereas others expend considerable effort to recruit reluctant respondents. This additional effort is only worthwhile if it results in a sample that is more representative of the target population. In this study, the authors used data collected from in-person interviews of 5,616 female controls to compare characteristics of willing, accessible respondents with those of their less accessible or less willing counterparts to determine whether or not the two groups differed with respect to lifestyle, socioeconomic status, health history, and demographic characteristics. Late responders were younger, were more likely to be non-White, were less likely to have attended college, and were more likely to be current smokers than early responders. Initial refusers were similar to late responders with respect to education and race. Initial refusers were also older, were less likely to be currently married, were less likely to have a managerial occupation, had fewer lifetime sexual partners, and were more likely to have a history of diabetes than early responders. These findings suggest that additional effort expended in recruiting reluctant respondents may often result in more accurate estimates of population characteristics that are of interest in epidemiologic research.

bias (epidemiology); case-control studies; data collection; epidemiologic methods; selection bias; telephone

Abbreviations: Abbreviation: RDD, random digit dialing.


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
In most epidemiologic studies, investigators expend considerable effort to recruit difficult-to-reach respondents so that participants will be representative of the population of interest. The validity of a study comes into question if response is poor (13). Response proportions generally improve as effort to recruit reluctant respondents increases. However, the pursuit of reluctant respondents is expensive and time-consuming. It is unclear whether the additional expense of attempting to convert persons who refuse to participate is worthwhile. Furthermore, some investigators restrict the amount of time between contact and interview to minimize recall bias or to avoid delaying the completion of the study.

Several studies have compared respondents who refuse to participate, or those who initially refuse and later agree, with respondents who readily agree to participate. Most have found that reluctant or difficult-to-reach respondents are older and less educated than respondents who readily agree (49). Differences with respect to income, occupation, race, and marital status have been inconsistent (4, 5, 7, 912). Three studies have compared reproductive or lifestyle characteristics of initial refusers with those of initial responders (3, 7, 11). Three studies have found that while characteristics of initially refusing respondents differed from those of other respondents, the inclusion of those persons’ data had little overall effect on the results (4, 7, 11). In addition, one study found that data from initial refusers were less useful because these persons refused to answer more questions than other respondents (9). Since recruiting reluctant respondents is expensive and lengthens the time required to complete interviews, it is important to determine whether or not this additional expense is warranted.

We undertook this study to further address the question of differences in demographic, reproductive, health, and lifestyle characteristics between control respondents according to the level of effort required to recruit them.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
We combined interview data obtained from 5,616 female controls selected by random digit dialing (RDD) for six population-based case-control studies of anogenital cancers, breast cancer, endometrial cancer, thyroid cancer, oral cancer, and rheumatoid arthritis (1318). All six studies were conducted in western Washington State. All of the studies were approved by the institutional review boards of the participating institutions, and all respondents gave written informed consent. The controls were not screened for the presence of any of these diseases prior to interview. Table 1 shows the response proportions and characteristics of each study. Details on these studies are presented elsewhere (1318). All controls were selected using either unrestricted RDD or the Waksberg-Mitofsky (19) modification of RDD. Phone numbers for the unrestricted RDD were created by selecting an area code and telephone prefix at random from all such combinations in the geographic area of interest and adding a random four-digit number to create a telephone number. Studies that used the Waksberg-Mitofsky modification added a second stage of number selection. The area code, prefix, and next two digits of all telephone numbers that were found to belong to residential households in the primary stage of calling became the "primary sampling unit" for the secondary stage. Two random numbers were added to the sampling unit to create an additional phone number. This phone number was called and screened according to the study protocol. Additional phone numbers were generated and resolved until a total of "k" residential numbers were achieved for each sampling unit. This clustering factor, "k," was 2 for some studies and 5 for others.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Response proportions and characteristics of random digit dialed female controls from six studies conducted in western Washington State, 1987–1998
 
The RDD screening protocol was similar for all of the studies. Each number was called at least nine times at different times and on different days of the week over a 3-week period before it was abandoned. Residential status could not be determined for 4.3 percent of the 96,806 phone numbers dialed (3.7 percent of calls were never answered, 0.6 percent always resulted in a fast-busy tone, and 0.3 percent always resulted in a slow-busy tone). When a residential household was reached, the RDD interviewer screened the household for multiple telephone numbers, county of residence, and the ages and genders of household members. If there was more than one phone number in the household and the phone was not dedicated to a computer or fax line, the household was considered ineligible if the last digit of the phone number was odd and eligible if this digit was even.

Controls were selected using a one-step stratification design (20). Controls were selected so that each 5-year age group would have approximately the same number of controls as did the cases for that particular study. Eligible controls were asked whether they would be willing to receive a letter describing the study. If they were willing, a letter describing the study was mailed within 2 weeks of initial contact. A trained interviewer contacted the potential control after the letter was received to answer questions and to schedule an in-person interview. All phone numbers at which the respondent refused to answer the screening questions for determination of eligibility or, if screened and eligible, refused to receive a letter about the study were recontacted by a different RDD interviewer 3–6 months after the original phone call. In addition, most phone numbers at which all nine calls were answered by a machine were recontacted at a later date. The recontact included a question as to whether the respondent had had that phone number at the time of the original phone call. Approximately half of the respondents who originally either refused screening or (if eligible after screening) refused to receive a letter agreed on the recontact call. Screening was completed for almost half of the households in which all nine original phone calls had been answered by machine. Interviews were conducted in person by trained interviewers at a place and time most convenient to the respondent. The interviews lasted about an hour. Respondents for the anogenital and oral cancer studies were paid $10 for participation. Respondents for the other studies were not paid.

All respondents were assigned a "reference" year that approximated the distribution of diagnosis years in the corresponding case group. Each control was also assigned a randomly selected "reference" month. Only events that occurred prior to this reference date were recorded during the interview.

We categorized the respondents recruited by RDD into early responders (those who were interviewed within 1 month of the date the letter describing the study was mailed (n = 2,747)), intermediate responders (those who were interviewed 2–6 months after initial letter contact (n = 837)), late responders (those who were interviewed more than 6 months after initial letter contact (n = 444)), and initial refusers (those who initially refused to participate but agreed after recontact (n = 376)). All respondents who initially refused were grouped together regardless of how much time had elapsed between initial letter and interview. We excluded those who were interviewed 1–2 months after initial letter contact (n = 1,212), because some of those interviews might have been delayed as a result of interviewer schedules rather than respondent delay. Our categorization was motivated by a common assumption, described by Lin and Schaeffer (21), that the characteristics of people who are difficult to interview become more like those of refusers as the difficulty of interviewing them increases. Table 2 shows the distribution of responses by study.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Distribution of responses (%) of random digit dialed female controls from six studies conducted in western Washington State, by study and response category, 1987–1998
 
We selected questions from each interview that related to health history and to demographic, reproductive, and lifestyle characteristics. Body mass index was computed as weight in kilograms divided by height in meters squared. Data that were not collected in a similar manner across studies were excluded from analysis. The footnotes in table 3 identify variables for which data were not available for all studies. Some data were missing because of respondent refusal or because the respondent answered "don’t know."


View this table:
[in this window]
[in a new window]
 
TABLE 3. Characteristics (%) of early responders, intermediate responders, late responders, and initial refusers among random digit dialed female controls from six studies conducted in western Washington State, 1987–1998
 
The case-control study of rheumatoid arthritis included a second control group selected from the enrollment files of the Group Health Cooperative, a health maintenance organization serving western Washington State (18). Seventy-eight percent of the selected controls completed the interview. The Group Health Cooperative research department abstracted medical record data without identifiers for all 59 of the selected controls who refused to be interviewed and a random sample (n = 49) of the controls who were interviewed. The abstractors recorded only events that occurred prior to the assigned reference date. Data were recorded from the medical visits that occurred closest in time to the reference date. The following data were abstracted from the medical records: age at reference date, number of pregnancies, ever having a livebirth, ever using hormones, smoking history, height, body weight, and date on which weight was recorded.

All proportions for the intermediate responders, late responders, and initial refusers were standardized to the age distribution of the early responders so that the proportions of respondents with each characteristic could be compared with each other without distortion from the different age structures of the four groups. All variables were included simultaneously in polytomous logistic regression models with the early responders designated the reference group, using Stata statistical software (22). After adjustment for all other variables analyzed, p values were computed for individual variables using Wald’s test. Differences were considered statistically significant if the two-sided p value was less than 0.05. Results of likelihood ratio tests for inclusion of each variable in models adjusted only for age were similar to the Wald’s test statistics. Additional significance tests were performed using Bonferroni’s correction to account for multiple comparisons (23). Results of both tests are presented in the tables.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Comparison of intermediate and late responders with early responders
The intermediate- and late-responding women were slightly younger, were more likely to be non-White, and were less likely to have ever attended college than early responders (table 3). Early responders were more likely to have a family history of cancer in a first- or second-degree female relative than the intermediate- or late-responding women. Current marital status and occupation were similar for all three groups.

Women who were intermediate or late responders were similar to early responders with respect to number of pregnancies and oral contraceptive use but were less likely to have ever been tested for infertility. Late responders were less likely to have ever had an induced abortion and to have used noncontraceptive hormones than early responders, but the differences were not statistically significant.

Intermediate- and late-responding women had similar histories of hypertension, diabetes, and cancer and similar body mass indexes.

Late-responding women were more likely than early responders to be current smokers, whereas the smoking history of intermediate responders was similar to that of early responders. Ever use of alcoholic beverages and recent exercise were similar in all three groups.

Comparison of initial refusers with early responders
Initial refusers were older and were more likely to be non-White than early responders. They were less likely to be currently married, to have attended college, and to be employed in a managerial or professional job. They were also less likely than early responders to have a family history of cancer and to use noncontraceptive hormones, but these differences were not statistically significant.

Early responders and initial refusers were similar with respect to all reproductive characteristics except lifetime number of sexual partners. Initial refusers had fewer lifetime sexual partners than early responders.

Initial refusers and early responders had similar health histories, with the exception of diabetes mellitus. Initial refusers were more likely to report a history of diabetes.

Intermediate responders, late responders, and initial refusers were similar to each other and different from early responders with respect to education, non-White race, and history of infertility testing. In addition, late responders were similar to initial refusers with respect to marital status, number of sexual partners, and use of noncontraceptive hormones.

Medical record data
There were no significant differences in medical record data between women who were interviewed and those who refused to be interviewed in the Group Health Cooperative population. Refusers had a higher body mass index than those who were interviewed, but the difference was not statistically significant (table 4).


View this table:
[in this window]
[in a new window]
 
TABLE 4. Comparison (%) between interviewed female respondents and refusing respondents with regard to Group Health Cooperative medical record data, western Washington State, 1987–1991
 
There were very few questions that the respondents refused to answer or for which the response was "don’t know." The question that was most often refused was the question on income. Of the early-responding, intermediate-responding, late-responding, and initially-refusing female respondents, 76 (2.8 percent), 21 (2.5 percent), 9 (2.0 percent), and 16 (4.3 percent), respectively, refused to identify which category of income corresponded to their annual household income or responded that they did not know their household income. The item nonresponse for all other questions was less than 1 percent.


    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Our ability to find differences between the response groups was limited by the relatively small number of respondents who completed the interview more than 6 months after contact and who initially refused to be interviewed but were subsequently recruited.

We found only one other study that compared respondents who agreed to be interviewed but delayed the interview with respondents who were interviewed soon after contact (24). Robins (24) conducted in-person interviews of former clients of a child guidance clinic and found that the only difference between early and late responders was proximity to the study site. In contrast, we found several differences between late and early responders and/or intermediate and early responders. Late responders may be as willing to be interviewed as early responders but be less accessible than early respondents because of lifestyle factors and younger age.

Our finding that initial refusers were older, less educated, and less likely to be employed in professional or managerial occupations than early responders is in general agreement with most (4, 7, 11, 12, 24, 25) but not all (25) prior studies. One other study concurred with our results that respondents who initially refused were more likely to be non-White than early responders (9), whereas three others did not (4, 5, 12). Our finding that respondents who initially refused were less likely to be currently married than early responders is in contrast to the studies of both Fitzgerald and Fuller (5) and Kristal et al. (7).

The only other similar study that compared reproductive characteristics differed from our study in that only women of reproductive age who were at risk of pregnancy were included (11). Our findings are consistent with this study with respect to lifetime number of sexual partners but not gravidity or history of induced abortion. Our results did not change when we examined reproductive characteristics only among women of reproductive age.

Our finding that histories of hypertension and cancer were similar for early responders and initial refusers but that initial refusers were more likely to report a history of diabetes is consistent with another comparison of female participants and refusers (3).

Kristal et al. (7) found that initial female refusers were less likely to use alcohol and to smoke cigarettes than those who initially agreed, whereas Criqui et al. (3) found that female refusers were more likely to be current smokers than study participants in a study of heart disease. Those results are in contrast with our findings of no differences with respect to alcohol and cigarette use.

Since all of the studies except one were related to cancer, it is not surprising that early responders were much more likely to have a first- or second-degree relative who had been diagnosed with cancer than either intermediate responders, late responders, or initial refusers. This supports Massey et al.’s (26) observation that interest in the study topic improves response. A similar result was found in a study of risk factors for cardiovascular disease (3). In that study, female participants were more likely to have a family history of cardiovascular disease than nonparticipants (3). Although a less-biased group of participants may be recruited if potential respondents are not aware of the specific disease under study, response may be slower.

Although our comparison of medical record data between members of a health maintenance organization who agreed to be interviewed and members who refused to be interviewed revealed no significant differences, the small number of respondents in this group limited our ability to find differences.

Some of the differences between our study and other studies that have compared respondents who initially agreed to participate with those who initially refused may be related to the type of interview administered. Most other studies used telephone interviews or mailed questionnaires, whereas ours utilized in-person interviews. Another reason may be geographic variation or differences over time. Only three of the prior studies were conducted during the 1990s (4, 7, 9); the remainder were conducted prior to 1990. Two of the prior studies that compared respondents who initially agreed to participate with those who initially refused were carried out in the same general geographic area as our study (7, 11). Another difference between our study and prior studies is that we separated respondents who initially agreed into early and late responders, whereas most other studies combined these respondents and compared the combined group with persons who had initially refused.

We found several significant differences when intermediate responders, late responders, and initial refusers were compared with early responders. However, no clear pattern of characteristics that could be useful in generalizing these differences emerged. The only variables that showed a consistent gradient according to difficulty of recruitment were education and race. Age differences between late and early responders and initial refusers and early responders pointed in opposite directions. These findings argue against the common assumption that the characteristics of respondents become more like those of refusers as the difficulty of recruiting them increases. Lin and Schaeffer (21) and Fitzgerald and Fuller (5) reached similar conclusions.

To reduce recall bias, epidemiologic case-control studies which examine events that occurred before the diagnosis date in cases or a similar "reference" date in controls often exclude respondents who are not interviewed within a specified time period after the diagnosis/reference date. In the current study, 24.4 percent of the initial female respondents completed the interview 2 or more months after the letter describing the study was mailed (which was generally within 2 weeks of the RDD contact), and 8.5 percent completed the interview more than 6 months after original contact. Since the initial refusers were recontacted 3–6 months after the original RDD contact, all of them were interviewed 3 or more months after original contact. The reduction of recall bias must be weighed against the increase in response bias. Weighting factors that are thought to be associated with nonresponse would not adequately compensate for the loss of late responders or initial refusers, since the characteristics of these two groups appear to be different.

Initial refusers comprised only 6.7 percent of our total sample, so excluding them from any of the studies would generally have little effect on the overall distribution of the control group by the characteristics examined. Three studies have shown this to be the case (4, 7, 11). However, analyses by subgroups could be distorted, particularly if the analyses involved two characteristics that were differentially distributed by response. This potential for bias will increase if study participants become more difficult to recruit in the future and an effort is not made to include reluctant respondents (1, 3).

Characteristics of reluctant respondents may well vary over time and by geographic area, so it is difficult to quantify the potential bias resulting from exclusion of inaccessible or less-willing participants. Expending the additional effort required to convert refusers into participants and to delay interviews rather than accept refusal remains the best defense against the creation of unrepresentative or biased samples.


    ACKNOWLEDGMENTS
 
This study was funded in part by grants RO1-CA47749, RO1-CA41410, RO1-CA52656, RO1-CA48996, and PO1-CA42792 from the National Cancer Institute and by contract N01-HD-62914 with the National Institute of Child Health and Human Development.


    NOTES
 
Correspondence to Dr. Lynda Voigt, Fred Hutchinson Cancer Research Center, P.O. Box 19024, Seattle, WA 98109-1024 (e-mail: lvoigt{at}fhcrc.org). Back


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Hartge P. Raising response rates: getting to yes. Epidemiology 1999;10:105–7.[ISI][Medline]
  2. Austin MA, Criqui MH, Barrett-Connor E, et al. The effect of response bias on the odds ratio. Am J Epidemiol 1981;114:137–43.[Abstract]
  3. Criqui MH, Barrett-Connor E, Austin M. Differences between respondents and non-respondents in a population-based cardiovascular disease study. Am J Epidemiol 1978;108:367–72.[Abstract]
  4. Lavrakas PJ, Bauman SL, Merkle DM. Refusal report forms (RRFs), refusal conversions, and non-response bias. Presented at the 47th annual meeting of the American Association for Public Opinion Research, St. Petersburg, Florida, May 15–19, 1992.
  5. Fitzgerald R, Fuller L. I hear you knocking but you can’t come in. Sociol Methods Res 1982;2:3–32.
  6. Kaldenberg D, Koenig HF, Becker BW. Mail survey response rate patterns in a population of the elderly. Public Opin Q 1994;58:68–76.[Abstract]
  7. Kristal AR, White E, Davis JR, et al. Effects of enhanced calling efforts on response rates, estimates of health behavior, and costs in a telephone health survey using random-digit dialing. Public Health Rep 1993;108:372–9.[ISI][Medline]
  8. Groves RM, Lyberg LE. An overview of nonresponse issues in telephone surveys. In: Groves RM, Biemer PP, Lyberg LE, et al, eds. Telephone survey methodology. New York, NY: John Wiley and Sons, Inc, 1988:191–211.
  9. Triplett T, Blair J, Hamilton T, et al. Initial cooperators vs. converted refusers: are there response behavior differences? In: 1996 Proceedings of the Section on Survey Research Methods, American Statistical Association. Alexandria, VA: American Statistical Association, 1996:1038–41.
  10. Groves RM. Survey errors and survey costs. Probing the causes of nonresponse and efforts to reduce nonresponse. New York, NY: John Wiley and Sons, Inc, 1989:185–238.
  11. Holt VL, Daling JR, Stergachis A, et al. Results and effect of refusal recontact in a case-control study of ectopic pregnancy. Epidemiology 1991;2:375–9.[ISI][Medline]
  12. O’Neil MJ. Estimating the nonresponse bias due to refusals in telephone surveys. Public Opin Q 1979;43:218–32.[Abstract]
  13. Carter JJ, Madeleine MM, Shera K, et al. Human papillomavirus 16 and 18 L1 serology compared across anogenital cancer sites. Cancer Res 2001;61:1934–40.[Abstract/Free Full Text]
  14. Hill DA, Weiss NS, Voigt LF, et al. Endometrial cancer and patterns of intrauterine device use. Int J Cancer 1997;70:278–81.[ISI][Medline]
  15. Peacock SL, White E, Daling JR, et al. Relation between obesity and breast cancer in young women. Am J Epidemiol 1999;149:339–46.[Abstract]
  16. Schwartz SM, Daling JR, Doody DR, et al. Oral cancer risk in relation to sexual history and evidence of human papillomavirus infection. J Natl Cancer Inst 1998;90:1626–36.[Abstract/Free Full Text]
  17. Stanford JL, Weiss NS, Voigt LF, et al. Combined estrogen and progestin hormone replacement therapy in relation to risk of breast cancer in middle-aged women. JAMA 1995;274:137–42.[Abstract]
  18. Voigt LF, Koepsell TD, Nelson JL, et al. Smoking, obesity and alcohol consumption and the risk of rheumatoid arthritis. Epidemiology 1994;5:525–32.[ISI][Medline]
  19. Waksberg J. Sampling methods for random digit dialing. J Am Stat Assoc 1978;73:40–6.[ISI]
  20. Harlow BL, Davis S. Two one-step methods for household screening and interviewing using random digit dialing. Am J Epidemiol 1988;127:857–63.[Abstract]
  21. Lin IF, Schaeffer NC. Using survey participants to estimate the impact of nonparticipation. Public Opin Q 1995;59:236–58.[Abstract]
  22. Stata Corporation. Stata statistical software, release 6.0. College Station, TX: Stata Corporation, 1999.
  23. Fisher LD, van Belle G. Biostatistics: a methodology for the health sciences. New York, NY: John Wiley and Sons, Inc, 1993:611–13.
  24. Robins LN. The reluctant respondent. Public Opin Q 1963;27:276–86.[Abstract]
  25. Diehr P, Koepsell TD, Cheadle A, et al. Assessing response bias in random-digit dialing surveys: the telephone-prefix method. Stat Med 1992;11:1009–21.[ISI][Medline]
  26. Massey JT, O’Connor D, Krotki K. Response rates in random digit dialing RDD telephone surveys. In: 1997 Proceedings of the Section on Survey Research Methods, American Statistical Association. Alexandria, VA: American Statistical Association, 1997:707–12.