1 Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
2 Department of Public Health Sciences, University of Toronto, Toronto, Ontario, Canada.
3 Arthritis Community Research and Evaluation Unit, Toronto, Ontario, Canada.
4 Division of Rheumatology, Sunnybrook & Womens College Health Sciences Centre, Toronto, Ontario, Canada.
5 Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada.
6 Institute for Clinical and Evaluative Sciences, Toronto, Ontario, Canada.
7 Department of Family and Community Medicine, St. Michaels Hospital, Toronto, Ontario, Canada.
8 Inner City Health Research Unit, St. Michaels Hospital, Toronto, Ontario, Canada.
9 Toronto Rehabilitation Research Institute, Toronto, Ontario, Canada.
10 Division of Orthopaedic Surgery, Hospital for Sick Children, Toronto, Ontario, Canada.
11 Department of Surgery, University of Toronto, Toronto, Ontario, Canada.
Received for publication December 2, 2002; accepted for publication May 15, 2003.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
contact tracing; data collection; directories; eligibility determination; nonrespondents
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In 19951996, population-based surveys were carried out in two Ontario, Canada, communities (3, 4): Torontos Borough of East York, an urban area, and Oxford County, a mostly rural area. The sample listing used was the Ontario Ministry of Finances 1994 residential tax records, which became available to the investigators in September 1995. Eligible respondents were those aged 55 years or older living in either area at the commencement of questionnaire mailing (November 1, 1995): 26,293 persons in East York and 21,925 in Oxford County. After two follow-up mailings and attempted telephone contacts were completed, 14,082 (53.6 percent) surveys were returned from East York and 14,369 (65.5 percent) from Oxford County (table 1).
|
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Those not responding after two additional mailings were telephoned in accordance with National Center for Health Statistics methods (8). Because telephone numbers were not included in the Ministry of Finance database, current telephone directories were used to search for each nonrespondents telephone number. As shown in table 1, these follow-up efforts clarified the status of 20,842 persons (79.3 percent) in East York and 18,427 (84.0 percent) in Oxford County, including a large number identified as being either ineligible or unable to complete the survey. Eliminating these persons resulted in adjusted response rates of 63.8 percent (14,082/22,063) for East York and 72.8 percent (14,369/19,735) for Oxford County.
While a wide range of resources, such as motor vehicle, real estate, utility company, and death records, have been used to trace survey nonrespondents whereabouts (9, 10), we chose to use relevant past editions of telephone and city directories, readily available in print or microfiche at local libraries. This approach involved searching telephone directories by last name and, where applicable, city directories by street address.
Because resource constraints precluded tracing all nonresponders, 500 "unknowns" from each community were randomly selected for more extensive tracing using publicly available directories. This number was chosen so that the proportions would be estimated with a precision of at least ±5.0 percent (11), providing sufficient precision to reapply the estimated proportions to the entire group of unknowns in each area.
Tracing involved a search of each areas 1995 telephone directory to identify a listing with the last name and address of each person sought. If such a listing was found, the 1996 telephone directory was also searched. If a consistent listing was found in 1996, the person was considered a true nonresponder, having resided at the study address when the survey was mailed. If a corresponding listing within the study community could not be found for that person in the 1996 telephone directory (but was present in 1995), the person was deemed ineligible for the survey, having moved before the survey was mailed. To confirm each apparent move, the 1996 telephone directory was also examined to determine whether the person was listed at a new address.
For those communities for which city directories were available (i.e., all of East York and the city of Woodstock in Oxford County), the 1995 city directory was searched to identify the listing corresponding to the address of each person sought. If the intended recipient could not be identified by using the 1995 city directory, then early editions of the directory (back to 1989) were also searched. When a study participant was identified by using a city directory, the listing for that address in the following years edition of the city directory was reviewed. If a new occupant could be identified at the applicable study address, an attempt was made to identify his or her listing in the applicable telephone directory, thus confirming that the telephone at the address remained publicly listed, providing further evidence that the study subject had not unlisted the telephone number but had in fact actually moved.
If listings corresponding to a study participant and his or her address could not be identified in the telephone or city directories, the persons survey status remained "unknown."
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
While the 500 randomly selected East York unknowns were being traced, the status of 23 was confirmed through sources unrelated to the tracing procedures. As such, the East York sample ultimately consisted of the remaining 477 persons. The results of the tracing efforts differed for the two areas (table 2). Although similar proportions in both East York and Oxford County remained "unknown" (26.4 percent and 21.6 percent, respectively), unknowns in East York were more likely to be classified as "ineligible at baseline" (41.9 percent vs. 16.0 percent) and less likely to be classified as "true nonresponders" (31.7 percent vs. 62.4 percent).
|
Applying the proportions found "ineligible at baseline" to the total number of unknowns in each area resulted in a further 2,284 (41.9 percent of 5,451) and 560 (16.0 percent of 3,498) persons being estimated as ineligible in East York and Oxford County, respectively. These estimates resulted in readjusted response rates of 71.2 percent (14,082/19,779) for East York and 74.9 percent (14,369/19,175) for Oxford County.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The use of tracing to determine respondent eligibility has not, to our knowledge, been described previously. These efforts, while requiring significant resources to complete effectively, were worthwhile because they provided a more complete understanding of study nonresponse and, furthermore, enabled presentation of more informative, adjusted survey response rates.
This study was possible only because of the existence of and access to the 1994 Ministry of Finance database. However, it was important to assess the accuracy of the Ministry of Finance listing of those aged 55 years or older in each study area. A review of 1991 and 1996 census data (7, pp. 909 and 1377; 12, pp. 296 and 453) revealed that the numbers of persons aged 55 years or older listed in the Ministry of Finance database for East York and Oxford County (26,293 and 21,925, respectively) were similar to the corresponding census-derived estimates (26,397 and 21,975). In addition, the age-sex distributions of the persons listed in the Ministry of Finance database were also found to be similar to the census data.
Although tracing only a random subset of nonresponders is more efficient and feasible than tracing the entire sample, it does provide proportions estimated with imprecision. However, the impact of this imprecision can be evaluated by conducting sensitivity analyses. For example, in our study, applying the lower limit of the 95 percent confidence intervals for the proportion who were "ineligible at baseline" (37.5 percent for East York and 12.8 percent for Oxford County) revealed "worst-case" adjusted response rates of 70.3 percent (14,082/20,019) for East York and 74.5 percent (14,369/19,287) for Oxford Countyboth satisfying the recommended minimum survey response rate of 70 percent (1).
The ability to complete tracing efforts such as these is greatly aided by sample lists that provide accurate, complete, and thorough baseline information concerning each person in the study sample. In addition, such efforts are possible only when publicly available telephone and city directories are availableideally, for all years and persons studied. The importance of detailed address information and available city directories is highlighted by data from Oxford County, where the overall mobility rate (16.0 percent) was lower than the rate for those with detailed addresses and for whom city directories were available (42.2 percent). Without detailed addresses and available city directories for all of Oxford County, it was impossible to determine whether this difference was real or simply the result of having less information for 344 of the Oxford County unknowns.
Because moving is only one reason for someone no longer being listed in a telephone directory, alternative explanations were also examined. First, in areas served by more than one telephone provider, people may have simply changed providers. However, during the time of this survey, Bell Canada held a monopoly to provide residential telephone services. In addition, while cellular telephones were in use in 1996, we expect that very few study subjects (i.e., those aged 55 years or older) would have replaced their residential telephone with cellular service.
A second reason that persons may no longer be listed in the telephone directory is if their listing is taken over by another family member. In this situation, however, the telephone number would be expected to remain unchanged to avoid telephone disconnection and connection fees. Our tracing efforts did not identify any such listing name changes.
A third reason that persons may no longer be listed in the telephone directory is if they change to an unlisted number. Several pieces of evidence argue against such an explanation in this study, however. First, according to city directory listings, the traced "unknowns" had been listed publicly at their address for an average of 8 years. Therefore, it is unlikely that many of these persons would have simultaneously switched to unlisted numbers en mass in 1996. Second, while telephone contacts resulted in the completion of a large number of surveys (table 1), no recorded messages identifying the "number unlisted at the subscribers request" were encountered during this follow-up process. In addition, for 90 percent of the unknowns, either a telephone number could not be located or, when telephone contact was attempted, the number was found to be either out of service or incorrect. Third, 140 of the tracing-identified moves were confirmed by identifying either a new address for the study subject or the telephone listing of a new occupant at the study address: 45/70 (64.3 percent) in Oxford Countys city of Woodstock and 95/200 (47.5 percent) in East York. Because of less specific city directory listings for apartments, a larger proportion of moves from detached homes were confirmed in both areas: 26/30 (86.7 percent) in Woodstock and 40/56 (71.4 percent) in East York.
Given sufficient person-power, these tracing efforts would be more ideally carried out in "real-time," with data collection still under way, rather than post hoc to better understand and clarify the surveys response rate. Of course, for this study, that would have involved tracing nearly 9,000 persons instead of just the 977 who were selected randomly.
Despite requiring a significant investment of resources, tracing efforts can provide, as demonstrated by this study, valuable information concerning a survey, particularly quantifying deficiencies in the sample lists caused by inclusion of ineligible persons. This information then provides a basis for estimating readjusted response rates and serves to strengthen understanding of and confidence in the surveys results.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|