Roger A Harrison, NHS Research Fellow, Directorate of Public Health, Wigan and Bolton Health Authority, Bryan House, 61 Standishgate, Wigan WN1 1AH, UK. E-mail roger_harrison{at}yahoo.com
Keywords Health surveys, randomized controlled trials, postal service, research design, public health, response rates
Accepted 25 March 2002
Postal health surveys in the general population can provide important information to assist with planning and service provision.1 High response rates are needed to ensure validity and to minimize costs from following-up non-responders.2
Only a small number of studies have examined methods to increase response rates in postal surveys. Using a post-office stamp rather than a pre-paid business reply envelope may seem more personal and encourage participants to return the questionnaire. However, results from trials evaluating this are inconsistent and limited to health professionals in North America.35 Consequently there is little evidence to inform the choice of reply envelopes in postal surveys of the general UK population. This was determined in the current study.
Methods
The sample consisted of 633 adults randomly selected from a health authority register in the north west of England. A computer-generated randomization list, stratifying on sex and age (1845, 4664, 65 years) was used to allocate participants to receive one of two types of reply envelopes, both printed with the return address. One group (POS) received a reply envelope with a first-class post-office stamp, while the second group (PPB) received a pre-paid business-franked envelope to return their health questionnaire. The initial mail-out included the 16-page health questionnaire, a personalized letter from the Director of Public Health and one of the two return envelopes. Questionnaires were sent out in March 2001 and non-responders were sent a reminder postcard 10 days later. The primary outcome was the difference in response rates between the two types of reply envelopes, after follow-up reminders had been sent measured at 4 weeks after posting the survey. The
2 test was used to examine statistical significance. Approval was obtained for the study from the Local Research Ethics Committee.
Results
Of the 633 participants, 316 (49.9%) were randomized to the POS and 317 (50.1%) to the PPB (Figure 1). After the initial mail-out, 128 (40.5%) and 118 (37.2%) questionnaires were returned within 10 days, from the two allocation groups, respectively. After sending reminders to the 182 non-responders in the POS group and 195 in the PPB group (excluding 6 and 4 participants respectively later found to be not living at that address), the return rate increased to 182 (58.7%) and 179 (57.2%) in the two groups. This difference of 1.5% between the two groups is unlikely to be important and was not statistically significant (
2 = 0.08, P = 0.77). No important difference was observed in return rates between the POS and the PPB group for men or women and across three age groups. However, return rates overall were higher for women than for men (65.3% versus 50.5%,
2 = 15.1, P = 0.01) and increased with older age (46.5% for ages 1845 years; 65.3% for ages 4664 years and 82.5% for ages
65 years.
2 = 41.8, P = 0.01).
|
This study found no important effect on response rates to a population postal health questionnaire from providing a first-class post-office stamped reply envelope compared with a pre-paid business reply envelope. The study had 80% power to detect an increase of 10% between response rates in the two groups and a type II error is unlikely. The study was not designed to identify factors influencing response to the different postal methods. However, it is possible that the overall survey package has a greater effect on response rates than the method of return. Hence it is important to consider the type of information being sought and the organization seeking it. Furthermore, it is possible that the effect of different methods for returning the questionnaire will vary by the population being surveyed. In the current study, older people and women were more likely to return the questionnaires regardless of the postage method. Hence other methods may need to be employed to increase response rates among these groups.
In the current study it was not possible to compare the costs of preparing the two different types of envelopes because the pre-paid business envelopes were regularly produced in bulk for the whole organization. However, the potential economic savings from using business-franked envelopes compared with postage stamps will depend upon individual organizations circumstances and the number of envelopes used. It is also likely to vary between different countries.
KEY MESSAGES
|
Acknowledgments
This study was carried out as part of the Health Needs Assessment programme at Wigan and Bolton Health Authority, Wigan UK. We would like to thank Terry ONeil for his comments on the draft manuscript.
Notes
Directorate of Public Health, Wigan and Bolton Health Authority, Bryan House, 61 Standishgate, Wigan WN1 1AH, UK.
References
1 Hensher M, Fulop N. The influence of health needs assessment on healthcare decision-making in London health authorities. J Health Serv Res Policy 1994;4:9095.
2 Abramson JH. Survey Methods in Community Medicine. 4th Edn. Edinburgh: Churchill Livingstone, 1990.
3 Streiff MB, Dundes L, Spivak JL. A mail survey of United States hematologists and oncologists: A comparison of business reply versus stamped return envelopes. J Clin Epidemiol 2001;54:43032.[CrossRef][ISI][Medline]
4 Urban N, Anderson GL, Tseng A. Effects on response rates and costs of stamps vs business reply in a mail survey of physicians. J Clin Epidemiol 1993;46:45559.[CrossRef][ISI][Medline]
5 Shiono PH, Klebanoff MA. The effect of two mailing strategies on the response to a survey of physicians. Am J Epidemiol 1991;134:53942.[Abstract]