Do postage-stamps increase response rates to postal surveys? A randomized controlled trial

RA Harrison, D Holt and PJ Elton

Roger A Harrison, NHS Research Fellow, Directorate of Public Health, Wigan and Bolton Health Authority, Bryan House, 61 Standishgate, Wigan WN1 1AH, UK. E-mail roger_harrison{at}yahoo.com

Keywords Health surveys, randomized controlled trials, postal service, research design, public health, response rates

Accepted 25 March 2002

Postal health surveys in the general population can provide important information to assist with planning and service provision.1 High response rates are needed to ensure validity and to minimize costs from following-up non-responders.2

Only a small number of studies have examined methods to increase response rates in postal surveys. Using a post-office stamp rather than a pre-paid business reply envelope may seem more personal and encourage participants to return the questionnaire. However, results from trials evaluating this are inconsistent and limited to health professionals in North America.3–5 Consequently there is little evidence to inform the choice of reply envelopes in postal surveys of the general UK population. This was determined in the current study.

Methods

The sample consisted of 633 adults randomly selected from a health authority register in the north west of England. A computer-generated randomization list, stratifying on sex and age (18–45, 46–64, >=65 years) was used to allocate participants to receive one of two types of reply envelopes, both printed with the return address. One group (POS) received a reply envelope with a first-class post-office stamp, while the second group (PPB) received a pre-paid business-franked envelope to return their health questionnaire. The initial mail-out included the 16-page health questionnaire, a personalized letter from the Director of Public Health and one of the two return envelopes. Questionnaires were sent out in March 2001 and non-responders were sent a reminder postcard 10 days later. The primary outcome was the difference in response rates between the two types of reply envelopes, after follow-up reminders had been sent measured at 4 weeks after posting the survey. The {chi}2 test was used to examine statistical significance. Approval was obtained for the study from the Local Research Ethics Committee.

Results

Of the 633 participants, 316 (49.9%) were randomized to the POS and 317 (50.1%) to the PPB (Figure 1Go). After the initial mail-out, 128 (40.5%) and 118 (37.2%) questionnaires were returned within 10 days, from the two allocation groups, respectively. After sending reminders to the 182 non-responders in the POS group and 195 in the PPB group (excluding 6 and 4 participants respectively later found to be not living at that address), the return rate increased to 182 (58.7%) and 179 (57.2%) in the two groups. This difference of 1.5% between the two groups is unlikely to be important and was not statistically significant ({chi}2 = 0.08, P = 0.77). No important difference was observed in return rates between the POS and the PPB group for men or women and across three age groups. However, return rates overall were higher for women than for men (65.3% versus 50.5%, {chi}2 = 15.1, P = 0.01) and increased with older age (46.5% for ages 18–45 years; 65.3% for ages 46–64 years and 82.5% for ages >=65 years. {chi}2 = 41.8, P = 0.01).



View larger version (34K):
[in this window]
[in a new window]
 
Figure 1 Response rates by allocation group before and after sending the reminder postcard

 
Discussion

This study found no important effect on response rates to a population postal health questionnaire from providing a first-class post-office stamped reply envelope compared with a pre-paid business reply envelope. The study had 80% power to detect an increase of 10% between response rates in the two groups and a type II error is unlikely. The study was not designed to identify factors influencing response to the different postal methods. However, it is possible that the overall survey package has a greater effect on response rates than the method of return. Hence it is important to consider the type of information being sought and the organization seeking it. Furthermore, it is possible that the effect of different methods for returning the questionnaire will vary by the population being surveyed. In the current study, older people and women were more likely to return the questionnaires regardless of the postage method. Hence other methods may need to be employed to increase response rates among these groups.

In the current study it was not possible to compare the costs of preparing the two different types of envelopes because the pre-paid business envelopes were regularly produced in bulk for the whole organization. However, the potential economic savings from using business-franked envelopes compared with postage stamps will depend upon individual organization’s circumstances and the number of envelopes used. It is also likely to vary between different countries.


KEY MESSAGES

  • Stamped addressed envelopes had no important impact on increasing response rates compared to pre-paid business franked envelopes.
  • Methods need to be identified to increase response rates particularly in younger people and men.
  • The costs of using different postal methods will depend on the existing infrastructure and the number of envelopes to be prepared.

 

Acknowledgments

This study was carried out as part of the Health Needs Assessment programme at Wigan and Bolton Health Authority, Wigan UK. We would like to thank Terry O’Neil for his comments on the draft manuscript.

Notes

Directorate of Public Health, Wigan and Bolton Health Authority, Bryan House, 61 Standishgate, Wigan WN1 1AH, UK.

References

1 Hensher M, Fulop N. The influence of health needs assessment on healthcare decision-making in London health authorities. J Health Serv Res Policy 1994;4:90–95.

2 Abramson JH. Survey Methods in Community Medicine. 4th Edn. Edinburgh: Churchill Livingstone, 1990.

3 Streiff MB, Dundes L, Spivak JL. A mail survey of United States hematologists and oncologists: A comparison of business reply versus stamped return envelopes. J Clin Epidemiol 2001;54:430–32.[CrossRef][ISI][Medline]

4 Urban N, Anderson GL, Tseng A. Effects on response rates and costs of stamps vs business reply in a mail survey of physicians. J Clin Epidemiol 1993;46:455–59.[CrossRef][ISI][Medline]

5 Shiono PH, Klebanoff MA. The effect of two mailing strategies on the response to a survey of physicians. Am J Epidemiol 1991;134:539–42.[Abstract]