1 Departments of Clinical Epidemiology and Biostatistics, and Orthopaedic Surgery, McMaster University, Hamilton, Ontario, Canada.
2 University of Minnesota, Minneapolis, MN, USA.
3 St Michaels Hospital, Toronto, Ontario, Canada.
Correspondence: Mohit Bhandari, McMaster University Health Sciences Center, Department of Clinical Epidemiology and Biostatistics, Room 2C3, 1200 Main Street West, Hamilton, Ontario, L8N 3Z5, Canada. E-mail: bhandari{at}sympatico.ca
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods Using focus groups, key informants, and sampling to redundancy techniques, we developed a questionnaire of surgeons preferences in the treatment of tibial shaft fractures. Twenty-two well-respected and widely known orthopaedic traumatologists endorsed the questionnaire. We randomized 395 surgeon members of the Orthopaedic Trauma Association to receive either a questionnaire that included a letter informing them of the opinion leaders endorsement, or a questionnaire without the endorsement.
Results Surgeons who received the letter of endorsement had a significantly lower response rate at 2, 4, and 8 weeks. The absolute difference in response rates was 7.8% (4.6% versus 12.4%, P < 0.05) at 2 weeks, 13.1% at 4 weeks (28.6% versus 41.7% P < 0.02), and 12.3% at 8 weeks (47.5% versus 59.8% P = 0.02).
Conclusions The addition of a letter listing expert surgeons who endorse the survey lead to significantly lower primary response rates. Those interested in influencing physician responses cannot always assume a positive effect from endorsement by opinion leaders
Accepted 10 February 2002
Mailed surveys represent a useful tool to study physicians beliefs, attitudes, and concerns in health care settings. To minimize the risk of non-responder bias, any survey must achieve the highest possible response rate. Response rates following the first mailing of questionnaires in health care research have averaged 62% (SD = 21).1 However, physicians tend to produce lower mean response rates54% (SD = 17)than other health workers.1
Strategies to improve response rates such as pre-notification letters, faxing, personalized cover letters, limiting questionnaire length, monetary incentives, and the use of university envelopes have achieved varying success.2 Investigators have attributed the huge variability in responses to increasing physician practice workloads, placing questionnaire completion at low priority. This has been especially true in surveys of surgeons who have responded at rates from 15% to 27%.35
Opinion leaders in their surgical speciality are those surgeons nominated by their colleagues as educationally influential.6 Opinion leaders have been shown to have significant influence on the practice of health professionals and patient outcomes.7 However, the effect is not consistent, and it is not always clear in which circumstances opinion leaders are likely to influence the practice of their peers.811
We hypothesized endorsement by opinion leaders in orthopaedic surgery would increase primary response rates among surgeons. We therefore examined the effect of opinion leader endorsement in a mailed questionnaire of fracture care.
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() |
---|
Identification of opinion leaders
We identified 22 opinion leaders in orthopaedic trauma with the following criteria: (1) moderated or chaired an educational session on aspects of trauma care at an international meeting, (2) invited speaker on orthopaedic trauma at an international orthopaedic meeting, and (3) published at least three peer-reviewed papers in trauma within the last 4 years.
We asked opinion leaders to complete the questionnaire and to agree to have their name placed on a list of surgeons who endorsed the survey as an important study. All 22 agreed. We constructed a letter, on a single coloured sheet of paper, that included the following statement in large, bold font: Orthopaedic traumatologists who have already completed the questionnaire and endorse it as an important study. Following the statement was a list that included the name of the surgeons and their city, state or province, and country.
Questionnaire administration
We used a computerized random number generator to randomize all 395 surgeon members of the Orthopaedic Trauma Association (OTA) to receive either a questionnaire with the endorsement list of 22 opinion leaders, or a questionnaire without the list. One of us coded packages to be sent to OTA surgeon members as A or B. An independent investigator, unaware of the surgeons allocated intervention, mailed the packages which corresponded with the letter (A or B) beside the surgeon members name. The mailed questionnaires included a personalized cover letter that ensured confidentiality of surgeons responses and a stamped return envelope. We did not include a monetary incentive during survey administration. No additional interventions to improve primary response rates were initiated prior to 8 weeks from the initial mailing. We obtained institutional review board and ethics approval for this study. At 8 weeks, one of us, blinded to surgeon intervention group, tabulated the questionnaires received in each group.
Statistical analysis
We summarized response rates by the proportion of respondents at each time point. Chi-square or Fishers Exact tests were used to compare the proportion of respondents between groups. We calculated all proportions utilizing 395 as the denominator, thereby including those surgeons who were unavailable at the time of the survey (labelled as non-respondents). We calculated the minimum response rate, i.e. the number of returned questionnaires divided by the number of questionnaires (returned and non-returned). All statistical tests were two-tailed.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() |
---|
In this study, we evaluated the influence of orthopaedic opinion leaders endorsement in improving survey response rates. Contrary to our hypothesis, we found a significantly lower response rate among those surgeons who received the endorsement letter.
Our finding is surprising. Both the theory of diffusion of innovations and the social influences model of behaviour change suggest that using local opinion leaders to transmit norms and model appropriate behaviour may improve health professional practice.9,17 Thomson and colleagues, in a systematic review of randomized trials, identified eight studies involving more than 296 health professionals to assess the effects of local opinion leaders on the practice of health professionals or patient outcomes.7 The studies targeted a variety of patient problems, including acute myocardial infarction, cancer pain, osteoarthritis, rheumatoid arthritis, chronic lung disease, vaginal birth after cesarean section, labour and delivery, and urinary catheter care. Six of seven trials that measured health professional practice demonstrated some improvement in at least one outcome variable, and in two trials, the results were statistically significant and clinically important. Of the three trials that measured patient outcomes, only one achieved an impact upon practice that was of practical importance: local opinion leaders were effective in improving the rate of vaginal birth after previous cesarean section.
In an effort to identify the reasons why physicians do not respond to questionnaires, Florttorp and colleagues conducted a qualitative study of focus groups with physicians who had previously not responded to a questionnaire. General practitioners who did not respond to questionnaires expressed a negative attitude towards superspecialists, who were perceived as arrogant and disrespectful.18 If we extrapolate this findings to surgeons, it is plausible that the respondents perceived those on the endorsement list as superspecialist orthopaedic traumatologists.
Our results challenge the assumption that those interested in influencing physician behaviour can always assume a positive, or at least a neutral, effect of interventions utilizing opinion leaders. However, it remains unknown whether these results are generalizable to surveys that utilize additional methods to reduce non-responder bias (monetary incentives). Further research is required to delineate the circumstances in which opinion leaders positively influence physician behaviour, and circumstances in which they have no effect, or a detrimental effect.
KEY MESSAGES
|
![]() |
Acknowledgments |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Kellerman Scott E, Herold J. Physician response to surveys: A review of the literature. Am J Prev Med 2001;20:6167.[Medline]
3 Matarasso A, Elkwood A, Rankin M, Elkowitz M. National plastic surgery survey: face lift techniques and complications. Plast Reconstr Surg 2000;106:118595.[CrossRef][ISI][Medline]
4 Almeida OD Jr. Current state of office laparoscopic surgery. J Am Assoc Gynecol Laparosc 2000;7:54546.[ISI][Medline]
5 Khalily C, Behnke S, Seligson D. Treatment of closed tibial shaft fractures: a survey from the 1997 orthopaedic trauma association and osteosynthesis international-Gerhard Kuntscher Kreis Meeting. J Orthop Trauma 2000;14:57781.[CrossRef][ISI][Medline]
6 Hiss RG, MacDonald R, David WR. Identification of Physician Educational Influentials in Small Community Hospitals. Washington DC: Proceedings of the 17th Annual Conference on Research in Medical Education, 1978, pp. 28388.
7 Thomson OBrien MA, Oxman AD, Haynes RB, Davis DA, Freemantle N, Harvey EL. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2000;2:CD000125.[Medline]
8 Hong SW, Ching TY, Fung JP, Seto WL. The employment of ward opinion leaders for continuing education in the hospital. Med Teach 1990;12:20917.[ISI][Medline]
9 Lomas J, Enkin M, Anderson GM, Hannah WJ, Vayda E, Singer J. Opinion leaders vs audit feedback to implement practice guidelines. Delivery after previous cesarean section. JAMA 1991;265:220207.[Abstract]
10 Soumarai SB, McLaughlin TJ, Gurwitz JH, Guadagnoli E, Hauptman PJ, Borbas C. Effect of local medical opinion leaders on the quality of care for acute myocardial infarction. A randomized controlled trial. JAMA 1998;279:135863.
11 Stross JK, Bole GG. Evaluation of an educational program for primary care practictioners on the management of osteoarthritis. Arthritis Rheum 1985;28:10811.[ISI][Medline]
12 Feinstein AR. The theory and evaluation of sensibility. In: Feinstein AR (ed.). Clinimetrics. New Haven, CT: Yale University Press, 1987.
13 Shiono PH, Klebanoff MA. The effect of two mailing strategies on the response to a survey of physicians. Am J Epidemiol 1991;134: 53942.[Abstract]
14 Cartwright A. Professional as responders: variations in and effects of response rates to questionnaires. BMJ 1978;21:41921.
15 Perneger TV, Etter JF, Rougemont A. Randomized trial of use of a monetary incentive and a reminder card to increase the response rate to a mailed health survey. Am J Epidemiol 1993;138:71422.[Abstract]
16 Kaner EF, Haighton CA, McAvoy BR. So much post, so busy with practiceso, no time!: a telephone survey of general practictioners reasons for not participating in postal questionnaire surveys. Br J Gen Pract 1998;48:106769.[ISI][Medline]
17 Greer AL. The state of the art versus the state of the science: the diffusion of new medical technologies into practice. Int J Tech Assess Health Care 1988;4:526.
18 Flottorp S, Oxman A, Bjorndal A. The limits of leadership: opinion leaders in general practice. J Health Serv Res Policy 1998;3: 197202.[Medline]