Effect of timing on the response to postal questionnaires concerning satisfaction with anaesthesia care{dagger},{ddagger}

D. Saal1,*, M. Nuebling2, Y. Husemann3 and T. Heidegger1

1 Department of Anaesthesiology, St Gallen Cantonal Hospital, CH-9007 St Gallen, Switzerland. 2 Empirical Consulting, Hauptstrasse 67.1, D-79211 Denzlingen, Germany. 3 Picker Institut, Ibelweg 18a, CH-6300 Zug, Switzerland

* Corresponding author. E-mail: detlef.saal{at}kssg.ch

Accepted for publication September 17, 2004.


    Abstract
 Top
 Footnotes
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Background. There is little information on the effect of time on the assessment by the patient of quality of anaesthesia care. This study compared the patient's assessment of anaesthesia care after three different periods of time following discharge from hospital.

Materials. Three groups of patients were assigned to receive a standardized, validated psychometric questionnaire either 1, 5, or 9 weeks after discharge from hospital. We measured response rate and the total mean problem score of six dimensions.

Results. Groups 1, 2, and 3 received 748, 743, and 723 questionnaires, respectively. The response rates including one reminder were 67.3 (95% confidence interval [CI] 63.9–70.6%), 64.5% (CI 61.1–67.9%), and 58.9% (CI 55.5–62.4%), respectively (Group 1 vs Group 3, P<0.001, and Group 2 vs Group 3, P<0.05). The total mean problem scores were not significantly different with 17 (CI 1.4%), 17 (CI 1.4%), and 15% (CI 1.3%), respectively. In two out of six dimensions (‘Continuity of personal care by anaesthetist’ and ‘Nursing care in recovery room’) significantly less problems were reported after 9 weeks. The other dimensions of the questionnaire showed no consistent differences between groups.

Conclusions. The response rate is significantly lower at 9 weeks compared with 1 and 5 weeks after discharge. The total mean problem score remains unchanged but certain fields show fewer problems after 9 weeks compared with 1 and 5 weeks. Questionnaires on patient satisfaction with anaesthesia care should be sent within 5 weeks of discharge.

Keywords: anaesthesia, audit, mailed survey questionnaire ; anaesthesia, quality management ; patient satisfaction


    Introduction
 Top
 Footnotes
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Patient satisfaction is a mainstay of quality of care.15 Health and patient satisfaction are the ultimate outcomes of quality of care. Questionnaires have been shown to be suitable for evaluation and comparison of patient satisfaction ratings.1 2 610 Psychometrically developed questionnaires on patient satisfaction with anaesthesia care are rare11 and require further refinement with regard to validity, reliability, and confounding variables.12

Little is known about the impact of different follow-up times.10 13

If regular reliable assessment of care is necessary, the hospital must send questionnaires to patients at defined intervals between discharge from hospital.13 However, instruments designed to check the quality of patient care are efficient if they are applied at the best time to survey patients, and at a time that elicits the highest response rate. We therefore evaluated the influence of time after discharge from hospital on response rate and patient satisfaction with anaesthesia care with a validated questionnaire.


    Methods
 Top
 Footnotes
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Following approval by the St Gallen Cantonal Hospital Ethics Committee, all in-patients admitted for elective surgery over a 15-week period (January–May 2003) were assigned to one of three groups who were sent the questionnaire described below by post, either 1, 5, or 9 weeks after discharge. No day surgery patients were included as this only represented a small proportion of the whole population and we wanted to compare our results with the previous studies.

All patients were treated at St Gallen Cantonal Hospital, a tertiary-care institution and were more than 15 yr, undergoing elective surgery and to be discharged to their home. Patients also had to be resident in Switzerland.

The questionnaire used was developed by the Picker Institut in collaboration with six hospitals in Switzerland and Austria.11 It was used in pilot studies in April 2000, September 2000–January 2001 and in 2002.14 During these stages, it was validated and tested for reliability. The content of the questionnaire was developed by an expert team and four focus groups of patients led by a psychologist. After factor and reliability analysis, the items were grouped into six dimensions: ‘Information/involvement in decision-making’, ‘Respect/confidence’, ‘Delays’, ‘Nursing care in recovery room’, ‘Continuity of personal care by anaesthetist’ and ‘Pain management’. The questionnaire contains 55 questions. Participants confirmed that it is easy to read and fill out.

The problem score for a single item is calculated as the percentage of persons stating a deficit for this particular aspect. The problem score of a dimension (group of items) is the proportion of problems mentioned for all relevant questions in the dimension concerned: 29 problem score questions are embedded in the questionnaire.

Analysis of the above studies showed that the dimensions ‘Information/involvement in decision-making’ and ‘Continuity of personal care by anaesthetist’ are the most important, with beta coefficients of 0.60 and 0.27, respectively.11 These dimensions consistently showed the highest problem scores in the surveys conducted between 2000 and 2003.11 14

Three different mailing intervals were to be compared. Based on the last study with this questionnaire in 2002 at St Gallen Cantonal Hospital, we expected a response rate of between 55 and 60%. The total problem score as a measure of dissatisfaction was 16%, with a 95% confidence interval (CI) of ±1. Accordingly, each group would require a minimum of 300 completed questionnaires to show differences of ±12% in response rate and ±3.5% in the total problem score with a power of at least 0.8. This would mean sending out about 600 questionnaires to each group to achieve the expected response. According to the admission rate at our hospital, we calculated that we would need 15 weeks to obtain the desired numbers.

We studied patients admitted for elective surgery in a 15-week period between January 31, 2003 and May 22, 2003. Allocation was adjusted to balance the groups and is explained below.

Patients were grouped week-wise (for practical reasons) with the following pattern.

In the first cycle, the questionnaire was sent to Group 3, 9 weeks after discharge (minimum 60 days, maximum 66 days), to Group 2, 5 weeks after discharge (minimum 32, maximum 38 days), and to Group 1, 1 week after discharge (minimum 4 days, maximum 10 days).

The groups changed weekly, under the assumption that the surgical programme across all specialities did not change significantly over the trial period. We also assumed that an interval of 3 weeks until repetition of the same group would not be influenced by seasonal or other changes. The cycle was repeated four times, and was designed to end with the last group being mailed 1 week after discharge to keep the mailing period for the whole study as short as possible. If no response from the patient was received after 2 weeks, a reminder containing an identical questionnaire (on yellow instead of white paper) was mailed once.

The questionnaires were sent out by St Gallen Cantonal Hospital using a mailing list of all enrolled patients. The questionnaires contained a follow-up number. They were accompanied by a covering letter, which explained the purpose of the study and assured anonymity. A prepaid envelope was provided for return of the completed questionnaire to the Picker Institut, who conducted the analysis.

Patients were not informed about the study while in hospital to exclude the bias of knowing ahead that they will be asked about their opinion later. We wanted patients to answer our questionnaire at a certain date to be able to discriminate between the different time intervals for the analysis.

The statistical analysis was conducted anonymously. The Picker Institut was supplied only with anonymous patient characteristics from the hospital, together with the follow-up number. The hospital was supplied only with the aggregated results of the statistical analysis without follow-up numbers, so that identification of patients was not possible. Besides descriptive techniques, the statistical analysis included analysis of variance (ANOVA) with one or more factors, multiple comparisons of means and multiple logistic regression analysis. SPSS 10 was used for all analyses.


    Results
 Top
 Footnotes
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
A total of 2214 questionnaires were sent over the 15-week study period. The patients were evenly distributed across the surgical specialities of our hospital (general surgery, orthopaedics, urology, ophthalmology, ear-nose-throat, neurosurgery, and gynaecology). Age, sex, type of anaesthesia, and ASA (American Society of Anesthesiologists) classes I–IV were also evenly distributed (Table 1). Group 2 (5 weeks) contained slightly more privately insured patients. The median length of stay was 6 days with a range of 1–104 days but 98% of all patients stayed between 1 and 31 days in hospital. The groups did not differ significantly in this respect with a median stay of 6 days in Group 1, 7 days in Group 2, and 6 days in Group 3. 97.6, 97.6, and 98.7% stayed in hospital for the period of 1–31 days, respectively.


View this table:
[in this window]
[in a new window]
 
Table 1 Characteristics of patients of the three groups in the study

 
748, 743, and 723 questionnaires were sent out to Groups 1, 2, and 3, respectively. The response rate, including the reminder, was 67, 65, and 59%, respectively. Patients younger than 30 yr of age responded significantly less than all the other age groups (Table 2). The differences between Groups 1 and 2 (P<0.001), and Groups 2 and 3 (P<0.05) were significant, both before and after adjustment for the confounding variables age, ASA class, and insurance status. Duration of stay showed an even effect on response rate in all three groups without significant differences between groups. Generally patients with a stay of less than 3 days responded less (59%) than patients who stayed longer: 4–6 (64%), 7–11 (67%), and more than 12 (64%). However, if broken down to single days, the distribution is uneven. The subsets of groups are too small for adequate analysis.


View this table:
[in this window]
[in a new window]
 
Table 2 Response rate and patient characteristics

 
The response was lower from patients in ASA class IV. This was also the case for state-insured patients as compared with private patients with a response rate of 59 and 75% each.

The multivariate estimation model on participation as the dependent variable showed that the following were significant predictors for a lower response rate: the 9-week group, ASA class IV, and patients with state insurance.

The time between mailing of the questionnaire and receiving a response did not differ between groups. The means for Group 1, 2, and 3 were 12.4 days (SD 12.6), 10.7 days (8.1), and 11.8 days (10.4), respectively. Overall, 46.4% of patients responded to the first questionnaire, and 17.1% to the reminder. There were no significant differences between groups as for the percentage answering after the first and the reminder questionnaire, with 72.7, 72.6, and 74% of Groups 1, 2, and 3, respectively, replying to the first questionnaire. The time pattern between mailing of the reminder questionnaire and response was similar to that of the first mailing.

The total problem score was 17 (95% CI 1.4%), 17 (95% CI 1.4%), and 15% (95% CI 1.3%) for Groups 1, 2, and 3 respectively. There were no significant differences (Fig. 1). Two out of the six dimensions showed a significant reduction of reported problems after 9 weeks: ‘Continuity of personal care by anaesthetist’ (Group 1 vs 3, P<0.05) and ‘Nursing care in recovery room’ (Group 1 vs 3, and 2 vs 3, both P<0.05).



View larger version (23K):
[in this window]
[in a new window]
 
Fig 1 Total problem score of Groups 1–3 and distribution of the dimensions' problem scores.

 
Differences between the other dimensions were inconsistent and not significant.


    Discussion
 Top
 Footnotes
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
The response rate in this patient sample was similar at 1 and 5 weeks but decreased by the ninth week. This pattern has not been reported elsewhere.10 13 15 16 The total mean problem score remained unchanged. A lower problem score in the fields ‘Continuity of personal care by anaesthetist’ and ‘Nursing care in recovery room’ after 9 weeks, can be interpreted as a less critical attitude. Ware18 also found less critical appraisal with time in his study of a general medical population. Other studies have found the opposite,3 13 17 19 and reported a more critical attitude, with less satisfaction, as time passes. Ley19 described a curvilinear relationship with higher satisfaction rates 1 and 8 weeks after discharge and a lower degree of satisfaction in the intervening period (2 and 4 weeks).

Once surgery is over, the emotional importance of the entire experience of hospitalization diminishes with time as daily life re-establishes itself, which could explain a less critical attitude.1 10 18 This may also result in the lower response rate we noted, which was also described by Freise.13

We appreciate that different settings could be the reason for contradictory results. In our interpretation, less emotional participation seems to be a major factor in response to questionnaires, which is reflected in our study, but further studies are needed. It is worth noting that the response rate in patients less than 30 yr was markedly lower as has been observed in other studies.20 21 The response from privately insured patients was significantly higher than from state-insured patients.

Whilst the response rate and description of certain problems in some areas were lower 9 weeks after discharge, the differences between 1 and 5 weeks were very small.

The importance of these results is that between 1 and 5 weeks there is no loss of critical appraisal and a consistent response rate. This greatly facilitates quality assurance by allowing group-mailings every 5 weeks instead of mailing on a daily basis while retaining reproducible and valid responses.

There are few systematic studies with validated instruments in quality management.21 Likewise, the time factor in surveys seems poorly assessed. We consider it an important step to prove with sufficient patient numbers and a valid instrument how timing of mailing questionnaires affects response.

We know from this and the previous surveys that ‘Information/involvement in decision-making’ and ‘Continuity of care by anaesthetist’ show the most important deficiencies.11 14 By improving them, an increase in general satisfaction with anaesthesia care should be feasible. A recent study showed that improving anaesthetists' communication skills might increase patient satisfaction, although the questionnaire used had not been validated.22

Nevertheless, we think the use of a questionnaire helps to identify problems existing in anaesthesia care. Previous studies have shown that there are substantial differences between individual hospitals and this is reflected in the different patterns within the six fields of our questionnaire.11 14

Our response rates showed that about one third of patients did not respond after discharge, which is within the usual range with both recruitment and collection by mail.21 Ley reported that patients replying to postal questionnaires were not more likely to be satisfied than those who did not reply.19 However, another study suggested that patients who were more satisfied were less likely to respond.18 In a review of the subject, eight out of 14 studies reported no difference.21

As the majority of our non-respondents were younger patients, there might be a loss of critical appraisal, as is known from other studies.23

There is no strict correlation between contact with anaesthesia and the receipt of the questionnaire at home. However, mailed questionnaires are a well-established and widely applied technique. Patients were therefore under no obligation at all to respond to the questionnaire.

The median length of stay was 6 days. The quality of care should have been remembered fairly well until receipt of the questionnaire after discharge. The duration of stay was distributed evenly with no significant differences between the three groups but the influence of the duration of stay in hospital on response remains unknown for our population.

In conclusion, a good response to a questionnaire on satisfaction with anaesthesia care can be expected from in-patients with our population profile if the questionnaire is sent to them within 1–5 weeks following discharge from hospital. This suggests that group mailings of questionnaires from up to 5 weeks from discharge are possible without loss of response, which would facilitate quality assurance.


    Acknowledgments
 
The authors thank our computer specialist Robert Schöpflin for his organizational help in selecting eligible patients and our secretary Jeanette Willi for the organization and mailing of the questionnaires. Our study was funded by the Department of Anaesthesiology and the Quality Committee of St Gallen Cantonal Hospital, Switzerland. We state that we were independent from other funding.


    Footnotes
 Top
 Footnotes
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
{dagger} A preliminary account of our results was given in a published abstract at the Euroanaesthesia Congress at Lisbon on the 5–8 June, 2004. Back

{ddagger} Declaration of interest. The questionnaire used in this study has been supplied by the Picker Institut. It has been paid for by our funding as mentioned in the Acknowledgements. Back


    References
 Top
 Footnotes
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
1 Wüthrich-Schneider E. Patientenzufriedenheit—Wie messen? Schweizerische Aerztezeitung 2000; 81: 1116–9

2 Fung D, Cohen MM. Measuring patient satisfaction with anaesthesia care: a review of current methodology. Anesth Analg 1998; 87: 1089–98[CrossRef][ISI][Medline]

3 Carr-Hill RA. The measurement of patient satisfaction. J Pub Health Med 1992; 14: 236–49[Abstract]

4 Tong D, Chung F, Wong D. Predictive factors in global and anesthesia satisfaction in ambulatory surgical patients. Anesthesiology 1997; 87: 856–64[CrossRef][ISI][Medline]

5 Donabedian A. The role of outcomes in quality assessment and assurance. Qual Rev Bull 1992; 11: 356–60

6 Walker AH, Restuccia JD. Obtaining information on patient satisfaction with hospital care: mail versus telephone. Health Serv Res 1984; 19: 291–306[ISI][Medline]

7 Roberts JG, Tugwell P. Comparison of questionnaires determining patient satisfaction with medical care. Health Serv Res 1987; 22: 637–54[ISI][Medline]

8 Cleary PD. The increasing importance of patient surveys. Br Med J 1999; 319: 720–1[Free Full Text]

9 Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to Their Development and Use, 2nd edn. Oxford: Oxford University Press, 1995

10 Kosinski A, Raspe H. Patientenzufriedenheit nach einer stationären rehabilitation: Unterschiedliche ergebnisse bei unterschiedlichen befragungszeitpunkten? Gesundheitswesen 1998; 60: 75–9[Medline]

11 Heidegger T, Husemann Y, Nuebling M, et al. Patient satisfaction with anaesthesia care: development of a psychometric questionnaire and benchmarking among six hospitals in Switzerland and Austria. Br J Anaesth 2002; 89: 863–72[Abstract/Free Full Text]

12 Le May S, Hardy JF, Taillefer MC, Dupuis G. Patient satisfaction with anesthesia services. Can J Anesth 2001; 48: 153–61[Abstract/Free Full Text]

13 Freise DC. Die Höhe des Rücklaufs in Abhängigkeit vom Befragungszeitpunkt. In: Freise DC, eds. Teilnahme und Methodik bei Patientenbefragungen. Sankt Augustin: Asgard-Verlag, 2003; 109–19

14 Heidegger T, Nuebling M, Germann R, et al. Patient satisfaction with anesthesia care: information alone does not lead to improvement. Can J Anaesth 2004; 51: 801–5[Abstract/Free Full Text]

15 Wren GR, Longest BB Jr, Keith JH, Walker JR. Patient opinion polls. Hospitals 1971; 45: 80

16 Bredart A, Razavi D, Robertson C, et al. Timing of patient satisfaction assessment: effect on questionnaire acceptability, completeness of data, reliability and variability of scores. Patient Educ Couns 2002; 46: 131–6[CrossRef][ISI][Medline]

17 Bendall-Lyon D, Powers TL, Swan JE. Time does not heal all wounds. Patients report lower satisfaction levels as time goes by. Mark Health Serv 2001; 21: 10–4[Medline]

18 Ware JE Jr, Snyder MK, Wright WR, Davies AR. Defining and measuring patient satisfaction with medical care. Eval Program Plann 1983; 6: 247–63[CrossRef][Medline]

19 Ley P, Bradshaw PW, Kincey JA, Atherton ST. Increasing patients' satisfaction with communications. Br J Soc Clin Psychol 1976; 15: 403–13[ISI][Medline]

20 Ronckers C, Land C, Hayes R, Verduijn P, van Leeuwen F. Factors impacting questionnaire response in a Dutch retrospective cohort study. Ann Epidemiol 2004; 14: 66–72[CrossRef][ISI][Medline]

21 Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health Care 1998; 10: 311–17[Abstract/Free Full Text]

22 Harms C, Young JR, Amsler F, Zettler C, Scheidegger D, Kindler CH. Improving anaesthetists' communication skills. Anaesthesia 2004; 59: 166–72[CrossRef][ISI][Medline]

23 Pascoe GC. Patient satisfaction in primary health care: a literature review and analysis. Eval Program Plann 1983; 6: 185–210[CrossRef][Medline]




E-letters:

Read all E-letters

Experience of surveying satisfaction at two weeks.
Alfred P J Lake
British Journal of Anaesthesia, 31 Jan 2005 [Full text]

This Article
Abstract
Full Text (PDF)
All Versions of this Article:
94/2/206    most recent
aei024v1
E-Letters: Submit a response to the article
E-letters: View responses
Alert me when this article is cited
Alert me when E-letters are posted
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Disclaimer
Request Permissions
Google Scholar
Articles by Saal, D.
Articles by Heidegger, T.
PubMed
PubMed Citation
Articles by Saal, D.
Articles by Heidegger, T.