1 Department of Anaesthesiology, St Gallen Cantonal Hospital, Rorschacherstrasse 95, CH-9007 St Gallen, Switzerland. 2 Picker Institute Europe, Zug, Switzerland. 3 Empirical Consulting, Freiburg, Germany. 4 Department of Anaesthesiology, Rätisches Cantonal Hospital, Chur, Switzerland. 5 Department of Anaesthesiology and Intensive Care Medicine, Landeskrankenhaus Feldkirch, Austria. 6 Department of Anaesthesiology and Critical Care Medicine, The Leopold-Franzens-University of Innsbruck, Austria. 7 Department of Anaesthesiology, St Vincents Hospital, Linz, Austria. 8 Department of Anaesthesiology, University Hospital of Bern, Switzerland*Corresponding author
Declaration of interest. Y. Husemann is the country manager of the Picker Institute Europe in Switzerland, which partly funded this study.
Some of the data were presented at the meeting of the European Society of Anaesthesiology (Gothenburg, April 2001), and published in abstract form (Eur J Anaesthesiol 2001; 18: A12).
*The appendix is available to subscribers with the online version of the journal at the journal website.
Accepted for publication: July 30, 2002
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods. We used a rigorous protocol: generation of items, construction of the pilot questionnaire, pilot study, statistical analysis (construct validity, factor analysis, reliability analysis), compilation of the final questionnaire, main study, repeated analysis of construct validity and reliability. We compared the mean total problem score and the scores for the dimensions: Information/Involvement in decision-making, and Continuity of personal care by anaesthetist. The influence of potential confounding variables was tested (multiple linear regression).
Results. The average problem score from all hospitals was 18.6%. Most problems are mentioned in the dimensions Information/Involvement in decision-making (mean problem score: 30.9%) and Continuity of personal care by anaesthetist (mean problem score: 32.2%). The overall assessment of the quality of anaesthesia care was good to excellent in 98.7% of cases. The most important dimension was Information/Involvement in decision-making. The mean total problem score was significantly lower for two hospitals than the total mean for all hospitals (significantly higher at two hospitals) (P<0.05). Amongst the confounding variables considered, age, sex, subjective state of health, type of anaesthesia and level of education had an influence on the total problem score and the two dimensions mentioned. There were only marginal differences with and without the influence of the confounding variables for the different hospitals.
Conclusions. A psychometric questionnaire on patient satisfaction with anaesthesia care must cover areas such as patient information, involvement in decision-making, and contact with the anaesthetist. The assessment using summed scores for dimensions is more informative than a global summed rating. There were significant differences between hospitals. Moreover, the high problem scores indicate a great potential for improvement at all hospitals.
Br J Anaesth 2002; 89: 86372
Keywords: anaesthesia, audit; research, anaesthesia; measurement techniques, outcome; surgery
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Construction of the pilot questionnaire
For the construction of the pilot questionnaire, the items were grouped in chronological order based on the usual course of treatment. Before constructing the pilot questionnaire, we tested the items for comprehensibility and readability on lay members of staff. The questions were aimed at patients who had undergone elective surgery under general or regional anaesthesia. The pilot questionnaire contained questions on the instrument itself (Table 1), questions on the patients overall impression, and space for free comment.
|
Final questionnaire
The final questionnaire was applied between September 2000 and January 2001 in six major hospitals in Switzerland and Austria. The number of beds at each hospital ranged from 250 to 1480, and the number of anaesthetics per year from 5100 to 28 700. A total of 3785 questionnaires were sent to patients 12 weeks after discharge. The aim was to obtain at least 300 evaluable questionnaires from each hospital. Non-respondents were sent a second questionnaire 2 weeks later, together with a reminder letter.
The analysis of the construct validity and the creation of the dimensions with reliability analysis was repeated. Using standardized beta weights, we then calculated how great the relative influence of each of the dimensions was (importance).11
We also compared some perioperative characteristics (ASA class, extent of surgery, length of hospital stay, type of anaesthesia) of non-respondents to assess selection bias and representativeness.
Benchmarking
The comparison was based on the means of the total problem score and the two most frequently mentioned dimensions with problems (Information/Involvement in decision-making and Continuity of personal care by anaesthetist). Possible effects of the following variables on the frequency of problems in each dimension were investigated: age, sex, state of health, length of hospital stay, extent of surgery, ASA class, type of health insurance, number of hospital stays in the past 6 months, type of anaesthesia, and level of education. If an influence was found, we investigated whether the composition of the patient sample from the hospitals (case-mix) involved was the reason for the distortion of the benchmarking values. This made it possible to adjust the comparison between the individual hospitals for the confounding variables (i.e. to take into account effects caused by different composition of the patient samples). For the comparative analysis, the mean value over all hospitals (the total problem score and the scores for the two above-mentioned dimensions) were taken as reference. The hospitals were given letter codes AF to preserve anonymity.
![]() |
Statistical analysis |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
An individual global summed problem score was calculated on the basis of this problem rating (proportion of problems mentioned for all relevant questions). We checked for construct validity by establishing with multiple linear regression whether the single items had an influence on the different aspects of overall care (for example: How would your rate the overall care you received for your anaesthesia?) or on the global summed problem score. If an item was of low statistical importance, we had to decide whether to retain or exclude it, based on its content.
Creation of dimensions
We then created higher-level dimensions to categorize the patients perception of quality on the basis of these individual problem ratings. To achieve this, we first of all subjected all problem indicators to exploratory factor analysis (principal component analysis, varimax rotation, scree test), to determine the number of dimensions that could be created from the problems mentioned.12
The internal consistency of the dimensions determined by factor analysis was then checked using reliability analysis. We calculated Cronbachs coefficient alpha.7 11 A score from 0 to 100 was then calculated for each dimension, which reflected the proportion of problem ratings in the respective area. The results of this analysis were discussed and the questionnaire was modified accordingly for the main survey.
Comparison of the hospitals and influence of potential confounding variables
The mean problem scores between the hospitals were compared using analysis of variance and simple linear regression, with the total mean problem score as one reference and the best value as the second. The effects of the potential confounding variables on each dimension were first determined univariately, followed by multivariate analysis using multiple linear regression (forward-stepwise method). In the case of significant effects of these confounders, adjustment was performed. The findings are expressed in per cent (mean) or as mean (SD). Ranges are presented where appropriate. All analyses were conducted using the SPSS 10 analysis package (SPSS Inc. Chicago, IL, USA).
The multiple linear regression models, with and without adjustment for the confounding variables, are presented in the appendix.*
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
The overall response rate (including responses to reminder letters) was 61%. Ninety-three percent of respondents said the questionnaire was very easy or easy to complete, and 96% said it was very easy or easy to understand. Ninety percent of respondents completed the questionnaire within 20 minutes. Eighty-nine percent of respondents felt that all important questions related to anaesthesia had been asked, and 9% that some of them had been asked.
Pilot study
Missing-value analysis
Additional categories were incorporated into the final questionnaire, such as the possible response: I underwent combined (regional and general) anaesthesia, for the question on the type of anaesthesia.
Analysis of distribution
Some questions, particularly those that assessed overall satisfaction, showed very skewed distributions. The question Would you recommend the anaesthesiologist who looked after you to your family and friends? was excluded from the final questionnaire for this reason.
Only a few suggestions for missing questions were received, which supports the content validity of the pilot instrument. Additional questions suggested by patients therefore only resulted in minor modifications to the list of questions and were mainly related to the long-term after-effects of anaesthesia.
The analysis of construct validity using multiple linear regression of each individual aspect on the overall satisfaction was performed to establish the importance of individual aspects in the overall assessment. Since the questions on the overall impression (for example: How would you rate the overall assessment of care you received for your anaesthesia) were only of limited suitability because of their extremely skewed distributions (only 0.51.8% of all ratings were fair and poor), this analysis was primarily based on the global summed problem score. Taking into account the eight most important items, this resulted in an R2 value of 0.85.
The factor and reliability analysis to enable grouping of the individual aspects into problem dimensions resulted in three scales with a Cronbachs alpha of >0.7 each (Information/Involvement in decision-making/Continuity of personal care by anaesthetist; Respect/confidence and Delay management), and two scales with a Cronbachs alpha <0.7 (Pain management and Nursing care in the recovery room).
Final questionnaire
A total of 2348 questionnaires from six hospitals were included in the analysis. The response rate was 62% (including responses to reminder letters) (range 5369%). The age range of the respondents was 1692 yr, and the female:male ratio was 51:49.
The analysis of the perioperative characteristics of the non-respondents showed that the type of anaesthesia and the extent of surgery had no influence on participation. The duration of the hospital stay (non-respondents had stays of 1 day less) and the ASA class (the patients in higher classes were slightly less prepared to participate) had a slightly positive effect on the readiness to participate amongst the respondents. Seen overall, any skew was minimal. The proportion of the declared variance in the readiness to participate was well below 0.5% in each case.
Creation and testing of dimensions
A total of 29 dichotomous problem ratings were included in the factor analysis. On the basis of the scree test,12 a six-factor solution with an explained variance of 45.3% was selected as the best classification. The subsequent reliability analysis essentially confirmed the results of the factor analysis. The result showed that three of the dimensions had good internal consistency (Cronbachs alpha >0.7) and that two were of lower quality (Cronbachs alpha =0.43). The dimension Nursing care in the recovery room consisted of only two items (Cronbachs alpha =0.69) (Table 3). It was then possible to calculate for each patient the problem scores for the six dimensions as a percentage of the problem ratings he or she gave.
|
Table 4 shows the distribution of the potentially confounding variables in the six hospitals. Table 5 shows the results of the univariate and multivariate analysis of the confounding variables considered. The results of the multiple linear regression were as follows: (i) Information/Involvement in decision-making was by far the most important dimension, with a beta coefficient of 0.60.1 Only the confounding variables subjective state of health (the worse the state of health, the more critical the patient) and type of anaesthesia (patients who underwent general anaesthesia were more critical) had an effect on the scores. (ii) Continuity of personal care by anaesthetist (problem score: 32.6%; beta coefficient: 0.27) was influenced only by the factors age (the older the patient, the less critical), sex (men were less critical), and level of education (less criticism from those with lower level). (iii) Mean total problem score (problem score 18.6%) was influenced only by the factors age, sex, type of anaesthesia and subjective state of health (in accordance with the changes in the above-mentioned dimensions).
|
|
|
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Limitations of the study
We did not conduct a test-retest procedure because we were of the opinion that it was too much to expect patients to complete a third, and possibly even a fourth, questionnaire, and also felt that this would considerably increase the effort required in terms of practicability and cost. The value of retesting with regard to reliability is also controversial. It is generally accepted that, in order to obtain reliable results, the focus should be on internal consistency.6
Although we had some perioperative characteristics from the non-respondents, we did not have further important additional information from these patients. In their study, Fung and Cohen13 also documented some personal details (age, sex, type of surgery), but did not state why the non-respondents did not reply. It would also be interesting to hear these patients opinions on their degree of satisfaction with their anaesthesia care. Socio-medical investigations have shown that non-respondents may evaluate care less favourably than those who do respond.14 In contrast, however, Ware and co-workers15 found that patients who were more satisfied with their quality of care were less likely to return questionnaires.
Our response rate of 62% (n=2348) is lower than that achieved by Fung and Cohen (71%, n=45) and by Whitty and colleagues (73%, n=173).16 This may be because our questionnaire was returned to an independent institute for evaluation.
It might be claimed that bias was introduced because two of the hospitals concerned participated in the pilot study, resulting in better scores in the main study. However, this is unlikely for the following reasons: first, different patients were surveyed in the pilot study and the final study; second, the two hospitals were not informed of the results of the pilot study, thus excluding the possibility of a shorter learning curve; third, there were only very slight differences between the overall problem scores for these two hospitals in the pilot study and the final study.
Although data do exist on predictors of postoperative outcome,1719 no findings are available on the influence of confounding variables on patient satisfaction.5 6 Our results showed that the subjective state of health, age, sex, level of education and type of anaesthesia had an effect on the dimensions Information/Involvement in decision-making and Continuity of personal care by anaesthetist. However, the extent of surgery, type of insurance, number of hospital stays in the past 6 months, duration of hospital stay and the ASA class had no influence on the number of problems mentioned. The variables selected may not be the most appropriate, since some of them were derived from outcomes research,17 and we were only able to perform analyses with the confounding variables we selected.20 We did not include, for example, social desirability as a confounding variable, as proposed by Le May and co-workers,6 as the social desirability bias was minimized in our study by sending out questionnaires and not having interviewers present.11
Comparisons with results from other studies
Comparisons with other studies on patient satisfaction with anaesthesia care are difficult, since at present there is little or no similar published work in this area.5 6 Moreover, current studies on patient satisfaction are of questionable value.6 7 21
The development of a psychometric questionnaire must follow a rigorous protocol.5 7 11 12 22 For example, Sitzia7 states that the prerequisites for a valid and reliable questionnaire are the presence of some elements of content validity and construct or criterion validity and reliability (internal consistency). According to Le May and co-workers6 and Wu and co-workers,21 none of the papers quoted by them followed such a rigorous protocol to measure patient satisfaction with general or regional anaesthesia in in-patients.
The generation of items must incorporate the patients perspective by using focus groups, for example.3 11 23 24 Otherwise, the surveys reflect the bias of the experts who constructed them. This is an instrument for content validity testing. Focus groups are required by several authors.3 23 25 We found, however, only two publications on anaesthesia in which focus groups were used for item generation.13 16 By using a pilot study including open questions, we also incorporated a second element of content testing.7
The statistical analysis of the patients responses to the pilot questionnaire is an important part of the development of a psychometric questionnaire.5 There is no consensus about the accepted level of reliability, but the most popular measure, the Cronbach coefficient alpha, should exceed at least 0.6 or 0.7.26 Our results showed that the dimensions Information/Involvement in decision-making, Respect/Confidence and Delay management exceeded these levels (alpha >0.7). In agreement with others, we found that the dimension Information/Involvement in decision-making was one of the areas where most problems were mentioned.16 23 27
In addition to the number of problems mentioned, the importance of the problems is also of great significance. As with reliability measurement, there are several methods available to assess this. The right answer is far from clear.11 One of the accepted methods is weighting using multiple linear regression via beta weights.11 In agreement with Fung and Cohen13 who studied out-patients, our results showed that information (and communication) is the most important dimension. It is well recognized that involvement in decision-making improves patient satisfaction.28
It is also difficult to compare our results with those made at other hospitals because studies conducted so far have compared anaesthesia-related incidents and not patient satisfaction.9 Measuring the quality of care by an anaesthesia team by comparing major outcomes has emerged as unsatisfactory because such events (in particular, death) occur only very rarely. Cohen and co-workers10 concluded that their investigation of 25 000 patients in four hospitals was not powerful enough to demonstrate a difference in mortality. Because of the low incidence of major adverse outcomes, it is therefore very unreliable to use these as a basis to draw conclusions about the quality of an anaesthesia department.10
Unlike patient experience with hospital care,29 as far as we are aware, there are no benchmarking studies on patient satisfaction with anaesthesia care conducted using a multi-dimensional, validated questionnaire. Analysis of patient surveys by Coulter and Cleary23 also revealed problems with provision of information, respect for patients preferences, and continuity of their care. Their findings also showed a striking difference between the total number of problems mentioned between the best and the worst hospitals.
The analysis of the confounding variables and the consequent adjustment based on the composition of the patient sample enabled us to compare the hospitals with each other. The effects found were very slight, however, because the hospitals were very similar with regard to the influence of the confounding variables considered. Such an analysis should nevertheless be conducted to elicit any possible effects and make appropriate adjustments.10 30 31 It is known from the literature that older patients are less critical.32 This was confirmed by our findings but other studies found no significant relationship of this sort.25 In line with most other studies, our findings also showed a positive correlation between health status and satisfaction.31
In summary, the development of a psychometric questionnaire for the assessment of patient satisfaction with anaesthesia care requires the following of a rigorous protocol, including the patients perspective. The implications for anaesthetic practice are that all hospitals involved should first of all consider how they can improve the situation with regard to the provision of information on anaesthesia care. As a continuous quality-improvement process, all measures taken should be evaluated and compared subsequently. The importance of communication which can simply be used as a generic term for almost all our dimensionscannot be emphasized strongly enough: Patients dont care what we know, they want to know that we care.
![]() |
Acknowledgements |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Ebner H, Köck CM. Qualität als Wettbewerbsfaktor für Gesundheitsorganisationen. In: Heimerl-Wagner P, Köck CM, eds. Management in Gesundheitsorganisationen. Wien: Ueberreuter, 1996; 72101
3 Myles PS, Hunt JO, Nightingale CE, et al. Development and psychometric testing of a quality of recovery score after general anaesthesia and surgery in adults. Anesth Analg 1999; 88: 8390
4 Juran JM. Juran on Quality by Design, 1st Edn. New York: Free Press, 1992
5 Fung D, Cohen MM. Measuring patient satisfaction with anaesthesia care: a review of current methodology. Anesth Analg 1998; 87: 108998[ISI][Medline]
6 LeMay S, Hardy JF, Taillefer MC, Dupuis G. Patient satisfaction with anaesthesia services. Can J Anaesth 2001; 48: 15361
7 Sitzia J. How valid and reliable are patient satisfaction data? An analysis of 195 studies. Int J Qual Health Care 1999; 11: 31928
8 Myles PS, Williams DL, Hendrata M, Anderson H, Weeks AM. Patient satisfaction after anaesthesia and surgery: results of a prospective survey of 10 811 patients. Br J Anaesth 2000; 84: 610[Abstract]
9 Bothner U, Georgieff M, Schwilk B. Building a large-scale perioperative anaesthesia outcome-tracking database: methodology, implementation, and experiences from one provider within the German quality project. Br J Anaesth 2000; 85: 27180
10 Cohen MM, Duncan PG, Pope WDB, et al. The Canadian four-centre study of anaesthetic outcomes: II. Can outcomes be used to assess the quality of anaesthesia care? Can J Anaesth 1992; 39: 4309[Abstract]
11 Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to their Development and Use, 2nd Edn. Oxford: Oxford University Press, 1995
12 De Vellis RF. Scale Development. Theory and Applications, 1st Edn. London: Sage Publications, 1991
13 Fung D, Cohen MM. What do outpatients value most in their anaesthesia care? Can J Anaesth 2001; 48: 1219
14 Ley P, Bradshaw PW, Kinsey JA, Atherton ST. Increasing patients satisfaction with communication. Br J Soc Clin Psychol 1976; 15: 40313[ISI][Medline]
15 Ware JE, Snyder MK, Wright WR, Davies AR. Defining and measuring patient satisfaction with medical care. Eval Program Plann 1983; 6: 24763[Medline]
16 Whitty PM, Shaw IH, Goodwin DR. Patient satisfaction with general anaesthesia. Too difficult to measure? Anaesthesia 1996; 51: 32732[ISI][Medline]
17 Wolters U, Wolf T, Stützer H, Schröder T. ASA classification and perioperative variables as predictors of postoperative outcome. Br J Anaesth 1996; 77: 21722
18 Duncan PG, Cohen MM. Anaesthetic morbidity: factors of significance to anaesthetic practice. Can J Anaesth 1987; 34: 28[Abstract]
19 Farrow SC, Fowkes FG, Lunn JN, Robertson IB, Sweetnam P. Epidemiology in anaesthesia: a method for predicting hospital mortality. Eur J Anaesthesiol 1984; 1: 7784[Medline]
20 Datta M. You cannot exclude the explanation you have not considered. Lancet 1993; 342: 3457[ISI][Medline]
21 Wu CL, Naqibuddin M, Fleisher LA. Measurement of patient satisfaction as an outcome of regional anaesthesia and analgesia: a systematic review. Reg Anesth Pain Med 2001; 26: 196208
22 Crocker L, Algina J. Introduction to Classical and Modern Test Theory. Fort Worth: Harcourt Brace Jovanovic College Publishers, 1986
23 Allshouse KD. Treating patients as individuals. In: Gerteis M, Edgman-Levitan S, Daley J, Delbanco TL, eds. Through the Patients Eyes. San Francisco: Jossey-Bass Publishers, 1993; 1944
24 Dolan P, Cookson R, Ferguson B. Effect of discussion and deliberation on the publics views of priority setting in health care: focus group study. Br Med J 1999; 318: 91619
25 Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality of care. Inquiry 1988; 25: 2536[ISI][Medline]
26 Malhorta NK, Marketing Research an Applied Orientation, 2nd Edn. Upper Saddle River, NJ: Prentice Hall, 1996
27 Wisiak UV, Kröll W, List W. Communication during the preoperative visit. Eur J Anaesthesiol 1991; 51: 32732
28 Hornberger JC, Habraken H, Bloch DA. Minimum data needed on patient preferences for accurate, efficient medical decision making. Med Care 1995; 33: 297310[ISI][Medline]
29 Coulter A, Cleary PD. Patientsexperiences with hospital care in five countries. Health Aff 2001; 20: 24452
30 Greenfield S, Aronow H, Elashoff RM, Watanabe D. Flaws in mortality data. The hazards of ignoring comorbid disease. JAMA 1988; 260: 22535[Abstract]
31 Aharony L, Strasser S. Patient satisfaction: what we know and what we still need to explore. Med Care Rev 1993; 50: 4979[Medline]
32 Pascoe GC. Patient satisfaction in primary health care: a literature review and analysis. Eval Program Plann 1983; 6: 185210[Medline]