1 Department of Neuroscience, Anaesthesia and Intensive Care Unit, Marche Polytechnic University, Ancona, Italy. 2 Department of Anaesthesiology, University La Sapienza, Rome, Italy
* Corresponding author: Anestesia e Rianimazione Clinica, Ospedale Regionale Torrette, Via Conca 1, 60020 Torrette di Ancona, Italy. E-mail: donati{at}indi.it
Accepted for publication April 30, 2004.
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods. Data were collected in two hospitals. All types of surgery were included except for cardiac surgery and Caesarean delivery. Age, sex and preoperative information, including the presence of cardiocirculatory and/or lung disease, renal failure, diabetes mellitus, hepatic disease, cancer, Glasgow Coma Score, ASA grade, surgical diagnosis, severity of the procedure and type of surgery (elective, urgent or emergency), were recorded for each patient. The model was developed using a data set incorporating data from 1936 surgical patients, and validated using data from a further 1849 patients. Forward stepwise logistic regression was used to build the model. Goodness of fit was examined using the HosmerLemeshow test and receiver operating characteristic (ROC) curve analyses were performed on both data sets to test calibration and discrimination. In the validation data set, the new model was compared with POSSUM and P-POSSUM for both calibration and discrimination, and with ASA alone to compare discrimination.
Results. The following variables were included in the new model: ASA status, age, type of surgery (elective, urgent, emergency) and degree of surgery (minor, moderate or major). Calibration and discrimination of the new model were good in both development and validation data sets. This new model was better calibrated in the validation data set (HosmerLemeshow goodness-of-fit test: 2=6.8017, P=0.7440) than either P-POSSUM (
2=14.4643, P=0.1528) or POSSUM, which was not calibrated (
2=31.8147, P=0.0004). POSSUM and P-POSSUM had better discrimination than the new model, although this was not statistically significant. Comparing the two ROC curves, the new model had better discrimination than ASA alone (difference between areas, 0.077, SE 0.034, 95% confidence interval 0.0120.143, P=0.021).
Conclusions. This new, ASA status-based model is simple to use and can be performed routinely in the operating room to predict operative risk for both elective and emergency surgery.
Keywords: assessment, perioperative ; risk, operative ; surgery, outcome
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The aims of this study were to develop a new model for assessing operative risk that is easy to both calculate and use, and to validate this new model against both POSSUM and P-POSSUM, an updated system that takes into account some of the shortcomings of the original POSSUM scoring system. These models are very widely used for the prediction of postoperative mortality, and probably represent the standard that any new and improved modelling process would hope to supersede. Finally, the discriminative ability of the new model was compared with the ASA score alone.
![]() |
Patients and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
To develop the new model, only preoperative variables were used. The model was produced using forward stepwise logistic regression (1990 BMDP Statistical Software Inc., Cork, Ireland, running under DOS and WindowsTM platforms). The logistic regression model is explained in the Appendix.
The validation data set was recorded from January to April 2002 of 1849 consecutive patients in the same two hospitals. The operative risk was calculated both for the new model and for POSSUM and P-POSSUM scores. The HosmerLemeshow goodness-of-fit test was used for calibration, comparing the expected and observed numbers of deaths by risk group, and the area under the receiver operating characteristic (ROC) curve was measured for discrimination. Pairwise comparisons of ROC curves from the new model, POSSUM and P-POSSUM were performed (MedCalc 7.1; Medcalc Software, Mariakerke, Belgium). In the validation data set, results from the new model were compared with ASA status alone for discrimination by pairwise comparison of ROC curves.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
|
The validation data set comprised 1849 patients (Table 2). The new model applied to this data set was also well calibrated using the HosmerLemeshow goodness-of-fit test (2=6.8017, P=0. 7440). In this data set the POSSUM score showed poor calibration (
2=31.8147, P=0.0004). Better calibration was seen for the P-POSSUM score, although this was still inferior to our new model (
2=14.4643, P=0.1528). The discriminatory ability of the POSSUM score, the P-POSSUM score and the new model were assessed using ROC curves (Fig. 1). The area under the ROC curve for the new model was 0.888, SE 0.025, 95% confidence interval (CI) 0.8380.937. The ROC curve for the POSSUM score was 0.915 (SE 0.016, CI 0.8840.947) and for the P-POSSUM score it was 0.912 (SE 0.033, CI 0.8980.924). Pairwise comparison of ROC curves between the new model and the POSSUM score showed a difference between areas of 0.028 (SE 0.035, CI 0.0400.095, P=0.423), and between the new model and P-POSSUM score of 0.024 (SE 0.035, CI 0.0440.092, P=0.491). Between the POSSUM and P-POSSUM scores, the difference between areas was 0.004 (SE 0.005, CI 0.0080.016, P=0.549).
|
The operative risk was calculated for different age groups on the basis of ASA class and the type of surgery (elective or emergency) for both major (Table 6A) and moderate to minor surgery (Table 6B). The risk was calculated on the median and range (minimum maximum) of values for each age group. These tables are provided in order to overcome what would otherwise be the considerable challenge of performing a calculation based on a logistic regression equation at the patient's bedside.
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Many studies have been published recommending a variety of scores.13 The perfect index would be one that is easy and quick to use, adoptable by all hospitals, and able to predict the operative risk in all surgical patients, whether elective or urgent/emergent. The Goldman Cardiac Risk Index introduced by Goldman and colleagues in 19778 agrees in part with these requirements. It is applicable to all types of surgical operation but it only calculates the risk of onset of cardiovascular complications. Other authors have also analysed cardiovascular risk for both elective and emergency surgery.9 10 Chung and colleagues proposed a predictive model on the basis of pre-existing medical conditions, but this model is only applicable in day-case surgery.11 POSSUM (and the derived P-POSSUM) is a good model as it can be used in all types of surgery, both elective and emergency. P-POSSUM (and POSSUM) has been used for many purposes: to compare mortality rates after surgery between patients in the USA and UK,12 to assess outcome after laparoscopic colectomy13 or after surgery for colorectal cancer14 and to predict mortality in infrarenal abdominal aortic aneurysm repair.15 However, as its use requires intra- and postoperative data it is neither simple nor rapid. Moreover, for its complete calculation blood samples and physiological measurements are necessary. However, its greatest limitation as a prognostic score is its applicability only after the surgical procedure. It cannot be used preoperatively, when the patient (and surgeon) should ideally be aware of the operative risk.
In agreement with the literature, we found in our study an overestimation of the operative risk for POSSUM that is more important in the lower deciles. Whiteley and colleagues reviewed the POSSUM model, changing the coefficients, and made similar criticisms.16 The lowest physiological and operative scores are 12 and 6 respectively; when applied to the POSSUM mortality predictor equation this gives a minimum risk of death of 1.1%. This is far too high, given that it represents the fittest individual undergoing the least intricate surgery. Previously published series of fit people undergoing uncomplicated hernia repair suggest that mortality rates are less than 0.001%.17
With regard to variables and their strength of prediction of risk, there are ample candidates to be included in a prognostic model. Some relate to the patient and some to the surgical procedure. Age is a significant patient factor and thus enters the model. Indeed, it significantly increases the accuracy of prediction (P=0.0228). This agrees with most of the published literature, which considers age to be an important factor for increased mortality risk.18 19 However, it is important to note that it is not age per se but the deterioration of organ function that occurs with age.18 The odds ratio of 1.03 found in our study is consistent with the study by Wolters and colleagues,20 who reported an odds ratio of 1.0105 per year of life increment.
The variable best correlated with an increase in operative risk was the physical condition of the subject as represented by the ASA grade, with an odds ratio of 2.97. It is important to note that it not only significantly increases the accuracy of prediction (P<0.0001), but after it entered the model all other preoperative risk factors, such as heart and lung disease and renal failure, were not included in the model as they were independently not significant. Since 1941,21 and with some subsequent modifications,6 the ASA grading has been the most important instrument for assessing the patient's baseline health status. It has also been applied with other variables to predict postoperative complications.20 Wolters and colleagues examined the strength of association between ASA grade and perioperative risk factors and postoperative outcome, with both univariate analysis and logistic regression.20 They found that intraoperative blood loss, duration of postoperative ventilation, duration of intensive care stay, rates of pulmonary and cardiac complications, and in-hospital mortality showed significant increases as ASA status advanced from I to IV. In contrast to our present study, their study did not intend to build a mathematical model to predict mortality and/or postoperative complications. However, their results demonstrated not only the association between ASA status and postoperative outcome, but also the great value of this type of statistical analysis in the improvement of patient therapy.
The importance of the type of surgery has been emphasized previously.22 23 Elective surgery and minor severity surgery reduce operative risk as the greater effect on poor outcome is attributable to emergency and/or high-severity surgery. A patient in poor physical condition who needs emergency surgery may perhaps benefit from a reduction in severity of the surgery, or deferring major surgery until their state of health has been optimized.24 Thus, our new model, which includes the mode and severity of surgery, improved on the discriminatory ability of the ASA grade alone.
In conclusion, this new model can be helpful for both surgeons and anaesthetists in daily practice, providing them with a true idea of the operative risk of death of the surgical patient. It will also be useful as an internal quality assessment. The next step will be to include postoperative complications in this model in order to have a more complete score for evaluating surgical patient outcome.
![]() |
Appendix |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
If we consider i as the population probability of an event (also called the expected value) we can write E(yi)=
i (E is for expected value). If an event has probability
i, the odds ratio for this is
i/(1
i) to 1.
The model is
![]() |
The term on the left-hand side of the equation is the log of the odds of success, and is called the logistic or logit transform.
The coefficient ß is related to the odds ratio in 2x2 tables. If the predictor variable (x) is binary, the odds ratio associated with x is given by exp(ß). If x is continuous, exp(ß) is the odds ratio associated with a unit increase in x. The parameters in the model are estimated with the maximum likelihood function.
Table 5 shows variables entering the model with their respective weights (i.e. the coefficient ß for every factor). The threshold for inclusion of the variables in the model was a significance to predict death lower than 0.05, while the removal limit was P>0.10. Tests for linearity were performed for ASA status and age, which were considered continuous variables. The variables were tested in two ways [25]. First, a quadratic term (x2) was included in addition to the linear term (x) in the model. A significant coefficient for x2 indicates a lack of linearity, but in this case there was not a significant coefficient for x2, either for age (P=0.11) or for ASA (P=0.18). Secondly, ASA grade and age were considered as categorical variables, with four categories for ASA and five for age (dividing age into five quintile groups), and the coefficients were examined. For a linear relationship, the coefficients themselves will increase linearly, and this happened both for ASA and age.
The program generates design variables for each categorical variable. These are used in the model instead of the value or category numbers recorded for the variable. The design variables that are generated either contrast the first category with later categories or are orthogonal polynomial components. Assuming three categories, the program generates by default two design variables, (1) and (2), of the following type (Table A1): design (1) (category one, 1; category two, 1; category three, 0); design (2) (category one, 1; category two, 0; category three, 1). The coefficient entering the model for category one is: design (1) design (2); for category two is equal to design (1) and for category three is equal to design (2).
|
![]() |
![]() |
Acknowledgments |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Klotz Hp, Candinas D, Platz A, et al. Preoperative risk assessment in elective general surgery. Br J Surg 1996; 83: 178891[ISI][Medline]
3 Kroennke K, Lawrence VA, Theroux J, Tuley MR, Hilsenbeck S. Postoperative complication after thoracic and major abdominal surgery in patients with and without obstructive lung disease. Chest 1993; 104: 144551[Abstract]
4 Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgical audit. Br J Surg 1991; 52: 35560
5 Pietropaoli P, Donati A, Arzano S, Messina T. Valutazione comparativa del Rischio Operatorio tra due studi, eseguiti a distanza di 10 anni, presso due identiche strutture. Minerva Anestesiol 1999; 65 (9S): 72936
6 Vacanti CJ, Van Houten RJ, Hil RC. A statistical analysis of the relationship of physical status to postoperative mortality in 68,388 cases. Anesth Analg 1970; 49: 5646[Medline]
7 Pasternak LR. Preanesthesia evaluation of the surgical patient. ASA Refresher Courses in Anesthesiology 1996; 24: 20519
8 Goldman L, Caldera DL, Nussbaum SR, et al. Multifactorial index of cardiac risk in noncardiac surgical procedures. N Engl J Med 1977; 297: 84550[Abstract]
9 Howell SJ, Sear YM, Yates D, Goldacre M, Foex P. Risk factors for cardiovascular death within 30 days after anaesthesia and urgent or emergency surgery: a nested casecontrol study. Br J Anaesth 1999; 82: 67984
10 Howell SJ, Sear YM, Yates D, Goldacre M, Sear JW, Foex P. Risk factors for cardiovascular death after elective surgery under general anaesthesia. Br J Anaesth 1998; 80: 700[CrossRef][ISI][Medline]
11 Chung F, Mezei G, Tong D. Pre-existing conditions as predictors of adverse events in day-case surgery. Br J Anaesth 1999; 83: 26270
12 Bennett-Guerrero E, Hyam JA, Shaefi S, et al. Comparison of P-POSSUM risk-adjusted mortality rates after surgery between patients in the USA and the UK. Br J Surg 2003; 90: 15938[CrossRef][ISI][Medline]
13 Senagore AJ, Delaney CP, Duepree HJ, Brady KM, Fazio VW. Evaluation of POSSUM and P-POSSUM scoring systems in assessing outcome after laparoscopic colectomy. Br J Surg 2003; 90: 12804[CrossRef][ISI][Medline]
14 Menon KV, Farouk R. An analysis of the accuracy of P-POSSUM scoring for mortality risk assessment after surgery for colorectal cancer. Colorectal Dis 2002; 4: 197200[CrossRef][Medline]
15 Shuhaiber JH, Hankins M, Robless P, Whitehead SM. Comparison of POSSUM with P-POSSUM for prediction of mortality in infrarenal abdominal aortic aneurysm repair. Ann Vasc Surg 2002; 16: 73641[CrossRef][ISI][Medline]
16 Whiteley MS, Prythrch DR, Higgins B, Weaver PC, Pruot WG. An evaluation of the POSSUM surgical scoring system. Br J Surg 1996; 83: 8125[ISI][Medline]
17 Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Br J Surg 1998; 85: 121720[CrossRef][ISI][Medline]
18 Priebe HJ. The aged cardiovascular risk patient. Br J Anaesth 2000; 85: 76378
19 Jin F, Chung F. Minimizing perioperative adverse events in the elderly. Br J Anaesth 2001; 87: 5336
20 Wolters U, Wolf T, Stützer H, Schröder T. ASA classification and perioperative variables as predictors of postoperative outcome. Br J Anaesth 1996; 77: 21722
21 Sakland M. Grading of patients for surgical procedures. Anesthesiology 1941; 2: 2814
22 Tiret L, Hatton F, Desmonts JM, Vourc'h G. Prediction of outcome of anaesthesia in patients over 40 years: a multifactorial risk index. Stat Med 1988; 7: 94754[ISI][Medline]
23 Arviddson S, Ouchterlong J, Sjostedt L, Swardsudd K. Predicting postoperative adverse events. Clinical efficiency of four general classification system. Acta Anaesthesiol Scand 1996; 40: 78391[ISI][Medline]
24 Curra JE, Grounds RM. Ward versus intensive care management of high-risk patients. Br J Surg 1998; 85: 95661[CrossRef][ISI][Medline]
25 Campbell MJ. Statistics at Square Two. London: BMJ Books, 2001; 2627, 3758
26 Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1980; 115: 92106