1 Section of Nephrology, Department of Medicine, University Hospital of North Norway, 2 Faculty of Medicine, University of Tromsø and 3 Department of Thoracic and Cardiovascular Surgery, University Hospital of North Norway, Tromsø, Norway
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods. All coronary bypass and valvular surgery patients from a 5-year period were included. Data on pre-operative risk factors for all patients who developed dialysis-dependent ARF and for a random sample of patients without ARF were retrospectively obtained from hospital databases and medical records. For each patient, a risk score for ARF was calculated on the basis of the algorithm. The sensitivity, specificity, positive and negative predictive values and area under receiver operating characteristic (ROC) curve of the score's ability to predict ARF were estimated.
Results. 2037 patients were included. The risk of ARF was 1.9%, and the area under the ROC curve 0.71. For a risk score of 6 or higher, the sensitivity was 0.53, the specificity 0.71, the positive predictive value 0.03 and the negative predictive value 0.99.
Conclusions. The validity of the algorithm was confirmed in a population differing in several aspects from the US populations where it was developed. Although it is useful for estimating the risk of ARF in groups of patients, the low risk of ARF limits the algorithm's ability to predict the outcome for individual patients.
Keywords: cardiac surgical procedures; coronary artery bypass; kidney failure acute; post-operative complications; receiver operating characteristic curve; sensitivity and specificity
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Changes in the treatment of post-operative ARF have had little impact on the outcome, possibly because improvements in therapy have been balanced by increasing pre-operative comorbidity [11]. For this reason, the possibility of identifying patients with a high risk of ARF before surgery has been investigated. In 1997, Chertow et al. [3] published an algorithm for predicting ARF from pre-operative data based on a study of patients in the Veterans Administration Continuous Improvement in Cardiac Surgery Program (CICSP). The algorithm estimates the risk of ARF from a severity score obtained by adding points for separate risk factors. It was validated in an independent sample of patients and demonstrated good discrimination. In a later study, Fortescue et al. [4] validated the algorithm in the Quality Measurement and Management Initiative (QMMI) cohort. Again, the algorithm performed well in discriminating for the development of ARF.
Cross-validation of risk-stratification algorithms is necessary before they can be accepted as useful in clinical practice [12,13]. So far, the CICSP algorithm has only been validated in patients in the US in the first half of the 1990s. The aim of the present study was to validate the algorithm in patients operated on at the University Hospital of North Norway between 1995 and 2000. Also, we wanted to compare risk factors for ARF after cardiac surgery with those found in the US cohorts and to assess the algorithm's usefulness by estimating the positive and negative predictive values of its predictions of ARF.
![]() |
Subjects and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Patients in need of renal replacement therapy within 30 days of the operation were considered to have experienced ARF as a complication. These patients were identified by review of medical records. A 33% random sample of the patients without ARF was drawn as controls, but because of financial limitations only 32% were actually examined. These 32% still constitute a valid random sample, and no bias was introduced by omission of the last 1%. Randomization was performed with a pseudo-random number generator.
Data
Age, gender and serum creatinine were obtained directly from hospital databases. Other data necessary to implement the CICSP algorithm were extracted from medical records by a medical student (K.R.S.H.) under the supervision of a nephrologist (B.O.E.). As in the validation based on QMMI data, left-ventricular ejection fraction <35% was used as a proxy for cardiomegaly [4]. For the other variables in the algorithm, we applied the definitions of risk factors from the CICSP study [3].
Creatinine clearance was estimated from serum creatinine, age, weight and gender by the formula of Cockcroft and Gault [14].
Operative mortality was defined as death from any cause within 30 days of surgery.
Statistical methods
A multivariate logistic regression analysis with ARF as the dichotomous dependent variable and the risk factors of the CICSP algorithm as the independent variables was performed with the SAS statistical package (SAS Institute, Cary, NC). Calibration of the model was assessed using the HosmerLemeshow goodness-of-fit test [15].
The CICSP additive severity score was calculated for each patient by adding points for each risk factor present, as specified in the study of Fortescue et al. [4]. The points associated with each type of risk factor were based on the odds-ratios (OR) found in the logistic regression analysis of risk factors in the CICSP study [3].
The score's sensitivity, specificity and positive and negative predictive value for predicting ARF for different cut-off levels of the score were calculated according to standard formulae as follows [16]:
![]() |
![]() |
![]() |
![]() |
For the calculation of the predictive values, the number of patients without ARF with a given score was estimated from the number of those randomly drawn by dividing this number by 0.32, i.e. the proportion of randomly drawn patients without ARF.
A receiver operating characteristic curve (ROC curve) was constructed by plotting the sensitivity vs (1 specificity) for all possible cut-off levels. The ROC curve area was calculated by the non-parametric technique of Hanley and McNeill [17].
Ninety-five per cent confidence intervals (CI) of proportions were calculated as exact binomial CI.
Missing data for dichotomous variables were coded as absent, and average values were substituted for missing continuous variables. Analyses were also repeated without patients with missing information.
Statistical significance was set at 0.05.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Thirty-eight of the operations were complicated by ARF, giving a risk of 1.9% (95% CI 1.32.6%). For patients with coronary bypass only, the risk was 1.2% (95% CI 0.81.9%), and for patients with valvular surgery with or without coronary bypass 4.9% (95% CI 2.97.7%). The 30-day mortality for patients with ARF was 45% (95% CI 2961%), for patients without ARF 2.6% (95% CI 1.93.3%).
Of the patients without ARF, 640 were randomly drawn as controls. A comparison between the baseline characteristics of patients with and without ARF according to type of surgery is presented in Table 1.
|
|
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The algorithm has previously been validated in the QMMI cohort [4]. However, this cohort did not include valvular surgery. In the present study, 17% of the patients had undergone valvular surgery with or without surgery on the coronary arteries. Compared with the CICSP cohort, there was a higher risk of ARF among patients with valvular surgery in our hospital. The logistic regression showed that the OR for this type of surgery was higher (4.1 vs 2.0), although the 95% CI included the CICSP OR (Table 2). Another difference was a higher mortality in the CICSP cohort (for patients with ARF 64 vs 45%; for patients without ARF 4.3 vs 2.6%). These differences were both statistically significant. Since ARF and death are competing risks in this situation, one could speculate that the different risk of ARF for valvular surgery could be explained by a higher mortality for the US patients. Unfortunately, the mortality for valvular surgery was not given for the CICSP cohort [3].
As a part of the cross-validation, we wanted to explore the potential for clinical use of the algorithm. A clinical decision based on the algorithm would have to fix a cut-off level of the additive score for deciding which patients were eligible for some intervention, as e.g. withholding surgery or preventive measures. The probability of developing ARF for patients with a score higher than the cut-off is conventionally referred to as the positive predictive value, and the probability of not developing ARF with a lower score as the negative predictive value [16]. In Table 3, these probabilities have been calculated for all possible cut-off levels of the score. As can be seen, the maximum positive predictive value was 0.14, i.e. if the score was greater than the cut-off it was still far more likely that the patient would not develop ARF than the opposite. On the other hand, the negative predictive value was greater than or equal to 0.98 for all cut-off levels.
These results indicate that the algorithm will not be very useful as a tool for making decisions about individual patients. The reason for this is the very low risk of ARF in both the CICSP- and QMMI-cohorts and in our material. As well as being functions of the sensitivity and specificity of the algorithm, the positive and negative predictive values depend on the risk of ARF. To identify these few patients accurately, an algorithm with almost perfect discriminatory power would be needed, which is rarely seen in this kind of clinical decision-making. On the other hand, the algorithm will be useful for estimating risk in groups of patients, e.g. when comparing results from different hospitals.
We conclude that the CICSP algorithm for predicting post-operative ARF was valid for patients undergoing cardiac surgery at our hospital. The risk factors for ARF in the present study were consistent with the risk factors found in studies with greater statistical power. Although the low risk of ARF precludes the prediction of ARF for individual patients, the CICSP algorithm is useful as a tool for risk stratification in populations. In addition, the identification of risk factors lays the foundation for further research into the pathophysiological mechanisms of the condition and possible preventive measures.
![]() |
Acknowledgments |
---|
![]() |
Notes |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|