Evaluation of simple criteria to predict successful weaning from mechanical ventilation in intensive care patients

T. S. Walsh*, S. Dodds and F. McArdle

Anaesthetics, Critical Care and Pain Medicine, New Royal Infirmary of Edinburgh, Little France, Edinburgh, Scotland EH16 4SU, UK

*Corresponding author. E-mail: tim{at}walsh.sol.co.uk

Accepted for publication: January 7, 2004


    Abstract
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Background. There is increasing evidence that weaning protocols improve outcome from mechanical ventilation, but it is unclear how best to implement such protocols in large intensive care units. We evaluated a checklist of simple bedside criteria to determine whether it could be used reliably to predict successful discontinuation of mechanical ventilation.

Methods. We carried out a prospective observational cohort study in a 12-bedded general intensive care unit (ICU). We developed a checklist of metabolic, cardiorespiratory and neurological criteria that suggested that patients should start the weaning process. We performed daily assessments throughout ICU stay and recorded whether the criteria were met. Ultimate ventilator independence was used as the reference standard.

Results. We studied 325 sequential admissions to the ICU. Data were available for 98% of patients; 97% of admissions were mechanically ventilated on admission to ICU. Overall, 205 of the 308 ventilated patients (67%) achieved ventilator independence during ICU admission; the other patients died or were transferred ventilated to other ICUs. Eighty-three per cent of the patients who achieved ventilator independence met the set criteria. Fulfilling the criteria was a moderately strong predictor of ultimate ventilator independence: specificity 89%, positive predictive value 94%, positive likelihood ratio (LR) 7.6. When we analysed data by the day from admission on which patients were examined, the test was a strong predictor of subsequent ventilator independence when criteria were met by day 1 (LR 11.1) or day 2 (LR 6.9), but weaker when met by more than/equal to 4 days (LR <3). Patients who met criteria after more than/equal to 4 days often had prolonged weaning and a high incidence of re-intubation. Patients who achieved ventilator independence without fulfilling the criteria (n=35) had a short duration of mechanical ventilation (median 2 days, interquartile range 1–3 days). The most frequent reason for failing criteria before ventilator independence was a PaO2/FIO2 ratio less than 24 kPa (49% of cases).

Conclusions. A simple checklist can assist nurse assessment of suitability for weaning and could be used as a trigger to commence a weaning protocol. The day on which criteria are met is a useful way of stratifying patients for likely patterns of weaning.

Br J Anaesth 2004; 92: 793–9

Keywords: monitoring, diagnostic test; monitoring, intensive care; ventilation, mechanical; ventilation, weaning


    Introduction
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
The process of weaning from mechanical ventilation is central to the management of critically ill patients. Delayed or unnecessarily prolonged weaning increases intensive care unit (ICU) length of stay, increases the cost of ICU care, decreases the availability of ICU beds, and can adversely affect patient outcome.1 The process of discontinuing mechanical ventilation is complex and has been considered to occur in several stages. First, the patient must recover sufficiently from the acute derangements in physiology that complicated their episode of critical illness to consider commencing the process of weaning. Secondly, the patient must undergo progressive reductions in ventilator support until a point is reached where he/she is capable of spontaneous breathing. Thirdly, the patient must be disconnected from the ventilator, extubated, and breathe spontaneously. Various strategies have been suggested to identify patients that are suitable for weaning and to manage patients during the weaning process.25 The optimum strategy is controversial, but it is recognized that weaning protocols are important in order to avoid delays in the weaning process.1 Evidence-based appraisal of the literature suggests that physiological tests for weaning success have low predictive power for most groups of critically ill patients. A recent systematic review concluded that the implementation of weaning protocols, particularly when carried out by non-physician health care workers, was likely to have a major impact on weaning outcomes.4 Successful implementation of weaning protocols requires criteria that enable staff to identify patients who have a high probability of being successfully disconnected from mechanical ventilation so that the weaning protocol can be initiated. Ideally these criteria should be simple, easy to collect, have low or insignificant cost, and predict subsequent weaning and extubation success. Such criteria are likely to be specific to individual ICUs because of heterogeneity in case mix and organization between units. We carried out a prospective, single-centre study to evaluate the ability of a checklist of simple cardiorespiratory, neurological, and metabolic criteria to predict outcome from mechanical ventilation. Our aim was to evaluate the possible utility of these criteria to trigger a nurse-led weaning protocol.


    Methods
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Patients and setting
All patients admitted to the ICU of the Royal Infirmary of Edinburgh, Scotland were eligible for daily assessment during the study period (1/4/01 to 11/9/01; 164 consecutive days). Data were collected as a prospective audit, and as no interventions occurred the local ethics committee did not consider relative/patient consent necessary. The 12 bed general ICU admits about 700 patients annually comprising an approximate mix of 50% surgical (including about 50 liver transplants annually), 40% medical, and 10% trauma. The only planned admissions to the ICU are post-liver transplant patients; all other admissions come from emergency referrals. The unit does not routinely manage post-cardiac surgery patients or isolated neurotrauma. All admissions have data entered into the Scottish Intensive Care Society (SICS) Audit Group database.6 This was used to cross check completeness of patient data. The annual ICU audit report for the year immediately preceding the study showed the following unit characteristics: 676 admissions with a median (range) APACHE II score of 20 (3–51). Hospital mortality was 39.9% (SMR 1.08, measured against APACHE II prediction).

Weaning criteria
We used simple bedside criteria regarding metabolic, cardiovascular, respiratory, and neurological status to assess patients (Table 1). The aim of the criteria was to provide a checklist that, if all fields were met, suggested that acute physiological derangement had improved sufficiently to consider starting the weaning process. We termed these ‘weaning criteria’ and considered that the criteria were met when all criteria were present simultaneously. The criteria were developed locally by a process of literature review and by consensus. First, a group of two consultants and two experienced nurses reviewed published studies, in particular systematic reviews of the literature.3 5 Secondly, a checklist was derived pragmatically and discussed generally among ICU staff. Our original criteria had a higher level of haemoglobin concentration (9 g dl–1) and a lower level of PEEP (5 cm H2O) than most staff considered necessary for weaning. We therefore modified the original criteria for these items. The final criteria included a PEEP level <=10 cm H2O, reflecting current trends to maintain high levels of PEEP7 in patients recovering from acute lung injury, and a haemoglobin concentration >=7 g dl–1, reflecting evidence suggesting that restrictive transfusion practice does not prolong ventilation requirements in the critically ill.8


View this table:
[in this window]
[in a new window]
 
Table 1 The weaning checklist used
 
Data collection
Patient’s charts were examined on each day of admission by one of the authors (S.D. or F.McA.), and the presence or absence of each weaning criterion noted. Daily data for all admissions were entered onto a spreadsheet (Excel, version 4.0, Microsoft Corp.) for subsequent analysis. The day on which complete ventilator independence was successfully achieved was recorded, together with the day of discharge or death. For patients who required re-intubation and ventilation, the day of ventilator independence was the day on which ventilator independence was achieved for the final time during that ICU admission. Re-admissions to intensive care for mechanical ventilation were treated as separate ventilation episodes. Patients ventilated via a tracheostomy tube were considered ventilator independent when they were no longer connected to the ventilator.

Existing weaning practice
At the time of the study, weaning decisions were largely physician led. No formal screening protocol for suitability for weaning was used in the unit. Instead, medical staff assessed patients as part of routine daily review. Weaning strategies were individualized to patients, and extubation was only carried out after consultation with medical staff.

Analysis
We calculated the sensitivity and specificity, positive and negative predictive values, and positive and negative likelihood ratios (LRs) for the weaning criteria to predict successful ventilator independence during ICU stay. We also calculated the positive LRs depending on the day from admission that weaning criteria were assessed, and plotted the time taken from first meeting the weaning criteria to achieving ventilator independence.


    Results
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
There were 325 admissions to the ICU during the study period. No data were available for eight (2%) admissions; a further nine (3%) admissions were evaluated, but were never intubated and mechanically ventilated. The characteristics of the patients studied were: median age 61 (range 12–92) yr, mean APACHE II score 20.7 (range 2–50), ICU mortality 28%, median ICU length of stay 1.7 days. Data from the Scottish Intensive Care Society Audit Group database showed the following characteristics for the eight patients for whom weaning data were not available: mean APACHE II 18.1 (range 7–36; 1 readmission); median ICU length of stay 5.5 days; and all these patients survived ICU.

Data concerning weaning criteria were evaluated for the 308 admissions that were mechanically ventilated on admission to the ICU. A description of ventilation outcomes with reference to weaning criteria for these admissions is shown in Figure 1.



View larger version (20K):
[in this window]
[in a new window]
 
Fig 1 Outcomes in relation to the weaning criteria checklist for the patients studied.

 
Predictive value of the weaning criteria
We examined the ability of the weaning criteria to predict subsequent ventilator independence. For this analysis we excluded admissions that were discharged ventilator dependent to other facilities, because ultimate ventilation outcome, which was the reference standard, was unknown for these individuals. These 14 patients were all transferred to ICUs in other hospitals because of ICU bed shortages. After these exclusions, a total of 180 admissions met the weaning criteria and 114 admissions did not during their ICU stay (Fig. 1). Two patients were classified as becoming ventilator independent, but never met the weaning criteria, because they were extubated as part of terminal care. These were excluded from the analysis. The ability of weaning criteria to predict ventilator independence for the remaining 292 admissions is summarized in Table 2.


View this table:
[in this window]
[in a new window]
 
Table 2 Ability of weaning criteria to predict ventilator independence
 
Importance of the day on which weaning criteria first met
The ventilation outcomes for the patients who met the weaning criteria, subdivided into the day on which weaning criteria were first met, are described in Figure 2. These data indicated that the majority of admissions achieved ventilator independence within 1 day if weaning criteria were first met on day 1 of admission, but were less certain when weaning criteria were first met on day 2 or more.



View larger version (20K):
[in this window]
[in a new window]
 
Fig 2 Duration of mechanical ventilation in relation to the day from admission that the weaning criteria were first met.

 
We calculated positive LR for the test for data overall and by the day from ICU admission on which a patient was assessed. For this analysis, we excluded patients who were discharged from our unit ventilator-dependent (n=14) for the reasons described above, but included all other patients. We then considered all patients who were in the ICU on each day from admission. For example, for calculating the LR on day 2, we excluded patients with a length of stay of less than/equal to 1 day (n=41), and examined the remaining patients in the dataset (n=250). Any patient who met the test criteria on days 1 or 2 or both (n=91) was considered as ‘test positive’. LRs are presented for the test performed on various days from ICU admission in Table 3.


View this table:
[in this window]
[in a new window]
 
Table 3 Positive likelihood ratios of the test depending on the day from ICU admission on which the patient was examined. Data for patients who were discharged ventilator-dependent were excluded from the analysis
 
Patients who met weaning criteria, but were subsequently re-intubated
Of the 178 patients who met the weaning criteria, nine patients (5%) required re-intubation within 48 h of first extubation. Eight of these patients first met the criteria in more than 4 days from admission. Only one of these patients died during their ICU stay.

Patients who met weaning criteria, but failed them on at least one subsequent ICU day
Of the 178 patients who met the weaning criteria there were 17 (9.6%) who did not require subsequent re-intubation, but failed the criteria on at least 1 day after meeting the criteria for the first time. Twelve of these 17 patients did not first meet the criteria for more than 4 days after admission (median (IQR, range) day first met criteria: 8 (3.9; 0–34) days). Only one of these patients (who first met criteria after 34 days) subsequently died in ICU.

Patients who achieved ventilator independence, but never met weaning criteria
The 35 patients who were ventilated at ICU admission, achieved ventilator independence, but who never fulfilled the weaning criteria had a median (1st, 3rd quartile; range) number of days from admission to achieving ventilator independence of 2 (1, 3; 1–18) days. The median ICU length of stay of these patients was 4 (2, 6; 1–19) days. The reasons for not meeting the weaning criteria on the day ventilator independence was achieved are summarized in Table 4.


View this table:
[in this window]
[in a new window]
 
Table 4 Reasons for not meeting the weaning criteria at the assessment preceding ventilator independence, for those patients who achieved ventilator independence without ever meeting weaning criteria (n=35). All values number (%)
 

    Discussion
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
We have assessed the ability of simple criteria to predict ultimately successful discontinuation of mechanical ventilation. Our aim was to evaluate this as a checklist that the ICU nurse could use to identify patients who have reached a stage in their critical illness at which initiating the process of weaning should be considered. The rationale for the study was that weaning protocols have been shown to decrease ventilation time, the associated length of ICU stay, and the cost of ICU care.1 For these to occur effectively, ICU nurses or other non-physician workers need a simple tool that gives them confidence that a patient has a high probability of achieving ventilator independence.

Jaeschke and colleagues have considered how to assess the value of a weaning test in critically ill patients.9 They suggested that study data should be examined in a systematic manner using a number of questions.

Was there an independent, blind comparison with a reference standard?
The reference standard in our study was whether the patient was ultimately successfully weaned from mechanical ventilation. We defined this as disconnection from the ventilator, and failure to wean as death without disconnection from the ventilator. We excluded patients who were discharged ventilator-dependent to other acute ICUs as a result of ICU bed shortages, because ultimate weaning outcome in these individuals was not known. We believe that the reference standard used in the study was reasonable, although it was impossible to blind investigators from this outcome. The weaning checklist data were collected independently and clinicians and nursing staff weaning the patients did not see the checklist results. However, complete concealment of the data was not possible or ethical and it is likely that clinicians used some of these data when they made ventilation decisions. Collecting the checklist data independently was the most feasible method of blinding and minimized the chance of bias.

Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test would be applied in clinical practice?
We designed our study to include all patients admitted to the ICU during the study period. We validated the completeness of our dataset against an independent audit database that tracks all patients admitted to the ICU.6 We obtained data on 98% of eligible admissions, and characterized the admissions of the eight patients for whom no data were available. We therefore consider selection bias extremely unlikely. We also characterized the illness severity (APACHE II diagnosis), physical characteristics, ventilation outcome, length of stay, and mortality of the cohort to enable comparisons with other settings. This should enable others to assess the applicability of our findings to other ICU populations.

Did the results of the test being evaluated influence the decision to perform the reference standard?
The properties of the weaning criteria test would be distorted if they had influenced weaning decisions during the study because this would have introduced verification bias.10 During the data collection, weaning criteria were recorded by an individual who was not involved in making weaning decisions, but as discussed above it was impossible to blind clinicians to the components of the checklist as this would have been impractical and unethical. Blinding of test information is a common confounding factor in studies of weaning criteria. Our question related specifically to the association between fulfilling all components of the checklist and ultimate ventilation outcome. Although it is likely that individual physiological variables were used by clinicians during patient evaluation, we think it unlikely that this had a major confounding effect on our reference standard.

What are the sensitivity, specificity, and likelihood ratios for the test?
We have presented the overall sensitivity and specificity of the test for correctly identifying patients who are successfully disconnected from the ventilator and discharged from the ICU. For nurse-led weaning a test is needed that is simple, quick, and gives the nurse at the bedside confidence that the patient will eventually become ventilator independent having achieved his/her current status. The ideal test should have a high specificity, because this allows a high degree of confidence that the outcome of interest, namely ventilator independence, will occur.11 In our population the weaning criteria had, overall, a moderately high specificity (89%), which should allow the ICU nurse to be confident that considering the patient for weaning is appropriate.

Sensitivity, specificity, and predictive values are of limited value for predicting outcomes in weaning studies because they generally apply to single cut-off points. We calculated LR, which allow an assessment of changes in pre-test probability.5 Our overall positive LR of 7.6 and negative LR of 0.18 correlate with clinically important changes in probability suggesting that the weaning checklist may be clinically useful. Plotting the number of patients who were still ventilator dependent in relation to the day from ICU admission on which the criteria were first met (Fig. 2) suggested that this factor was an important determinant of the value of the test. We therefore calculated positive LRs for the weaning criteria depending on the day from admission on which the test was performed. This showed that the test had a large effect on the post-test probability if weaning criteria were met on the first full ICU day, a moderate effect on the second day, and smaller effects on subsequent days. We have used these LRs to simulate the pre- to post-test probability changes at various levels of pre-test probabilities on different days of admission (Table 5). These changes were measured from a Fagan nomogram.12 This simulation clearly shows that if the criteria were met on days 1 or 2 from admission a clinically significant increase in post-test probability occurs irrespective of the estimated pre-test probability of ultimate ventilator independence. When met on day 3 or more the changes in probability are more modest, consistent with positive LRs of 2–4. These patients took longer to achieve ventilator independence, had a high incidence of re-intubation after first meeting the criteria (31%), and in many cases (46%) failed the weaning criteria on subsequent days. If our weaning criteria were used as part of a daily nursing checklist, broadly two groups of patients would be identified. First, patients who fulfil the criteria during the first 1–2 days of ventilation are weaned successfully and quickly in the majority of cases. These patients comprised 40% of all admissions during the study period. For these patients the weaning criteria could be used to trigger a reduction in sedation, a spontaneous breathing trial, or protocolized reduction in ventilator support, and subsequent extubation without involvement of physicians. Secondly, the group who first met the criteria in more than/equal to 3 days from ventilation had a more variable weaning duration. For these patients, the criteria could also be used to trigger a reduction in sedation and ventilatory support, but additional evaluation by medical staff, such as a formal assessment of respiratory pattern, cardiovascular responses and comfort, may be advisable. These patients may be less suited to a nurse-led protocol, but the weaning criteria could be used as a trigger to involve more experienced staff or a ‘weaning team’.


View this table:
[in this window]
[in a new window]
 
Table 5 Simulations of the effect the weaning criteria have on post-test probability of successful weaning at different levels of pre-test probability. Pre-test probabilities have been set at 20, 40, and 60% to reflect possible levels of probability in a variety of types of ICU patient
 
We found that about 18% of patients who were weaned successfully failed the weaning criteria at the assessment immediately preceding disconnection from the ventilator. This resulted in a relatively low sensitivity (84%) and negative predictive value (70%). For a test designed to guide patients towards an intervention (namely weaning), this false negative rate is only of concern if it results in unnecessary delays in the outcome of interest. In our patients, the commonest reasons for failing the weaning criteria immediately before ventilator independence were PaO2/FIO2 ratio, neurological status, and spontaneous respiratory effort in almost 70% of cases (Table 4). These patients most likely progressed rapidly to extubation because of rapid improvement in their condition and reduction in sedation level within the 24 h before the next weaning criteria were charted. For these patients, failing the criteria was not apparently associated with delayed weaning so it is unlikely that using the criteria as part of a protocol would introduce delays in recognizing patients ready to wean. In practice, the aim of criteria should be to encourage continual assessment rather than a checklist at a fixed time point.

We did not formally include a review of sedation in our study, but our data emphasize the importance of daily assessment of sedation status in conjunction with a weaning protocol. Daily cessation or reduction in sedation may have decreased the number of failures as a result of inadequate spontaneous respiratory effort or impaired neurological status. Previous studies have shown that daily sedation cessation reduces ICU length of stay.13 This approach is likely to be particularly effective if combined with regular assessment of weaning criteria to trigger a weaning protocol.

In conclusion, we have shown that a simple bedside weaning checklist can reliably predict patients who achieve ventilator independence, particularly if the day from ventilation on which these criteria were first met is considered. These criteria are a potential method of introducing nurse-led weaning protocols.


    Acknowledgements
 
Supported in part by Datex-Ohmeda and by the Royal Infirmary of Edinburgh Intensive Care Unit Research Fund.


    References
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
1 Keenan SP. Weaning protocols: here to stay. Lancet 2002; 359: 186–7[CrossRef][ISI][Medline]

2 Esteban A, Alia I. Clinical management of weaning from mechanical ventilation. Intensive Care Med 1998; 24: 999–1008[CrossRef][ISI][Medline]

3 Butler R, Keenan SP, Inman KJ, Sibbald WJ, Block G Is there a preferred technique for weaning the difficult-to-wean patient? A systematic review of the literature. Crit Care Med 1999; 27: 2331–6[ISI][Medline]

4 Meade M, Guyatt G, Sinuff T, et al. Trials comparing alternative weaning modes and discontinuation assessments. Chest 2001; 120 (6 Suppl.): 425S–37S

5 Meade M, Guyatt G, Cook D, et al. Predicting success in weaning from mechanical ventilation. Chest 2001; 120 (6 Suppl.): 400S–24S

6 Scottish Intensive Care Society Audit Group. www. scottishintensivecare.org.uk. Accessed May 20, 2002

7 Ware LB, Matthay MA. The acute respiratory distress syndrome. N Engl J Med 2000; 342: 1334–49[Free Full Text]

8 Hebert PC, Blajchman MA, Cook DJ, et al. Do blood transfusions improve outcomes related to mechanical ventilation? Chest 2001; 119: 1850–7[Abstract/Free Full Text]

9 Jaeschke RZ, Meade MO, Guyatt GH, Keenan SP, Cook DJ. How to use diagnostic test articles in the intensive care unit: diagnosing weanability using f/Vt. Crit Care Med 1997; 25: 1514–21[ISI][Medline]

10 Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983; 39: 207–15[ISI][Medline]

11 Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM, 2nd Edn. Edinburgh: Churchill Livingston, 2000; 67

12 Fagan TJ. Nomogram for Bayes’s theorum. N Engl J Med 1975; 293: 257[ISI][Medline]

13 Kress JP, Pohlman AS, O’Connor MF, Hall JB. Daily interruption of sedative infusions in critically ill patients undergoing mechanical ventilation. N Engl J Med 2000; 342: 1471–7[Abstract/Free Full Text]





This Article
Abstract
Full Text (PDF)
All Versions of this Article:
92/6/793    most recent
aeh139v1
E-Letters: Submit a response to the article
Alert me when this article is cited
Alert me when E-letters are posted
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Search for citing articles in:
ISI Web of Science (1)
Disclaimer
Request Permissions
Google Scholar
Articles by Walsh, T. S.
Articles by McArdle, F.
PubMed
PubMed Citation
Articles by Walsh, T. S.
Articles by McArdle, F.