Unrecognized malfunction in computerized patient simulators

Editor—Life-like computerized patient simulators are now widely use in clinical training. In addition to their established role in crisis resource management education,1 2 simulators are also being used to teach pre-clinical physiology3 and to evaluate clinical performance.4 The widespread adoption of simulation-based crisis resource management training is a direct analogy to similar training in aviation and has been driven by the need to provide education that should reduce both medical error and risks to patients.5 This growth in medical simulation has been facilitated by the commercial availability of simulators, but it has taken place without independent verification of simulator performance characteristics and without defined standards of simulator adequacy. This contrasts with the aviation experience where standards for simulation are defined in advisory circulars and regulations.6 Such standards are intended to ensure that the level of simulation fidelity meets the goals of training and evaluation; thereby ensuring uniformity, while minimizing the risk of negative transfer (whereby inappropriate learning in a poor quality simulation could transfer to real life).

In an attempt to independently verify the performance characteristics of one brand of commercially available simulators, we undertook a multi-site comparison of the performance of six fully operational METI (Medical Education Technologies Inc., Sarasota, FL, USA) simulators. The METI Human Patient Simulator (HPS) is a plastic manikin with respiratory and cardiovascular functionality. This includes mechanical lungs, palpable pulses, heart and lung sounds; with life-like cardiorespiratory data that are displayed using standard clinical monitors. The HPS is operated by a computer program with sophisticated physiological and pharmacological models that rely on feedback from a range of peripheral devices. Clinical problems or scenarios are created by manipulating these integrated physiological and pharmacological models using METI-HPS scenario files.

To generate physiological data for comparison, we distributed an identical scenario file to the participating centres. Five of the simulators were METI-C simulators running software version 5.5, and simulator 1 was a METI-B running software version 5.2. The simulators were all breathing room air and half were intubated with a tracheal tube. The scenario began with a 2 min physiologically normal baseline, followed by automatic transitions that included progressive haemorrhage and increases in both shunt fraction and oxygen consumption. The final transition to total neuromuscular blockade resulted in death from hypoxaemic cardiac arrest. Three iterations of the scenario were run on each simulator. At 1 min intervals the simulated physiological data were automatically recorded to log files. Data were analysed using repeated measures of analysis of variance using SYSTAT version 10.

The data from simulator 6 were strikingly different from the others, caused by an unrecognized defective gas analyser that was impairing the physiological model. These data were excluded from the statistical analysis. The physiological trends recorded from the five remaining simulators were similar, but there were statistically significant differences (P<0.0001) between simulators for heart rate, systolic and diastolic blood pressure, cardiac output, tidal volume, ventilatory frequency, alveolar partial pressure of oxygen and carbon dioxide, and arterial partial pressure of carbon dioxide. The differences were greatest for tidal volume and are shown in Figure 1. There were no statistically significant differences within any one simulator during the repeated iterations of the scenario.



View larger version (21K):
[in this window]
[in a new window]
 
Fig 1 Variation in tidal volume found during an 8 min scenario. Three iterations are overlaid for each simulator. Simulators 1 and 3 were unintubated, simulator 2 was intubated for both the automatic start-up calibration and the scenario, simulators 4 and 5 were intubated for the scenario but after calibration. Simulator 6 was unintubated and was malfunctioning. All six simulators had reproducible performance.

 
We were surprised by the variability that we found. One simulator, although in use and apparently functioning normally, had a significant hitherto unrecognized malfunction. The five remaining simulators showed variability that although life-like, was greater than we had expected from computerized simulators. We assume that some of this variability may have been caused by the presence of a tracheal tube, but this remains unproven.

The important finding was an unrecognized malfunction in a fully operational simulator. This is important when considering the educational and evaluative roles that simulators are likely to play in the future. We must be confident that simulators are both reliable and valid, not only because of the need to avoid negative transfer and potentially harmful training, but also in the interests of fairness to examination candidates when simulators are used in certification and re-certification. Our initial question regarding the variability in simulated physiological performance in properly functioning simulators remains inadequately answered.

A. L. Garden1, B. J. Robinson2, C. U. Arancibia3, T. J. Carron3, S. Monk4, J. Vollmer4, W. Heinrichs4, C. Grube5, B. M. Graf5 and E. B. Johnson3

1 Boston, MA, USA 2 Wellington, New Zealand 3 Richmond, VA, USA 4 Mainz, Germany 5 Heidelberg, Germany

Acknowledgments

We would like to thank Tom Boeker MD, Heidelberg and the Bristol Medical Simulation Centre for their assistance.

References

1 Howard SK, Gaba DM, Fish KJ, Yang G, Sarnquist FH. Anesthesia Crisis Resource Management Training: teaching anesthesiologists to handle critical incidents. Aviation Space Environmental Medicine 1992; 63: 763–70[ISI][Medline]

2 Lee SK, Pardo M, Gaba D, et al. Trauma assessment training with a patient simulator: a prospective, randomized study. J Trauma 2003; 55: 651–7[ISI][Medline]

3 Zvara DA, Olympio MD, MacGregor DA. Teaching cardiovascular physiology using patient simulation. Academic Medicine 2001; 76: 534[Free Full Text]

4 Rosenblatt MA, Abrams KJ. The use of a human patient simulator in the evaluation of and development of a remedial prescription for an anesthesiologist with lapsed medical skills. Anesth Analg 2002; 94: 149–53[Abstract/Free Full Text]

5 Garden A, Robinson B, Weller J, Wilson L, Crone D. Education to address medical error—a role for high fidelity patient simulation. N Zealand Med J 2002; 115: 132–4

6 Federal Aviation Administration. 14 CFR, Chapter I, Part 121, Appendix H. Advanced Simulation, 1997; 1–8





This Article
Full Text (PDF)
E-Letters: Submit a response to the article
Alert me when this article is cited
Alert me when E-letters are posted
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Disclaimer
Request Permissions
Google Scholar
Articles by Garden, A. L.
Articles by Johnson, E. B.
PubMed
PubMed Citation
Articles by Garden, A. L.
Articles by Johnson, E. B.