1 MEMO (Medicines Monitoring Unit), Department of Clinical Pharmacology, University of Dundee, Dundee DD1 9SY; Departments of 2 Pharmacy, 3 Infectious Diseases and 4 Medical Microbiology, Ninewells Hospital, Dundee; 5 Department of Epidemiology and Public Health, University of Dundee, Dundee; 6 Health Services Research Unit, University of Aberdeen, Aberdeen, UK
Received 18 June 2003; returned 7 August 2003; revised 27 August 2003; accepted 28 August 2003
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods: The intervention is a policy for appropriate use of Alert Antibiotics (carbapenems, glycopeptides, amphotericin, ciprofloxacin, linezolid, piperacillintazobactam and third-generation cephalosporins) implemented through concurrent, patient-specific feedback by clinical pharmacists. Statistical significance and effect size were calculated by segmented regression analysis of interrupted time series of drug use and cost for 2 years before and after the intervention started.
Results: Use of Alert Antibiotics increased before the intervention started but decreased steadily for 2 years thereafter. The changes in slope of the time series were 0.27 defined daily doses/100 bed-days per month (95% CI 0.190.34) and £1908 per month (95% CI £1238£2578). The cost of development, dissemination and implementation of the intervention (£20 133) was well below the most conservative estimate of the reduction in cost (£133 296), which is the lower 95% CI of effect size assuming that cost would not have continued to increase without the intervention. However, if use had continued to increase, the difference between predicted and actual cost of Alert Antibiotics was £572 448 (95% CI £435 696£709 176) over the 24 months after the intervention started.
Conclusions: Segmented regression analysis of pharmacy stock data is a simple, practical and robust method for measuring the impact of interventions to change prescribing. The Alert Antibiotic Monitoring intervention was associated with significant decreases in total use and cost in the 2 years after the programme was implemented. In our hospital, the value of the data far exceeded the cost of processing and analysis.
Keywords: education, professional behaviour change, quality improvement
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Despite its importance, there is very little available information about hospital antibiotic consumption in the UK.10 Whereas GPs are generally equipped to quite a high level with information systems suitable for recording and analysing prescription data, UK hospitals are not. It is unlikely that many UK hospitals have information systems that allow easy assessment of their use of antimicrobials.7 It is notoriously difficult to manage what cannot be measured.7 Consequently, collection and analysis of hospital antibiotic prescribing data is a key component of the UKs Action Plan for Antimicrobial Resistance.11,12
In 2000, the Antibiotic Subcommittee of Tayside University Hospitals Trust devised an Alert Antibiotic Policy to reduce inappropriate use of key antibiotics, targeted because they should be reserved for infections caused by organisms that are resistant to first line antimicrobials. Our first evaluation of this intervention relied on analysis of patient records completed by the ward pharmacists. In total, 794 patient records were returned in the 2 years after introduction of the Alert Antibiotic Policy. The number of records declined from 41 per month in the first year to 25 per month in the second year. The proportion of prescriptions that were appropriate was similar in the 2 years (83% and 84%). We identified two fundamental problems with using patient records as a measure of outcome of the Alert Antibiotics Policy. First, patient records are not an objective measurement of outcome. Outcome measures such as appropriateness of care should be based on a questionnaire with known reliability and validity. Moreover, outcome measures that are obtained by chart extraction or collected by an individual require two or more raters with at least 90% agreement or kappa greater than or equal to 0.8.13 Second, patient records provided no information about practice before the start of the intervention, consequently it was impossible to say whether the reduction in number of records was the result of an impact of the intervention on the inappropriate initiation of Alert Antibiotics or simply to decline in return of records by the pharmacists.1315
The aim of this study was to use routine data from the pharmacy stock control computer to evaluate this intervention with an interrupted time series (ITS) analysis, in which a series of observations over time is interrupted by an intervention or treatment.16
![]() |
Materials and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The intervention, an Alert Antibiotic Policy, was implemented in Ninewells Hospital, a tertiary university hospital in Tayside, Scotland.
Design of the Alert Antibiotic Intervention
The design of our intervention to change prescribing was based on evidence about changing professional behaviour in the EPOC (Effective Practice and Organization of Care) section of the Cochrane Database.1723 From this evidence and an earlier systematic review of the literature on behaviour change,24 we identified three key elements for the intervention: development, dissemination and implementation. The Tayside Antibiotic Subcommittee developed the Alert Antibiotics Policy, the committee is multi-disciplinary, chaired by a surgeon and includes representation from junior staff and the clinical groups in addition to local opinion leaders. Our dissemination strategy was targeted at specific professionals and clinical teams via the clinical pharmacists. Our implementation strategy used immediate, concurrent feedback of information to the prescribers, who were contacted by the clinical pharmacist while their patient was still being treated.
The Policy targeted the following drugs:
(i) Carbapenems: imipenem and meropenem(ii) Glycopeptides: teicoplanin and vancomycin(iii) Intravenous (iv) amphotericin(iv) Ciprofloxacin (iv)(v) Linezolid (iv and oral)(vi) Piperacillintazobactam (Tazocin)(vii) Third-generation cephalosporins: ceftriaxone, cefotaxime and ceftazidime
The intervention went into effect in up to 40 medical and surgical wards of Ninewells Hospital, excluding haematology and paediatrics in August 2000, and is ongoing at this time. Guidelines for prescribing use were evidence based and agreed locally with the clinicians. Clinical pharmacists were asked to identify prescribing of the Alert Antibiotics in their wards and to confirm that their use was in line with the agreed indications. If it was not the pharmacists asked medical staff to review their choice of antibiotic and contact microbiology or infectious diseases physicians for advice if necessary.
Evaluation of the intervention
Segmented regression of interrupted time series analysis. Segmented regression analysis of interrupted time series data allows us to assess, in statistical terms, how much an intervention changed an outcome of interest, immediately and over time. When a separate control group is not available, analysis of the outcome of interest in the study group does not allow control for other events that may have influenced the outcome. Nonetheless, the level and trend of the pre-intervention segment serve as control for the post-intervention segment in single group time series, and still addresses important threats to internal validity and represent a methodologically acceptable design for measuring the impact of interventions. Correcting for autocorrelation avoids underestimating standard errors and overestimated significance of the effects of an intervention. For estimating seasonal autocorrelation, the autoregression model needs to evaluate correlations between error terms separated by multiples of 12 months. Accounting for seasonally correlated errors usually requires at least 24 monthly data points.25 Three outcomes in the ITS analysis are: first, change in level immediately after the intervention; second, difference between pre-intervention and post-intervention slopes; and third, the estimation of monthly average intervention effect after the intervention.16,25,26 The monthly average intervention effect estimates the monthly average changes in outcomes had the intervention not occurred,25 and can be estimated as the following:
(i) Outcome without the intervention = intercept at time 0 + (pre-slope x number of months)(ii) Outcome with the intervention = intercept at time 0 + (pre-slope x number of months) + ( slope x number of months since the intervention) +
level.25
Figure 1 is a graphic illustration of outcome measurements in segmented regression of interrupted time series analysis.
|
Linezolid and imipenem were excluded from the ITS analysis because there were less than 12 monthly data points before the intervention. The level of significance was 0.05.
Data collection
After evaluation of the intervention according to patient records and its shortcomings,14 we decided to use the pharmacy stock data. During the 4 year period of analysis no restriction policy for dispensing the Alert Antibiotics was implemented by the hospital pharmacy, therefore the pharmacy data about dispensed Alert Antibiotics would provide us with the best available independent indicator for evaluation of the intervention. The impact of the intervention on antibiotic prescribing was analysed using data about the quantity of Alert Antibiotics dispensed to the hospital wards per month for 2 years before and after the start of the intervention. A program was written for collation of data about dispensing of all dosage forms of the relevant drugs and conversion of the total grams dispensed into defined daily doses (DDD). The DDD as defined by the WHO Collaborating Centre for Drug Statistics Methodology is the assumed average dose per day for a drug used on its main indication in adults.27,28 The DDD was adjusted for bed occupancy and is presented as DDD/100 bed-days.
Cost analysis
As the average price of Alert Antibiotics dosage forms have decreased during the post-intervention period, we eliminated any effect of fluctuation in drug prices by using the average price of each dosage form over the 4 years of the study. Moreover, average bed-occupancy increased with time so we used the average bed occupancy over all 4 years in order to avoid overestimation of economic impacts of the intervention. The final results were measured with £/bed-days/month but presented in £/month.
The total cost of the Alert Antibiotic Policy and clinical pharmacy intervention was calculated by measuring the time required for consultation, completing the records, data entry and interpretation. The time required for additional meetings of the Antibiotic Policy Committee was quantified and additional consumable materials were recorded. Time was costed using the average hourly rate for each grade in each of the years of the intervention, including employers contributions and national insurance.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Before the intervention, both use and cost of Alert Antibiotics were increasing but after the intervention both use and cost declined (Figures 2 and 3). The intervention was associated with a significant change in level of use of teicoplanin and ceftazidime, but there was no significant change in level for all Alert Antibiotics combined (Table 1). There was a significant change in the slope so that overall use of all Alert Antibiotics decreased by 0.27 DDD/100 bed-days per month (95% CI 0.190.34, P < 0.0001). There were also significant decreases in slope for amphotericin, ciprofloxacin, piperacillintazobactam, teicoplanin, and vancomycin (Table 1). The slope of ceftriaxone usage increased significantly but there were no significant changes in slope for use of ceftazidime, meropenem or cefotaxime (Table 1). The results of statistical analysis of changes in cost were essentially similar to changes in use (Table 2). The analysis of change in slope showed a reduction in cost of Alert Antibiotics by £1908 per month in the 2 years after the intervention (95% CI £1238£2578; P < 0.0001).
|
|
|
|
A more conservative estimate of effect size was calculated from the difference between the post-intervention line and the last point on the pre-intervention line (Figure 1). This conservative estimate was a decrease in total use by 1.60 DDD/100 bed-days/month (95% CI 1.92.1, P <0.0001) (Table 1) and decrease in cost by £9590/month (95% CI £5554£13626, P <0.0001) (Table 2).
Costs of the intervention
The cost of the first year of the intervention, which included setting up the programs for extraction, formatting and analysis was £15 143 and the cost of running the intervention in the second year was £4990 (Table 3). The total cost of the intervention (£20 133) over the 2 years was therefore well below the most conservative estimate of the reduction in cost of Alert Antibiotics, which was £133 296 (the lower boundary of the 95% CI for change in slope after the intervention, £5554 per month times 24 months). However, assuming that the cost of Alert Antibiotics would have continued to increase without the intervention, the cost of Alert Antibiotics was estimated to have decreased by an average of £23 852 per month (95% CI £18 154£29 549, P <0.0001) (maximum estimate of reduction, Table 2).
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Measurement of hospital antibiotic use is a good start but needs to be supported by more detailed analysis of indications for use and outcomes. At present, collection of this information is laborious, requiring extraction from handwritten case records. The move towards electronic patient records is welcome but will not solve all of the problems inherent in the evaluation of antibiotic prescribing. Quality indicators require consensus about evidence-based clinical standards and about the information that must be recorded in order to evaluate compliance with standards.14 Audits from our hospital show that <50% of prescriptions for antibiotics are supported by any information about the indication for antibiotic treatment.15 These results are depressingly similar to an audit from the Central Middlesex Hospital over 20 years ago.29,30 Electronic medical records will be easier to analyse than paper records, but the value of the information will be questionable unless the records include standardized, evidence-based minimum datasets for recording the indications for and outcomes of antimicrobial treatment.
In conclusion, our results showed that pharmacy stock data are a goldmine for local antibiotic policy makers and hospital management. Adjustment of data by defined daily dose and for bed occupancy provide simple and objective measures of prescribing. These data combined with segmented regression analysis provide a strong, quasi-experimental design to evaluate the overall impact of interventions to change prescribing.25 Clearly these data do not provide information about indication for treatment or clinical outcomes. However, clinical pharmacists or other staff can use ward supply data as a method for identifying wards with unusual use and targeting more detailed audits. In our hospital, these data provide important support for the continuation of the Alert Antibiotic intervention and for targeting interventions to improve the quality of antibiotic prescribing.11,15,30 Having established the intervention and demonstrated its cost-effectiveness, we are now in a stronger position to request additional resources for analysis of time trends in microbiology results and indicators of clinical outcome, such as length of stay and readmissions.1,3133
The European Union has funded the European Surveillance of Antimicrobial Consumption, ESAC, (http://www.ua.ac.bemain.asp?c=*ESAC) to collect data from 31 countries of which 20 are able to provide national data about hospital use. The UKs current lack of hospital data is therefore the exception rather than the rule in Europe. Analysis of hospital antibiotic prescribing is practical and would address one of the key recommendations made by the House of Lords Select Committee on Science and Technology 5 years ago. In England, a Press Release on June 9th 2003 announced an investment of £12 million to allow Hospital Pharmacists to "monitor and control more carefully the use of antibiotics".34 Our results provide evidence that this investment is likely to result in net savings by the end of the 3 year period and we urge other UK hospitals to consider collection and analysis of these data. The data are available, the techniques required are not difficult and we would be happy to share our methods with others. Our software includes ATC (Anatomic, Therapeutic, Chemical) coding, to allow comparison with other hospitals in the ESAC project.
![]() |
Acknowledgements |
---|
![]() |
Conflicts of interest |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Appendix |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Calculation of expected use of vancomycin in month 30:
Baseline level = the intercept at time 0 = 0.185 DDD/100 bed-days.
Pre-slope = 0.039 DDD/100 bed-days/month.
Number of months = 30.
Number of months since the intervention = 6.
slope = 0.048 DDD/100 bed-days/month (Table 1).
level = 0.178 DDD/100 bed-days/month (Table 1).
At month 30 without the intervention, vancomycin use would be 1.355 DDD/100 bed-days (0.185+0.039 x 30) but with the intervention, it was 0.889 DDD/100 bed-days/month [0.185+0.039 x 30+ (0.048 x 6)0.178].
Therefore in month 30, we estimated that the Alert Antibiotics Monitoring Programme had resulted in a reduction in use of vancomycin by 0.466 DDD/100 bed-days, in comparison with expected use if the programme had not been implemented.
![]() |
Footnotes |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 . Appleby, D. H. & John, J. F., Jr (1980). Use, misuse, and cost of parenteral cephalosporins at a county hospital. Southern Medical Journal 73, 14735.[ISI][Medline]
3 . Day, D., Lubowski, T. J., Yamaga, C. et al. (1999). Computer-assisted evaluation of antibiotic regimen coverage and cost. Clinical Therapeutics 21, 141825.[CrossRef][ISI][Medline]
4 . Denton, M., Morgan, M. S. & White, R. R. (1991). Quality of prescribing of intravenous antibiotics in a district general hospital. British Medical Journal 302, 3278.
5 . Dunagan, W. C., Woodward, R. S., Medoff, G. et al. (1989). Antimicrobial misuse in patients with positive blood cultures. American Journal of Medicine 87, 2539.[ISI][Medline]
6 . Jogerst, G. J. & Dippe, S. E. (1981). Antibiotic use among medical specialties in a community hospital. Journal of the American Medical Association 245, 8426.[Abstract]
7 . House of Lords Select Committee on Science and Technology. (1998). Resistance to Antibiotics and Other Antimicrobial Agents. Session 199798, 7th Report, pp. 1108. The Stationery Office, London, UK.
8 . House of Lords Select Committee on Science and Technology. (2001). Resistance to Antibiotics. Session 200001, 3rd Report, pp. 134. The Stationery Office, London, UK.
9 . Standing Medical Advisory Committee, Subcommittee on Antibiotic Resistance. (1998). The Path of Least Resistance (Hart, M., Livermore, D. M. & Weinberg, J. R., Eds), pp. 1152. Department of Health, London, UK.
10 . Gould, I. M. & Jappy, B. (1996). Trends in hospital antibiotic prescribing after introduction of an antibiotic policy. Journal of Antimicrobial Chemotherapy 38, 895904.[Abstract]
11 . Department of Health, UK Antimicrobial Resistance Strategy and Action Plan. (2000). Department of Health [Online.] http://www.doh.gov.uk/arbstrat.htm (June 2003, date last accessed).
12 . Scottish Executive. (2002). Antimicrobial Resistance Strategy and Scottish Action Plan 14191. pp. 132. Scottish Executive, Edinburgh, UK.
13 . The Cochrane Effective Practice and Organisation of Care Group (EPOC) The Data Collection Checklist. (2002). Cochrane Effective Practice and Organisation of Care Group (EPOC) Methods Papers [Online.] http://www.epoc.uottawa.ca/ (June 2003, date last accessed).
14 . Nathwani, D., Gray, K. & Borland, H. (2002). Quality indicators for antibiotic control programmes. Journal of Hospital Infection 50, 1659.[CrossRef][ISI][Medline]
15 . Seaton, R. A., Nathwani, D., Phillips, G. et al. (1999). Clinical record keeping in patients receiving antibiotics in hospital. Health Bulletin 57, 12833.[Medline]
16 . Cook, T. D. & Campbell, D. T. (1979). Quasi Experimentation. Design & Analysis Issues for Field Settings, 1st edn, pp. 1405. Houghton Mifflin Company, Boston, MA, USA.
17 . Freemantle, N., Harvey, E. L., Wolf, F. et al. (2001). Printed educational materials: effects on professional practice and health care outcomes (Cochrane Review). Cochrane Library. Update Software, Oxford, UK.
18 . Grilli, R., Freemantle, N., Minozzi, S. et al. (1997). Mass media interventions: effects on health services utilisation (Cochrane Review). Cochrane Library. Update Software, Oxford, UK.
19 . Thomson OBrien, M. A., Oxman, A. D., Davis, D. A. et al. (1997). Audit and feedback versus alternative strategies: effects on professional practice and health care outcomes (Cochrane Review). Cochrane Library. Update Software, Oxford, UK.
20 . Thomson OBrien, M. A., Oxman, A. D., Davis, D. A. et al. (1997). Audit and feedback: effects on professional practice and health care outcomes (Cochrane Review). Cochrane Library. Update Software, Oxford, UK.
21 . Thomson OBrien, M. A., Oxman, A. D., Davis, D. A. et al. (1997). Educational outreach visits: effects on professional practice and health care outcomes (Cochrane Review). Cochrane Library. Update Software, Oxford, UK.
22 . Thomson OBrien, M. A., Oxman, A. D., Haynes, R. B. et al. (1998). Local opinion leaders: effects on professional practice and health care outcomes (Cochrane Review). Cochrane Library. Update Software, Oxford, UK.
23 . Walton, R. T., Harvey, E., Dovey, S. et al. (1997). Computerised advice on drug dosage to improve prescribing practice (Cochrane Review). Cochrane Library. Update Software, Oxford, UK.
24 . Grimshaw, J. M. & Russell, I. T. (1993). Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 342, 131721.[ISI][Medline]
25 . Wagner, A. K., Soumerai, S. B., Zhang, F. et al. (2002). Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics 27, 299309.[CrossRef][ISI][Medline]
26 . Matowe, L., Ramsay, C. R., Grimshaw, J. M. et al. (2002). Effects of mailed dissemination of the Royal College of Radiologists guidelines on general practitioner referrals for radiography: a time series analysis. Clinical Radiology 57, 5758.[CrossRef][ISI][Medline]
27 . WHO Collaborating Center for Drug Statistics Methodology. (2000). Guidelines for ATC Classification and DDD Assignment 2000. WHO Collaborating Center for Drug Statistics Methodology, Oslo, Norway.
28 . WHO Collaborating Center for Drug Statistics Methodology. (2000). ATC Index With DDDs 2000. WHO Collaborating Center for Drug Statistics Methodology, Oslo, Norway.
29 . Moss, F., McNicol, M. W., McSwiggan, D. A. et al. (1981). Survey of antibiotic prescribing in a district general hospital. II. Lower respiratory tract infection. Lancet ii, 4079.[CrossRef]
30 . Moss, F., McNicol, M. W., McSwiggan, D. A. et al. (1981). Survey of antibiotic prescribing in a district general hospital. III. Urinary tract infection. Lancet ii, 4612.
31 . Frank, M. O., Batteiger, B. E., Sorensen, S. J. et al. (1997). Decrease in expenditures and selected nosocomial infections following implementation of an antimicrobial-prescribing improvement program. Clinical Performance and Quality Health Care 5, 1808.[Medline]
32 . Gross, R., Morgan, A. S., Kinky, D. E. et al. (2001). Impact of a hospital-based antimicrobial management program on clinical and economic outcomes. Clinical Infectious Diseases 33, 28995.[CrossRef][ISI][Medline]
33 . Landman, D., Chockalingam, M. & Quale, J. M. (1999). Reduction in the incidence of methicillin-resistant Staphylococcus aureus and ceftazidime-resistant Klebsiella pneumoniae following changes in a hospital antibiotic formulary. Clinical Infectious Diseases 28, 10626.[ISI][Medline]
34 . Department of Health. (2003). CMO to step up fight against hospital infections: £12m for hospital pharmacists and central monitoring extended; new strategy to be published this Autumn. Department of Health Press Release: reference 2003/0222 [Online.] http://www.info.doh.gov.uk/doh/intpress.nsf/page/20030222?OpenDocument (June 2003, date last accessed).