From evidence-based guideline methodology to quality of care standards

Dilip Nathwani*

Infection Unit (Ward 42), Ninewells Hospital & Medical School, Dundee DD1 9SY, Scotland, UK

Keywords: evidence-based medicine, guidelines, quality of care


    Introduction
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
The culture of searching for evidence in medicine has revolved around three areas—the systemic–pathophysiological, the individual–clinical and the statistical–analytical approach.1 The origin of the first lies with Socrates (c. 400 BC) and Galen (c. AD 200), the second with the long evolution of the process we call ‘medical judgement’ and the last with the work of a number of pioneers such as James Lind, a Scottish naval surgeon (1716–1794), who validated his intuition about the best treatment for scurvy by undertaking a prospective randomized study on board a British naval ship.2 This and other work were the beginnings of systematic and analytical evaluation of evidence as a basis for clinical practice.


    Evidence-based guidelines
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
The last two decades have seen the emergence of evidence-based medicine (EBM), which regards rationale that is institutional, clinically unsystematic and pathophysiological as insufficient for clinical decision making, and stresses examining evidence from clinical research.3 This approach has stimulated the production of evidence-based clinical practice guidelines, which are defined as systematic statements developed to assist decisions by practitioners and patients about appropriate healthcare for specific clinical circumstances.4 In short, they are designed to help clinicians ‘do the right thing’. Importantly, they must also reflect the routine working practices of most doctors for them to be accepted as a gold standard.


    Recommended guideline development methodology
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
One of the key components of a successful or good guideline is the rigour of the development process.5 Unfortunately, most of the guidelines produced by specialist Societies do not meet the basic principles of guideline development. One important study reported that 67% of guidelines did not describe the type of stakeholders involved, 88% gave no information on how published studies were searched for and 82% gave no explicit grading of the strength of the recommendations.6 Of the 431 eligible guidelines considered between January 1988 and July 1998, very few appeared to be geared towards infection, although the exact numbers were not explicit in the paper. Another study7 revealed that the methodological quality of guidelines is generally poor and often exhibits great variation and conflicting recommendations. This study, of 279 guidelines published from 1985 to 1997, revealed a mean adherence to methodological standards on guideline development of 51.1%.

There is clearly a need for a common, international, valid and transparent approach to the development of good clinical practice guidelines.8,9 The AGREE (Appraisal of Guidelines, Research and Evaluation for Europe; http://www.agreecollaboration.org) collaboration has developed a European agreed generic methodology to assess the quality of guidelines and the guideline development process. The Scottish Intercollegiate Guideline Network (SIGN, http://www.sign.ac.uk) has long adopted the majority of these criteria and has recently updated many aspects of SIGN methodology, which are consistent with the AGREE criteria. SIGN 50 is an excellent and commendable resource of the guideline process for any group wishing to embark upon this.8 All information is available on the websites.

A more recent comparison of 18 clinical guideline programmes has confirmed a significant improvement in guideline development throughout the world, but once again the data emphasize the importance of a continuing international effort to encourage worldwide adherence to this process.10 Guideline developers in the infection community throughout the world need to be encouraged to embrace this process and to consider these issues when planning future guidelines.

The current guidelines for hospital- and ventilator-associated pneumonia being undertaken by a working group of the British Society for Antimicrobial Chemotherapy (BSAC) appear to have taken these important considerations on board.


    Other approaches to guideline development
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
Unfortunately, old habits die hard and many clinicians or organizations persist with supporting non-standardized ad hoc statements or reviews, usually from expert bodies. This undesirable methodology or GOBSAT (Good Old Boys Sat Around a Table) should no longer be valid or encouraged.11 These statements are based on received wisdom, clinical judgement and experience rather than current scientific evidence, lack an explicit decision making process and may be biased by undeclared conflicts of interest.11 A good example of this are the guidelines for improving the use of antimicrobial agents in hospitals—a statement by the Infectious Diseases Society of America (IDSA) published in 1988.12 Although a well respected document, commendable for aiming to promote good quality prescribing, the development process was not evidence based as it was drafted by three authors with subsequent review by 43 multidisciplinary members of the IDSA. Other deficiencies of this document were the unhelpful format, lack of summary of evidence and no linkage of this to the ultimate recommendations. This paper is a typical example of the flawed development methodology in use before EBM methodology.

‘Consensus’-based statements13 are popular and involve a broad-based panel who listen to the scientific data presented by experts, weigh the information and then compose a consensus statement that addresses a set of questions previously posed to the panel. Within this relatively small group, the interactions are such that some members will have a significant impact on the overall decisions. Other sources of bias include the type of questions set, the composition of the panel, the selection of the experts and the literature. Two examples of such a process are the consensus panel recommendations for managing serious candidaemia,14 and the synthesis of a consensus strategy for combating the prevention and control of antimicrobial-resistant microorganisms in hospital.15 The latter document is recognized as a good consensus statement: it aims to synthesize a strategy from expert opinion, experience and key existing evidence. It is explicit in its intention and development process and recognizes its inherent deficiencies. This document is acknowledged widely as pivotal in the building of an evidence-based approach to preventing antimicrobial resistance in hospitals. This is recognized in subsequent guidelines on the subject by the Society for Healthcare Epidemiology of America (SHEA) and the IDSA Joint Committee on the Prevention of Antimicrobial Resistance.16 However, this guideline approach, although valuable, does not follow the AGREE or SIGN methodology. When the AGREE instrument for appraising the quality of this guideline is applied (www.agreecollaboration.org, June 2001) a number of deficiencies in the areas of stakeholder involvement, rigour of development and clarity of presentation can be identified. This guideline is more in the form of a specialist but not systematic review, with only one table linking four recommendations to evidence. Greater adherence to AGREE methodology would be welcomed.


    Variability of quality of infection practice guidelines
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
There has been a plethora of documents on infection practice over many years from specialist Societies, or other well-respected bodies that have claimed ‘guideline status’, despite a significant absence of methodological rigour. Although it is arguable that some of these documents have been well accepted by the broad infection community, for example the first British Thoracic Society Community Acquired Pneumonia Guidelines17 and the BSAC Endocarditis Guidelines,18 both of which have considerably influenced ‘standard’ practice, one has to question the evidence base of some of these recommendations. This has recently been highlighted in the management of bacterial endocarditis.19 However, more encouraging signs are appearing. In the last 3–4 years, major infection and associated groups have been warming towards the EBM approach. This has been indicated in the IDSA guideline development standards20 for a profusion of evidence-based guidelines, the updated British Thoracic Society guidelines for community-acquired pneumonia (CAP),21 and the SIGN guidelines on acute sore throat and tonsillectomy (SIGN 34), management of genital Chlamydia trachomatis infection (SIGN 42), surgical antibiotic prophylaxis (SIGN 45) and those for management of lower respiratory tract infection (SIGN 59). The follow-up IDSA guidelines for managing candidiasis confirmed the adoption of this procedure in North America.22 The value of this more evidence-based approach is appreciated by clinicians. For example, in an important study by Farquhar et al.,23 clinicians’ attitudes to clinical practice guidelines were found to be generally favourable as they believed the guidelines were educational and would improve quality. A recent survey of Australian Intensivists and ID physicians revealed similar positive attitudes towards evidence-based antibiotic use in critical care.24 However, in the systematic review by Farquhar et al.,23 significant concerns were expressed about guideline practicality, their role in cost-cutting and their potential for increasing litigation. Italian physicians25 in secondary and primary care share these concerns, and appear to suggest poor attitudes towards guidelines in general and the proposed methodology for guideline development in particular. One fears that this scepticism may not be confined to Italian physicians only.


    Guideline implementation
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
A crucial question is how best to implement EBM guidelines so that a change in practice is made and sustained. There are many determinants governing changes in physician behaviour, and these appear to be dependent on improving knowledge, allowing a change in attitude, and a number of organizational, social and personal factors.26 Two recent reviews have addressed the implementation strategies used to effect a positive response to antibiotic guidelines.27,28

Ultimately, regardless of the optimal implementation strategy, if we believe guidelines improve the clinical and cost effectiveness of healthcare,27 then we need to ensure good compliance using internal and external quality assurance systems within any organization. There is increasing study into the best way of achieving good adherence to guidelines’ recommendations, the basis of which are evidence based.29 This process is regarded as at least equal to, if not more important than, ensuring the successful impact of guidelines and should form an important component of the guideline document.


    Quality assurance and development of standards
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
One of the key components of a guideline is the identification or setting of key standards and criteria for audit. These clinical standards, based on existing evidence, would be subsequently used as the criteria for evaluating the quality of care provided by an organization, individual unit or department. Such quality assurance may be undertaken through an internal or external peer review or through an accreditation process.30,31 In the UK, as in Australia, guidelines and clinical standards underpin much of the quality, or more recently, the clinical governance agenda.32 This process aims to make it a statutory responsibility that each organization be accountable for ensuring the monitoring and improving the quality of the healthcare it provides.32

Clinical standards primarily identify the essentials in the treatment of particular conditions if outcomes are to be optimized. Standards can be set at several levels: minimal, normative and exemplary.30 It is important to recognize what level of standards should be applied, as minimal standards are primarily aimed at promoting basic levels of care by identifying those areas or professionals who require remedial, or in rare cases punitive, action. Outcome-related standards are seen as the gold standard of performance measurement, but in reality they are difficult to capture, particularly in the short term.33,34 Increasingly, process or, to a lesser extent, structure measures are deemed more attractive, especially if they are linked through evidence to outcome.35 Indeed, guidelines or care pathways will outline intervention or processes of care that lead to a desired outcome.33 The timely (within 4–8 h of admission) administration of appropriate intravenous antibiotics for patients with severe CAP is regarded as a key quality indicator.36 This intervention is regarded as an important, validated, credible, consistent, simple and measurable process standard, based on evidence that is valued by clinicians, quality administrators and patients.36 A CAP audit in Tayside used this intervention as one of the key performance indicators in prospectively evaluating pneumonia care. This study revealed that a significant 39% of patients admitted with severe CAP did not receive antibiotics within the appropriate timeframe. Indeed, 29% did not receive intravenous antibiotics within 24 h of hospital admission.37 Poor performance on a process measure gives a clear indication of the remedial action that is required, and this can be linked to an incentive to bring about positive change. This audit in Dundee stimulated broad educational feedback and the development and implementation of a care pathway for CAP38 that will be subject to further evaluation (G. Barlow, personal communication). On the other hand, a commonly used crude outcome marker of death is more difficult to interpret as it is insensitive to the quality of healthcare received and can be influenced by a range of other factors.39 In our CAP audit38 compliance with the unit protocol appeared to be correlated with a reduction in mortality, but the association was by no means robust.


    Development of clinical standards in Scotland
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
In Scotland, the development of clinical standards has been undertaken by the NHS Quality Improvement Scotland (www.nhshealthquality.org) and is subject to audit by means of an internal self assessment followed by external review. This process aims to increase and promote greater public confidence in the overall standard of care.31 Standards represent an agreed level of performance and this level should be determined by those who are involved in delivering or receiving the service. The criteria attached to each standard provide detailed and practical information on how to achieve the standard and can be described as structure, process and outcome criteria. These standards focus on the patient journey, the care and treatment the patient receives and relate very closely to patient outcome. They are underpinned by evidence base, may be specific or generic and are categorized as being desirable or essential.31 An evaluation of national performance against healthcare-acquired infection standards has been published recently (www.nhshealthquality.org). The methodology adopted in developing these standards is worthy of greater attention and should form the core principles of any standard developmental methodology. However, other methods of developing a consensus of recommendations, based on evidence and good practice, are also popular and valued. Recommendations for the management of atrial fibrillation in hospital and primary care is an example of such work.40 The Scottish Infection Standards and Strategy Group (SISS; available at www.rcpe.ac.uk site), a specialty subgroup of the Bicollegiate Quality of Care Committee, has recognized this and other methodology and has developed good practice recommendations for optimizing the quality of prescribing in hospitals. These core recommendations adopt and adapt work based on existing evidence and national strategies aimed at optimizing antibiotic prescribing in hospitals.15,16,41,42 They represent a mixture of standards aimed at evaluating the process, and organizational interventions related to hospital antibiotic prescribing. Further development and validation of traditional and more ‘patient-based’ outcomes in clinical outcomes research and standards methodology is clearly also necessary.43 Until then it is my belief that these SISS recommendations by no means represent the final product but form the basis of a process that will continue to evolve and adapt in line with validated standard development methodology. It is hoped that such standards will become an established national tool for benchmarking the quality of antibiotic prescribing in hospitals, with the objective of encouraging change where suboptimal practice is identified. This process is a key requirement of the UK Antimicrobial Resistance Strategy.42


    Conclusion
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
The infection community needs to embrace evidenced-based guideline methodology as a means of ensuring one of the key components of the validity and success of guidelines. However, one must not underestimate the pivotal role of guideline dissemination and implementation in bringing about positive change. Consensus-based approaches may be useful in specific situations, especially where adequate evidence does not exist, but GOBSAT statements should be avoided, regardless of the stature of the specialist body or its members. Implementation strategies ought to be developed in parallel with this process so that they are effective at the clinical ‘coal face’. Standards of care emerging from evidence-based clinical practice guidelines, or consensus-based good practice recommendations for antibiotic prescribing in hospitals, are a helpful basis for quality assurance nationally and within organizations. The infection community needs to develop these standards further so they are applicable nationally, if not internationally, but with the important caveat that they ought to be realistic and achievable locally. Measuring compliance with standards should be promoted through audit, education and feedback, but may be required to be enforced through internal or external systems of governance. The development of an outcome-based national hospital-prescribing database or registry44 using readily available clinical datasets for this purpose should underpin this process.

Acknowledgements

I wish to thank all members of the SISS working group, especially Robert Laing and Ian Gould, for their interest and help with this work.


    Footnotes
 
* Tel: +44-1382-660111; Fax: +44-1382-816178; E-mail: dilip.nathwani@tuht.scot.nhs.uk Back


    References
 Top
 Introduction
 Evidence-based guidelines
 Recommended guideline...
 Other approaches to guideline...
 Variability of quality of...
 Guideline implementation
 Quality assurance and...
 Development of clinical...
 Conclusion
 References
 
1 . Trohler, U. (2001). The history of clinical effectiveness. Proceedings of the Royal College of Physicians of Edinburgh 31, Suppl. 9, 42–5.[Medline]

2 . Trohler, U. (1981). Towards clinical investigation on a numerical basis: James Lind at Haslar Hospital 1758–1783. In Proceedings of the Twenty-seventh International Congress of Medicine, Barcelona 1, 414–19. Academia de Ciences Mediques de Catalunya I Balears, Barcelona, Spain.

3 . Evidence-Based Medicine Working Group. (1992). Evidence-based medicine. A new approach to teaching the practice of medicine. Journal of the American Medical Associaion 268, 2420–5.[Free Full Text]

4 . Field, M. J. & Lohr, K. N., Eds. (1990). Institute of Medicine Committee to Advise the Public Health Service on Clinical Practice Guidelines. Clinical Practice Guidelines: Directions for a New Program. National Academy Press, Washington, DC, USA.

5 . Feder, G. (1998). Guidelines for clinical guidelines. British Medical Journal 317, 427–8.[Free Full Text]

6 . Grilli, R., Magrini, N., Penna, A., Mura, G. & Liberati, A. (2000). Practice guidelines developed by specialty societies: the need for critical appraisal. Lancet 355, 103–6.[CrossRef][ISI][Medline]

7 . Scottish Intercollegiate Guidelines Network (SIGN). (2001). SIGN 50—A Guideline Developers’ Handbook. SIGN, Edinburgh, Scotland, UK.

8 . Shaneyfely, T. M., Mayo-Smith, M. F. & Rothwangl, J. (1999). Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. Journal of the American Medical Association 281, 1900–5.[Abstract/Free Full Text]

9 . Cluzeau, F. for the AGREE Collaborative Group. (2000). Guideline development in Europe: an international comparison. International Journal of Technology Assessment in Health Care 16, 1036–46.

10 . Burgers, J., Grol, R., Klazinga, N., vasn der Bij, A., Zaat, J. & the AGREE Collaborative Group. (2003). Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. International Journal for Quality in Health Care 15, 31–45.[CrossRef][ISI][Medline]

11 . Miller, J. & Petrie, J. (2000). Development of practice guidelines. Lancet 355, 82–3.[CrossRef][ISI][Medline]

12 . Marr, J. J., Moffet, H. L. & Kunin, C. M. (1988). Guidelines for improving the use of antimicrobial agents in hospitals: a statement by the Infectious Diseases Society of America. Journal of Infectious Diseases 157, 869–76.[ISI][Medline]

13 . Murphy, M. K., Black, N. A., Lamping, D. L., McKee, C. M., Sanderson, C. F., Askham, J. et al. (1998). Consensus development methods and their use in clinical guideline development. Health Technology Assessment, 2, i–iv, 1–88.[Medline]

14 . Edwards, J. E., Bodey, G. P., Bowden, R. A., Buchner, T., de Pauw, B. E., Filler, S. G. et al. (1997). International conference for the development of a consensus on the management and prevention of severe candidal infections. Clinical Infectious Diseases 25, 43–59.[ISI][Medline]

15 . Goldman, D. A., Weinstein, R. A., Wenzel, R. P., Tablan, O. C., Duma, R. J., Gaynes, R. P. et al. (1996). Strategies to prevent and control the emergence and spread of antimicrobial-resistant microorganisms in hospitals. A challenge for hospital leadership. Journal of the American Medical Association 275, 234–40.[Abstract]

16 . Shlaes, D. M., Gerding, D. N., John, J. F., Craig, W., Bornstein, D. L., Duncan, R. A. et al. (1997). Society for Healthcare Epidemiology of America and Infectious Diseases Society of America Joint Committee on the Prevention of Antimicrobial Resistance: Guidelines for the prevention of antimicrobial resistance in hospitals. Clinical Infectious Diseases 25, 584–99.[ISI][Medline]

17 . British Thoracic Society. (1993). Guidelines for the management of community-acquired pneumonia in adults admitted to hospital. British Journal of Hospital Medicine 49, 346–50.[ISI][Medline]

18 . Shanson, D. C. (1998). New guidelines for the antibiotic treatment of streptococcal, enterococcal and staphylococcal endocarditis. Journal of Antimicrobial Chemotherapy 42, 292–6.[Free Full Text]

19 . Graham, J. C. & Gould, F. K. (2002) Role of aminoglycosides in the treatment of bacterial endocarditis. Journal of Antimicrobial Chemotherapy 49, 437–44.[Abstract/Free Full Text]

20 . Kish, M. A. (2001). Guide to development of practice guidelines. Clinical Infectious Diseases 32, 851–4.[CrossRef][ISI][Medline]

21 . British Thoracic Society Standards of Care Committee. (2001). British Thoracic Society Guidelines for the management of community-acquired pneumonia in adults. Thorax 56, Suppl. 4, 1–64.[Free Full Text]

22 . Rex, J. H., Walsh, T. J., Sobel, J. D., Filler, S. G., Pappas, P. G., Dismukes, W. E. et al. (2000). Practice guidelines for the treatment of candidiasis. Clinical Infectious Diseases 30, 662–78.[CrossRef][ISI][Medline]

23 . Farquhar, C. M., Kofa, E. W. & Slutsky, J. R. (2002). Clinicians’ attitudes to clinical practice guidelines: a systematic review. Medical Journal of Australia 177, 502–6.[ISI][Medline]

24 . Sintchenko, V., Iredell, J. R., Gilbert, G. L. & Coiera, E. (2001). What do physicians think about evidence-based antibiotic use in critical care? A survey of Australian intensivists and infectious disease practitioners. Internal Medicine Journal 31, 462–9.[CrossRef][ISI][Medline]

25 . Formoso, G., Liberati, A. & Magrini, N. (2001). Practice guidelines: useful and ‘participative’ method? Survey of Italian physicians by professional setting. Archives of Internal Medicine 161, 2037–42.[Abstract/Free Full Text]

26 . Cabana, M. D., Rand, C. S., Powe, N. R., Wu, A. W., Wilson, M. H., Abboud, P. A. C. et al. (1999). Why don’t physicians follow clinical practice guidelines? Journal of the American Medical Association 282, 1458–65.[Abstract/Free Full Text]

27 . Davey, P., Nathwani, D. & Rubinstein, E. (2003) Antibiotic policies. In Antibiotics and Chemotherapy (Finch, R. G., Greenwood, D., Norrby. S. R. & Whitley, R., Eds). Churchill Livingstone, London and Edinburgh, UK (in press).

28 . Brown, E. M. (2002). Guidelines for antibiotic usage in hospitals. Journal of Antimicrobial Chemotherapy 49, 587–92.[Free Full Text]

29 . Moulding, N. T., Silagy, C. A. & Weller, D. P. (1999). A framework for effective management of change in clinical practice: dissemination and implementation of clinical practice guidelines. Quality in Health Care 8, 177–83.[Abstract]

30 . Del Mar, C. (2001). Guiding guidelines into practice. Australian Prescriber 24, 50–1.

31 . Steel, D. R. (2001). From evidence-based medicine to clinical standards. Proceedings of the Royal College of Physicians of Edinburgh 31, Suppl. 9, 74–6.

32 . Scally, G. & Donaldson, L. J. (1998). Clinical governance and the drive for quality improvement in the new NHS in England. British Medical Journal 317, 61–5.[Free Full Text]

33 . Goddard, M., Davies, H. T., Dawson, D., Mannion, R. & McInnes, F. (2002). Clinical performance measurement: part 1—getting the best out of it. Journal of the Royal Society of Medicine 95, 508–10.[Free Full Text]

34 . Davies, H. T. O. & Crombie, I. K. (1997). Interpreting health outcomes. Journal of Evaluation in Clinical Practice 3, 187–200.[Medline]

35 . Crombie, I. K. & Davies, H. T. O. (1998). Beyond health outcomes: the advantages of measuring process. Journal of Evaluation in Clinical Practice 4, 31–8.[Medline]

36 . Nathwani, D., Rubinstein, E., Barlow, G. & Davey, P. (2001). Do guidelines for community-acquired pneumonia improve the cost-effectiveness of hospital care? Clinical Infectious Diseases 32, 728–41.[CrossRef][ISI][Medline]

37 . Nathwani, D., Williams, F., Winter, J., Winter, J., Ogston, S. & Davey, P. (2002). Use of indicators to evaluate the quality of community-acquired pneumonia management. Clinical Infectious Diseases 34, 318–29.[CrossRef][ISI][Medline]

38 . Marrie, T. J., Lau, C. Y., Wheeler, S. L., Wong, C. J., Vanderwoort, M. K. & Feagan, B. G. for the CAPITAL Study Investigators. (2000). A controlled trial of a critical pathway for treatment of community-acquired pneumonia. Journal of the American Medical Association 283, 749–55.[Abstract/Free Full Text]

39 . Mant, J. & Hicks, N. (1995). Detecting differences in quality of care: the sensibility of measures of process and outcomes in treating myocardial infarction. British Medical Journal 311, 793–7.[Free Full Text]

40 . Atrial fibrillation in Hospital and General Practice: the Sir James Mackenzie Consensus Conference. (1999). Proceedings of the Royal College of Physicians of Edinburgh 29, Suppl. 6, 1–4.

41 . Finch, R. G. & Low, D. E. (2002). A critical assessment of published guidelines and other decision support systems for the antibiotic treatment of community-acquired respiratory tract infections. Clinical Microbiology and Infection 8, Suppl. 2, 69–92.[CrossRef][ISI][Medline]

42 . UK antimicrobial resistance strategy and action plan. Department of Health. [Online.] http://www.doh.gov.uk/arbstrat.htm (1 April 2003, date last accessed).

43 . Barlow, G. D., Lamping, D., Davey, P. G. & Nathwani, D. (2003). Evaluating outcomes in community-acquired pneumonia: a guide for patients, physicians and policy-makers. Lancet Infectious Diseases, submitted for publication.

44 . McKee, M. (1993). Routine data: a resource for clinical audit. Quality in Health Care 2, 104–11.[Medline]