Difficulties of implementing clinical guidelines in medical practice

Editor's note

Francesco Locatelli, Simeone Andrulli and Lucia Del Vecchio

Department of Nephrology and Dialysis, Lecco Hospital, Lecco, Italy

Keywords: implementation; guidelines; clinical practice; chronic renal disease; nephrology

Introduction

Clinical practice guidelines are systematic statements designed to assist doctors in diagnostic and therapeutic decisions, and have a long tradition in clinical medicine. They have been developed by physicians as a means of improving the quality of care and objective medical decision making, as well as of optimizing the use of resources [1] and thus possibly decreasing healthcare expenditure.

Risks and benefits of clinical guidelines

Physicians are responsible for offering patients the best available treatment, but their often overloaded schedules can make it difficult for them to keep abreast of the latest medical advances in a rapidly evolving therapeutic field. The main aim of clinical guidelines is therefore that of aiding clinical decision making on the part of doctors who cannot integrate all of the published data concerning new technologies and knowledge in their everyday practice.

However, although guidelines may be important in promoting quality, they can also be harmful if they do not really advocate the best options for patients because of scientific uncertainties, biases in guideline development and/or patient heterogeneity [2]. This was particularly true some years ago (when many guidelines were consensus-based rather than being explicitly linked to proof of effectiveness), but a greater effort has more recently been made to develop recommendations based as much as possible on clear-cut data published in the medical literature.

Although there is still no methodological consensus as to which approach leads to more valid guidelines, these efforts have certainly improved the scientific quality of those that have been issued more recently, and their greater emphasis on describing the quality of the data and their degree of uncertainty has made them precious tools for conscious and judicious decision-making. On the other hand, even high-quality over-standardized medical behaviour may lead to harmful choices in the case of individual patients. Furthermore, non-medical considerations (such as economic evaluations and the intense pressure that these place on health care provision) can also create an environment in which guidelines are misused [3], as has been illustrated by the development of certain guidelines aimed more at limiting access to secondary care than as a means of assisting clinical decision making.

Implementation of clinical guidelines

Despite the efforts made in developing guidelines, strangely little has been done to encourage their implementation. However, as this is the only way of ensuring that they actually improve healthcare outcomes, it is important that an equivalent emphasis is placed on implementation strategies and the scientific evaluation of their effectiveness in real clinical settings [4], including the development of local implementation support systems, clinical audit programmes and methods of feeding back information concerning current practice. It is therefore worth adopting a multidisciplinary approach that reflects all of the various aspects of patient care [5].

A number of serious deficiencies in the adoption of guidelines in clinical practice have been pointed out [6], but only a few studies have examined whether (and under what conditions) they are really effective in changing physicians' practices and modifying patient outcomes [7]. Furthermore, their validity remains largely untested and there are considerable differences in the extent of their impact on performance [8].

The variables affecting the adoption of guidelines include their quality, incentives and patient factors [9]. However, it is also important to bear in mind that clinical decisions not only depend on published research data, but also on clinical expertise, patient preferences and the constraints of public health policies, community standards and budgetary limitations [10].

Compliance with clinical guidelines is an important indicator of quality and efficacy [11], because an improvement in outcome can only be expected if they are closely followed. On the other hand, it is likely that such an improvement can only be detected in the case that the clinical approach adopted before the implementation of a guideline was less than good. In the presence of an already high standard of medical care, it is unlikely that practice guidelines will lead to any significant benefits even if they are properly implemented.

Nephrology guidelines

None of the many different treatment approaches in the different fields of end-stage renal disease (ESRD) is necessarily wrong but, in order to optimize clinical outcome, it is important to involve the largest possible number of nephrologists in order to reach consensus decisions.

In March 1995, the National Kidney Foundation initiated its Dialysis Outcomes Quality Initiative (NKF-DOQI) in an effort to create evidence-based, best-practice clinical guidelines concerning hemodialysis adequacy, peritoneal dialysis adequacy, vascular access and anaemia. The final guidelines were issued in the fall of 1997 and represent the first unifying effort of the entire renal community. Given the importance of translating these guidelines into clinical practice, a 3-year implementation plan was simultaneously developed and activated by the end of 1997 [12].

Subsequently, the European Best Practice Guidelines for the management of anaemia in patients with chronic renal failure were developed on the basis of their NKF-DOQI counterparts, but with the further aim of reflecting European clinical practice and experience, and including additional publications and new analyses. According to J. S. Cameron [13], rather than being prescriptive, these guidelines were intended to provide clinical guidance on the basis of the best available evidence: given that every facility has unique people, equipment and procedures, a certain amount of autonomy is required in order to encompass individual variation [14].

Contribution of the paper by Ramsay et al.

The paper ‘Evaluation of clinical guidelines for the management of end-stage renal disease in Europe: the EU BIOMED 1 study’ by Ramsay et al. [15] published in this issue of the journal deals with the hot subject of the development, harmonization, implementation and evaluation of consensus-based clinical guidelines. Being well aware of variations in the management of ESRD patients, the authors selected three conditions known to affect many patients and to have considerable implications in terms of resources (line anaemia, renal bone disease, and cytomegalovirus disease in renal transplant recipients), and evaluated the effects of the implementation of the related guidelines in six renal centres in Europe. At the time the study was designed, the authors' systematic review of the literature did not identify any other similar studies in the nephrological field, and so this paper is certainly of particular importance.

Unfortunately, the results were not encouraging and the authors concluded: ‘In the first European collaboration on renal guidelines, the introduction of the guidelines improved the monitoring of the patients, but did not improve patient management or outcome’. However, despite these negative conclusions, the study has a number of undeniable strong points.

First of all, its design has the advantage of being prospective, multicentre and randomized, even though (as the authors themselves point out) the adopted cluster incomplete block design did not exclude the risk of a baseline imbalance in performance between the guideline and control centres because a given centre did not manage both control and guideline patients with the same condition. This is therefore neither a classical clinical trial nor an epidemiological study. Furthermore, the randomization related to the ‘six’ units (centres) was based on the pooled nephrologists of one centre whereas the statistical units (which give the information of interest) are ‘the patients’ of each centre.

Another major strong point is that the study design included an observational period before the implementation of the guidelines was started. This is particularly important because it minimizes the risk that spontaneous trends in patient outcomes may have been misinterpreted as benefits obtained from the implementation. Thus an analysis of the ESRD patients in the Lombardy Registry aimed at clarifying the clinical impact of anaemia and epoietin treatment on general and cardiovascular mortality and morbidity found that the mean haematocrit level progressively increased from 29% in 1994 to 32% in 1997, which possibly reflects a growing awareness among Lombardy nephrologists of the need to correct anaemia better in ESRD patients even before the anaemia management guidelines became formally available. Something similar is likely to occur in relation to many aspects of nephrological management, and this must be clearly recognized before assessing the role of guideline implementation on outcomes.

One point of weakness of the study is that it is not clear how the five European countries were selected and the six centres representing them were not randomized. This is important because the authors themselves point out that there are not only international, but also national variations in the management of patients with ESRD [16]. The authors recognize that the selection of the centres precluded any generalization of the results and state that ‘it was important to have a group of well-motivated nephrologists and methodologists (working together)’ and ‘there was a significant centre effect’.

This aspect has been properly managed in the ongoing Dialysis Outcome and Practice Patterns Study (DOPPS) [17], a prospective, longitudinal, observational study of haemodialysis patients and facilities in seven countries with large populations of dialysis patients: France, Germany, Italy, Japan, Spain, the United Kingdom and the United States. The aim of the study is to determine which practice patterns are associated with the best patient outcomes, after adjustment for a wide range of patient case-mix characteristics. The DOPPS has not only demonstrated that this kind of study is technically feasible, but has also revealed basic differences among centres that vary widely in terms of their size, type and patients in the seven participating countries. The particular characteristic of the DOPPS is that it is designed to detect the associations between practice patterns and outcomes, thus possibly modifying the practice pattern only if there is a clear suggestion that one practice is actually related to a good or bad result. This situation is reminiscent of the process of implementing a diet: you can prescribe a very precise diet (guidelines) or simply correct some behaviours without modifying the basic diet (practice pattern). The essential goals of the DOPPS are to ensure that it represents the target populations adequately and allows a detailed collection of practice patterns and longitudinal patient data. In contrast, describing the experience of a single centre or convenient group of centres (as is the case of the study by Ramsay et al.) may lead to sacrificing the ability to generalize the findings. The differences in the patients' basic characteristics emphasizes the importance of adjusting for this aspect when evaluating the potential relationships between modifiable practice patterns and patient outcomes because (unlike practice patterns) individual patient characteristics are sometimes unchangeable.

One clear example of this in the study by Ramsay et al. is the large difference in the number of patients on dialysis at start the of the study in the six centres (between 50 and 170) and, even more strikingly, the difference in the percentage of patients who started dialysis during the study (between 10% and 97% of those on dialysis at the beginning). As the authors have not addressed this issue in their paper, we do not know the reasons for these discrepancies, but these imbalances clearly raise some questions concerning a possible patient selection bias.

Returning to the conclusions of the study, we wonder about the usefulness of improving patient monitoring if this has no effect in terms of improving patient management or outcome. Although we agree that the results of the study support the potential of creating clinical guidelines with the aim of standardizing treatment protocols across international boundaries, it is more difficult to conclude that this in itself may improve the quality of medical care (which unfortunately remains a distant hope). The results are particularly frustrating because, as the authors point out, the intervention was very correct and above all demanding for the centres.

Although the (albeit statistically significant) result of only a 6% improvement in patient monitoring between the guideline and control centres is disappointing, the authors believe that it provides some evidence in favour of the view that guidelines increase the adherence to (and probably the the frequency of) appropriate investigations. However, in comparison with the guideline centres, there were paradoxical improvements in patient management (4%) and outcome (7%) in the control centres, which were strangely of the same magnitude as the improvement in monitoring although of only borderline statistical significance (CI: -1%/+9% and -1%/14%, respectively). It is perhaps worth remembering that P values kill a lot of patients!

As already mentioned, another very important aspect of the evaluation of guideline implementation is the ‘ceiling effect’: as the author's point out, when the outcome is already relatively good, it is difficult to improve it further. However, although this was true in relation to the management of anaemia (95% before the guidelines were introduced), it was not the case in terms of the outcome area of care because the baseline anaemia outcome score was low (respectively 55% and 42% in the guideline and control centres).

The authors acknowledge that, if guideline implementation was difficult in selected and well-motivated centres, it is likely to be even more difficult in unselected centres in Europe (and the rest of the world). We agree that, although guidelines have been developed in order to change clinical practices, their effectiveness will depend on how well they are implemented, which means that they should be developed with (or at least for) the clinicians who will ultimately use them however difficult and demanding this may be. Moreover, healthcare regulators need to understand that clinical practice guidelines are not standards to be used against malpractices, but recommendations intended to assist physicians and patients in making decisions. Finally, it is important to ensure that clinicians do not become the forced inmates of a prison of their own making.

Notes

See also Original Article by C. R. Ramsay et al., pp. 1394–1398.

Correspondence and offprint requests to: Prof. Dr Francesco Locatelli, Divisione di Nefrologia e Dialisi, Azienda Ospedale di Lecco, Ospedale A. manzoni, Via Dell 'Eremo 9/11, I-23900 Lecco, Italy. Back

References

  1. Cook DJ, Greengold NL, Ellrodt AG et al. The relation between systematic reviews and practice guidelines. Ann Intern Med1997; 127: 210–216[Abstract/Free Full Text]
  2. Woolf SH. Do clinical practice guidelines define good medical care? The need for good science and the disclosure of uncertainty when defining ‘best practices’. Chest1998; 113 [3 Suppl]: 166S–171S[Abstract/Free Full Text]
  3. Berger JT, Rosner F. The ethics of practice guidelines. Arch Intern Med1996; 156: 2051–2056[Abstract]
  4. Audet AM, Greenfield S, Field M. Medical practice guidelines: current activities and future directions. Ann Intern Med1990; 113: 709–714[ISI][Medline]
  5. Capelli JP. Implementing organizational systems to measure outcome-related processes of end-stage renal disease care. Am J Kidney Dis1994; 24: 346–354[ISI][Medline]
  6. Davis DA, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ1997; 157: 408–416[Abstract/Free Full Text]
  7. Walker RD, Howard MO, Lambert MD et al. Medical practice guidelines. West J Med1994; 161: 39–44[ISI][Medline]
  8. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet1993; 342: 1317–1322[ISI][Medline]
  9. Diamond LH, Daly DC. Forum of the End-Stage Renal Disease Networks' role in implementing the National Kidney Foundation-Dialysis Outcomes Quality Initiative Clinical Practice Guidelines. Adv Ren Replace Ther1999; 6: 28–35[ISI][Medline]
  10. Kliger AS, Haley WE. Clinical practice guidelines in end-stage renal disease: a strategy for implementation. J Am Soc Nephrol1999; 10: 872–877[Abstract/Free Full Text]
  11. Harr DS, Balas EA, Mitchell J. Developing quality indicators as educational tools to measure the implementation of clinical practice guidelines. Am J Med Qual1996; 11: 179–185[Medline]
  12. Levin N, Eknoyan G, Pipp M, Steinberg E. National Kidney Foundation: Dialysis Outcome Quality Initiative–development of methodology for clinical practice guidelines. Nephrol Dial Transplant1997; 12: 2060–2063[Abstract]
  13. Cameron JS. European best practice guidelines for the management of anaemia in patients with chronic renal failure. Nephrol Dial Transplant1999; 14 [Suppl 2]: 61–65[Free Full Text]
  14. Goldman RS. Continuous quality improvement in ESRD: the role of networks, the United States Renal Data System, and facility-specific reports. Am J Kidney Dis1998; 32 [Suppl 4]: S182–S189[Medline]
  15. Ramsay CR, Campbell MK, Cantarovich D et al. Evaluation of clinical guidelines for the management of end stage renal disease (ESRD) in Europe: the EU biomed 1 study. Nephrol Dial Transplant2000; 15: 1394–1398[Abstract/Free Full Text]
  16. Locatelli F, Marcelli D, Conte F et al. Patient selection affects end-stage renal disease outcome comparisons. Kidney Int2000; 57 [Suppl 74]: 94–99
  17. Young EW, Goodkin DA, Mapes DL et al. The Dialysis Outcomes and Practice Patterns Study (DOPPS): an international hemodialysis study. Kidney Int2000; 57 [Suppl 74]: 74–81