1 Health Services Research Unit, Aberdeen, 2 Department of Medicine and Therapeutics and Renal Unit, Aberdeen, UK, 3 University of Nantes, Nantes, France, 4 Renal Unit, Ninewells Hospital, Dundee, UK, 5 Department of Nephrology, Academic Hospital, Nijmegen, The Netherlands, 6 Aristolelian University of Thessaloniki, Thessaloniki, Greece and 7 University of Heidelberg, Heidelberg, Germany
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods. The trial was a prospective, multicentre, randomized balanced incomplete block design. Nephrologists from the six European renal units were randomized to develop and implement guidelines for two out of the three conditions and to act as a control for the third condition. Data were collected pre- (1 year) and post- (9 months) intervention on aspects of patient monitoring, management, and outcome.
Results. Eight hundred and twenty-nine dialysis patients from the six European dialysis centres were included in the study. Multivariate analysis (adjusting for case-mix and secular trends) showed a significant increase in the number of monitoring events in the guideline group compared with control group (6%, 95% CI, 111%). There was no concomitant increase in either appropriate management or the number of favourable patient outcomes.
Conclusions. In the first European collaboration on renal guidelines, the introduction of the guidelines improved the monitoring of the patients, but did not improve patient management or outcome. This study suggests the potential for creating clinical guidelines with the aim of standardizing treatment protocols across international boundaries, and improving the quality of the medical care provided.
Keywords: CAPD; clinical guidelines; cluster randomized trial; European guidelines; haemodialysis
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Although informal guidelines have been used by clinicians in their wards and clinics for many years, their effects on clinical practice and their ability to improve patient outcome have not generally been assessed. As more therapies become available, often at high cost, health services researchers are increasingly working with clinicians to develop and evaluate more formal clinical practice guidelines.
Such guidelines have been defined as systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances [1] and are used by clinicians to improve the quality, effectiveness and cost-effectiveness of the medical care they provide by setting standards and reducing variations in clinical performance.
A recent systematic review of rigorous published evaluations of the introduction of clinical guidelines identified 91 studies [2]. Eighty-one of the 87 studies that measured the process of care observed improvements following the introduction of guidelines. Furthermore, 13 of the 17 papers that measured the outcome of care reported improvements. The review concluded that valid clinical guidelines, appropriately disseminated and implemented, can change practice and lead to improvements in patient outcome.
The systematic review did not identify any studies of guideline development and implementation in nephrology. Our group of nephrologists and health services researchers aimed to develop, implement, and evaluate consensus guidelines for the management of aspects of ESRD within the context of a European multicentre study funded by the European Union BIOMED programme.
We were aware of variations in the management of patients with ESRD and selected three for the development of specific guidelines. We selected conditions that were known to affect many patients and which had considerable resource implications. The conditions finally selected were the management of renal anaemia and renal bone disease, and the prevention and treatment of cytomegalovirus (CMV) infection in renal transplant recipients.
![]() |
Subjects and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The study was a cluster randomized controlled trial using a balanced incomplete block design (Table 1) [3]. According to this design, each centre was randomized to develop and implement two out of three of the guidelines and to act as a control for the third. It was balanced in the sense that each centre acted as both a control and a guideline site and it is incomplete because a given centre did not manage both control and guideline patients for the same condition.
|
Intervention
Whenever possible, actions specified within the guidelines were based on published evidence and we agreed that where there was no consensus, this should be made explicit in the guideline. For more complex sections detailing indications for treatment and associated actions, algorithms were constructed for two of the guidelines (bone disease and CMV) in the form of decision trees. For sections on frequency of investigations and desirable outcomes, simple text was deemed sufficient.
At the end of the guideline development sessions, a meeting took place to highlight effective methods for the dissemination and implementation of the guideline in the local settings. There was no standard approach across centres and a variety of methods were used including regular local group meetings. Potential barriers to change and factors that influence the management of change were also discussed.
Data collection
A registration form was completed for all patients over the age of 16 years who were receiving dialysis during the initial 3-month period (stock patients) and subsequently for those patients who commenced dialysis during the study period. Data on biochemical indicators, demographic details, modality changes, and frequency of investigations were collected prospectively at 3-month intervals (called updates) over the 2-year period of 1 September 1993 to 31 August 1995. Data were collected for 12 months before the guidelines were developed and implemented, and for 9 months afterwards. A 3-month period after the first year of data collection was used to develop and implement the three guidelines (Figure 1).
|
|
Statistical analysis
Firstly, a univariate comparison of the unadjusted pre-guideline scores and post-guideline scores for the control vs guideline centres was undertaken. Secondly, a multivariate approach using generalized linear modelling was used to control for possible biases and clustering effects [4]. An analysis strategy was developed a priori that identified confounders requiring adjustment in the statistical model building. These confounders were mode of dialysis, time on renal replacement therapy, centre, phase of study (pre-/post-guideline), condition, and risk group for age and comorbidity [5], and included statistical interactions where appropriate. If the effect of such confounders was significant at the 5% level, they were included. The study had 80% power and 5% significance to detect an absolute difference of 5% in the monitoring of events, assuming the control sites had 60% appropriate events. This sample size calculation ignored the clustering effects, but this was offset by the design of the study.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
There was an increase in the number of monitoring events of 6% across the three guidelines, but no concomitant increase in the proportion of appropriately managed patients or improved patient outcomes. The appropriateness of the monitoring of the events was decided in advance by consensus. Thus the process of care rather than the outcome appeared to be affected.
The univariate analyses gave little indication of the effectiveness of the guidelines, but they were prone to biases. The multivariate analyses attempted to control for some of these biases by adjusting for factors, such as case mix and the numbers of patients per centre, before the guideline effect was tested. As anticipated there was a significant centre effect [6] and a significant difference between the three guidelines. Even adjusting for a number of confounders, there was still a baseline imbalance in the monitoring and outcome areas of patient care. Nevertheless, there appeared to be a statistically significant increase in the number of events that were monitored in the guideline centres, which was consistent across all three guidelines.
It is common to evaluate clinical interventions by analysing their effect on the three areas of care: structure (monitoring), management, and outcome. It is hypothesized that if you can firstly change process of care, then management of the patients will change and ultimately their outcomes will improve.
The design of the study (a balanced incomplete block) was an attempt to reduce the Hawthorne effect in which an improvement in performance results from taking part in research. As with all cluster randomized trials there was a risk of baseline imbalance in performance between the guideline and control centres [8]. Such an effect was observed in our trial for both the monitoring and outcome areas of care.
The questions by which the guidelines were to be evaluated had to be in areas where guidelines could improve clinical management, and hence had to be chosen carefully. The compliance for the anaemia management question was 95% before the guidelines were introduced and so there was little expectation that a possible guideline effect could be detected. There was a suggestion that the control centres had a larger improvement in patient outcomes when compared with the guideline centres, but control centres initially had poorer outcomes and hence the improvement in these centres may have been in response to some trend over time (regression to the mean). There is therefore some evidence that we may have been observing a ceiling effect occurring in the guideline centres because the outcomes were already relatively good and hence difficult to improve; an effect observed previously in monitoring dialysis patients' outcomes [9]. Also, it is possible that the length of follow-up of only 9 months was too short to show an effect. Alternatively, the guidelines could have been genuinely ineffective.
Since the development of these guidelines, there have been significant advances in the field of evidence-based medicine. There is increasing emphasis being placed on integrating clinical expertise with the best available evidence from systematic literature searches and randomized trials [10]. Consensus-based guidelines require to have their evidence base strengthened such that they can provide evidence of clinical effectiveness, and this should be taken into account when assessing guideline evaluations.
Although the guidelines were developed to change clinical practice, their effectiveness will depend on how well they were implemented and hence the guidelines were developed with the clinicians that will ultimately use them. This was a well-recognized strategy to increase compliance although there is increasing evidence that this method is not as effective as first anticipated [11]. We also distributed copies of the guidelines to all staff involved in patient care and local meetings were organized to discuss the guidelines. More rigorous methods of implementation such as patient-specific reminders may have proven more effective [12].
This was the first European guideline implementation project in ESRD. The results have not been entirely successful, given that we were not able to show improved patient outcomes. There was, however, a positive increase in the monitoring of patients after the guidelines were introduced. We encountered several difficulties. Evidence-based medicine was in its infancy as regards ESRD, the Cochrane Renal group had not been formed, and there were no systematic reviews on which to base our guidelines. There were all the difficulties of clustered randomized trials including the ceiling and Hawthorne effects. The centres varied in their performance at the start of the study, making the comparison between control and test sites difficult. Implementation of the guidelines was difficult and should be a major component of any future study. We have shown, however, that there was a willingness of clinicians and health services researchers to work across international boundaries and we agreed guidelines and made some changes in the delivery of patient care.
![]() |
Acknowledgments |
---|
![]() |
Notes |
---|
Correspondence and offprint requests to: Craig R. Ramsay, Health Services Research Unit, University of Aberdeen, Foresterhill, Aberdeen AB25 2ZD, UK.
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|