Rheumatologists' opinions on the feasibility of a measurement feedback system in rheumatoid arthritis and the influence of motivation

J. Fransen, S. Daneel, T. Langenegger1 and B. A. Michel for the members of the Swiss Clinical Quality Management in rheumatoid arthritis (SCQM){dagger}

Department of Rheumatology and Institute of Physical Medicine, University Hospital Zurich and
1 Adelheid Page Rehabilitation Hospital, Unteraegeri, Switzerland


    Abstract
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conflict of interest
 References
 
Objective. To assess rheumatologists' opinions about the feasibility of a measurement feedback system in rheumatoid arthritis (RA) and to analyse if motivational aspects play a role in assessing the value of the system and in determining the extent to which it is used.

Methods. A survey sample (n=105) was randomly selected from participants of a measurement feedback system. A survey questionnaire assessed opinions on system outcome, structures and processes, motivation and overall satisfaction. Survey results are given descriptively and groups differing in motivation are compared.

Results. The overall response rate was 62%. The system was generally perceived to fulfil its aims, but the effort required to use the system was rated less positive. Rheumatologists had as their motivation either ‘science/obligation’ or ‘individual patient evaluation’. Rheumatologists with the latter motivation were more satisfied with the measurement feedback system, perceived its feasibility as better, and made more use of it.

Conclusion. Motivation for participating in a measurement feedback system has a significant impact on overall satisfaction with the system and the use of the system. Influencing motivation and reduction of the amount of effort required to use the system might increase overall acceptance.

KEY WORDS: Rheumatoid arthritis, Measurement feedback, Quality management, Feasibility, Survey.


    Introduction
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conflict of interest
 References
 
The Swiss Clinical Quality Management in rheumatoid arthritis (SCQM) is a cooperation of rheumatology departments of all university hospitals, most major hospitals, and rheumatologists in private practice throughout Switzerland. Since 1997, the SCQM has provided rheumatologists with a measurement feedback system for monitoring the disease course of individual rheumatoid arthritis (RA) patients. Using knowledge of the disease course, rheumatologists can optimize drug therapy to reduce disease activity and prevent progression of joint damage and co-occurring disability [1]. The accumulated cohort data can be used to monitor the disease course and burden of disease, and to explore the use of different treatment strategies and predictors of outcome [1]. The long-term goal of the SCQM is to enable all RA patients to have the optimal treatment, thus minimizing the burden of the disease to the patient, to his or her social environment and to society [1].

Use of the measurement feedback system is free of charge. Rheumatologists mail standardized clinical and laboratory assessments, patient questionnaires and, annually, X-ray files, to the SCQM coordination centre, where the data are processed and stored in a database [2]. A comprehensive paper feedback report on the individual patient is mailed back to the rheumatologist. It contains a graphical and numerical displays of the course of disease activity (disease activity score, RA disease activity index, joint counts, pain), disability (Health Assessment Questionnaire) and joint damage (Ratingen X-ray score), laboratory values (e.g. erythrocyte sedimentation rate, blood count, alanine aminotransferase) and an overview of past and current drug therapy (for more detail see [2]). A distinction is made between annual assessments and intermediate assessments.

Rheumatologists are reminded by the coordination centre to perform annual assessments. Annual assessments are full assessments, and include disability and joint damage. Reduced intermediate assessments of disease activity and laboratory values are performed throughout the year at the discretion of the rheumatologist.

For widespread use in clinical practice, a measurement feedback system should be feasible and appreciated as a support in daily clinical practice and in making treatment decisions. In studying its efficacy, it is important to know that that compliance with a measurement feedback system is high, as it makes no sense to study the effectiveness of an intervention that cannot be adhered to. In an implementation study [3], as well as in daily practice, it was noticed that some rheumatologists did not actively make use of the measurement feedback system, while others used it intensively. To obtain information that might be useful in promoting broader use of the measurement feedback system, opinion about its feasibility was assessed among a random sample of rheumatologists with at least some experience with the system. In addition, we studied whether motivational aspects played a role in valuing the system and affected the extent to which it was used.


    Methods
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conflict of interest
 References
 
There were 267 rheumatologists using the measurement feedback system in 2001. A sample (n=105) was drawn with a random number generator, stratified for private practice (n=40), regional hospitals (n=25) and university hospitals (n=40). A numbered survey questionnaire was mailed to the sampled rheumatologists early in 2002, with a pre-addressed, prepaid return envelope enclosed. After 4 weeks, another questionnaire was sent to non-responders.

The survey questionnaire was designed to assess opinion about the feasibility of the measurement feedback system, and to be completed in 10–15 min. An intervention can be thought of as feasible if it meets criteria in three areas: (i) the degree to which the intervention meets its stated aims (outcome); (ii) the practicability of the structures needed to execute the intervention (structure); and (iii) the acceptability of the way in which the intervention is implemented (process). These points cover the three main aspects of clinical quality as formulated by Donabedian [4]: outcome, structure and process. There will be a trade-off (often implicit) of perceived advantages and drawbacks, leading to an overall assessment of value, which can be expressed as satisfaction with the intervention.

The survey questionnaire contained 23 feasibility items with a Likert scale response range of 1–6, where 1 is the positive and 6 the negative extreme [5]. Also included were questions on practice size, motivation to participate in the system, perceived advantages and drawbacks, and overall satisfaction. The questionnaire was pretested for apprehension and completeness.

Actual use of the measurement feedback system in 2001 was determined from the SCQM database for every responding rheumatologist, by the number of RA patients included in the system and the number of patients who had any intermediate visits.

The survey question about motivation to participate in the measurement feedback system was assessed qualitatively, and two clearly distinct motivational groups could be formed. The use of phrases such as ‘evaluation’, ‘long-term follow-up’, ‘decision support’, etc. was taken to reflect factors directly related, or ‘internal’, to the patient–physician relationship. The use of the expressions ‘obligatory’, ‘hospital custom’ or ‘science’ or leaving the question unanswered was taken to express factors not directly related, or ‘external’, to the patient–physician relationship. ‘Internal’ and ‘external’ do not refer to a specific psychological construct.

Likert scale item responses were analysed descriptively. Differences between the two motivation groups were analysed using item response means and the two-sample t-test, for ease of comparison. Differences in overall satisfaction were analysed using Fisher's exact test. Differences in use were analysed using the two-sample Wilcoxon test.


    Results
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conflict of interest
 References
 
Response
Of the 105 questionnaires sent, 55 were returned initially and 10 after the reminder. The overall response rate was 62%; it was 65% for private practitioners, 80% for regional hospitals and 47% for university hospitals.

Outcome
The responses to the survey items are described in the left-hand part of Table 1Go.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Survey item responses

 
The majority of respondents (strongly) agreed with the statements that the measurement feedback was useful in practice, provided a better insight into the disease process and was helpful in monitoring treatment effects and disease course.

The majority did not agree fully with the statement that the feedback on disease activity influenced their treatment decisions. However, many respondents stated that feedback made them feel more certain about treatment decisions taken.

Structure
Most respondents perceived the information as clearly presented and of relevance. But a large minority did not fully agree that the information was complete enough, for most patients, to serve as the basis for DMARD (disease-modifying anti-rheumatic drug) treatment decisions.

The time necessary to perform the assessments of disease activity was generally perceived as acceptable. A majority of respondents did not confirm the statement that, all in all, time could be saved later, or that the time spent was in balance with time gained. This concerned especially the larger annual assessments, not the intermediate assessments of disease activity.

Process
A majority of the respondents stated that they always, or most of the time, showed and explained the feedback system to the patients, and a majority had a look at it upon receipt. According to most respondents, they did not assess disease activity regularly between the annual assessments. A majority of respondents agreed that performing annual X-rays is acceptable for most cases; however, this had many drawbacks (see later). The number of respondents who said they did not feel positive about the effort required was large.

Motivation
The rheumatologists' motivation, described in terms of internal or external factors, made a difference in the rating of overall satisfaction (Table 1Go). There was a tendency for rheumatologists with internal motivation to rate feasibility issues more positively than rheumatologists with external motivation (Table 1Go). For example, the time spent on an intermediate visit was rated more positively by rheumatologists with internal motivation. Rheumatologists with internal motivation also showed and explained the feedback report to their patients more often.

Time
The two motivation groups rated the time taken using the system similarly. The majority (66%) stated that they used an additional 1–10 min when assessing disease activity. For the longer annual visits, the rating of the extra time spent ranged from 5–10 to >20 min. Forty-six per cent stated that the feedback report was received 10–18 days after the patient visit; 33% did not know when it arrived.

Use
The rheumatologists who described an internal motivation included more of their RA patients, and performed intermediate assessments in a larger proportion of these patients (Table 2Go). The differences in use could not be explained by a small difference between centres: in the hospitals 50% of respondents described an internal motivation and in private practice 70% did so.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Use of the measurement feedback system in 2001

 

Advantages and drawbacks
Fifty respondents described advantages and drawbacks. The advantages most often mentioned were that the system provided a long-term evaluation/follow-up of the disease course (n=30); better decision-making/treatment quality (n=25); a clear and comprehensive overview (n=25); objective, systematic assessments (n=23); and an aid for patient and physician communication (n=17). The drawbacks most often described were the amount of time taken (n=21); provides no benefit for decision support (n=11); the inefficiency of the paper bureaucracy (n=9); the need for annual X-rays for all patients (n=8); problems in obtaining patient cooperation (n=7); the lag time of the feedback report (n=5); the lack of scientific outcome (n=5); problems with the patient questionnaires (n=5); patients being a selection of the RA population (n=4); and difficulty with the standardized medication documentation (n=4).


    Discussion
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conflict of interest
 References
 
A return rate of 62% was reached, which can be considered satisfactory for (anonymous) surveys. The low return rate in university hospitals may have been caused by the large physician turnover in teaching hospitals.

In this sample the measurement feedback system was not used frequently, which is not in agreement with its purpose of serving as a means of supporting decisions about regular adaptations of DMARD therapy. Rheumatologists who participated in the measurement feedback system for the evaluation of their individual RA patients (internal motivation) perceived the measurement feedback system as more useful, were more satisfied and less bothered by the time required, and also made use of it more often. On the contrary, if the rheumatologist did not perceive the measurement feedback system as useful in treating their own patients (external motivation), feasibility was judged in a more negative way, and less use was made of it. Nevertheless, it is reassuring that the SCQM measurement feedback system was generally perceived to fulfil its aims, and this was clear from the advantages that were stated. However, overall satisfaction was not optimal, being 70% for those with internal motivation and 40% for those with external motivation. A satisfaction rate of 80% is not unusual for many situations in health-care [6]. Thus, there is reason to believe that satisfaction should be improved, ideally approaching 100%, in all fields, i.e. outcome, structure and process. However it can be seen from Table 1Go that dissatisfaction concentrates on the time required and decision-making.

The major advantage of instituting a standardized measurement feedback system in general is that important aspects of disease can be expressed in a more objective way. Then, treatment decisions can be based on (semi-)objective data, it is easier to set and check treatment targets, treatment results can be compared with trial results, and the data may serve as a communication aid [7]. However, it should be clear that measurement feedback is meant to support decision-making; it adds to, but does not replace, clinical thinking and other relevant sources of information. This principle should be clearly communicated to users. The major drawbacks of a standardized measurement feedback system are that it may be felt to reduce freedom, which makes the effort required seem a burden; it may make people think that numbers are considered more important than patients; and rheumatologists may be afraid of being controlled and judged on some abstract treatment result. Further, measurement feedback systems may be quite costly; e.g. an assessment costs the SCQM between 25 and 65 euros. Even if rheumatologists do not pay for the SCQM measurement feedback system, time also means money for a lot of physicians.

The SCQM measurement feedback system was initially developed in a consensus process involving key players among Swiss rheumatologists. Our survey of current users has suggested several things that can be done in order to increase the use of a measurement feedback system for RA. With regard to outcome, regular workshops could be provided on how to carry out assessments and the interpretation of scores, and how these can be used in decision-making. Especially for non-laboratory assessments, it can be important to improve familiarity, agreement, self-efficacy, and positive outcome expectancy [8]. With regard to structure, easy access to treatment guidelines and the possibility of consultancy in difficult cases could be offered, in addition to the provision of the basic feedback. Electronic solutions could replace paper ones to reduce effort and to increase speed. Patient questionnaires could be collected by a coordination centre instead of the rheumatologist. With regard to process, the frequency of X-rays could be changed, e.g. reduced in established disease and increased in early disease. Immediate feedback, instead of a time lag of several days, is a great advantage in decision-making. Then, treatment decisions can be made in the presence of the patient, which may enhance patient compliance. Providing the patient with illustrated information material that is well written and easy to understand may lighten the unpleasant task of persuading patients to join the measurement feedback system.

In our opinion, it is of major importance that the aim of a measurement feedback system should be communicated clearly, and that its use in evaluating individual RA patients should be stimulated. With regard to the commitment to science, it seems important to create ‘data ownership’ and to produce regular short communications for all data contributors, e.g. in a newsletter.


    Conflict of interest
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conflict of interest
 References
 
T. Langenegger and B. A. Michel are on the steering committee of the SCQM. S. Daneel is project coordinator at the SCQM. There are no other conflicts of interest.


    Acknowledgments
 
We are grateful to all the rheumatologists who found time to respond to the survey. We wish to thank Leanne Pobjoy for her assistance in the preparation of the manuscript. This work was part of a project sponsored by a grant from the Swiss Health Authorities (BAG).


    Notes
 
Correspondence to: J. Fransen, Department of Rheumatology, University Medical Centre Nijmegen, PO Box 9101, 6500 HB Nijmegen, The Netherlands. E-mail: j.fransen{at}reuma.azn.nl Back

{dagger} The members of Swiss Clinical Quality Management in Rheumatoid Arthritis are: Dr I. Büchler, Dr B. Christen, Dr A.-M. Chamot, G. Dalvit, Dr A. Forster, PD Dr P. A. Guerne, Dr T. Langenegger, Professor Dr B. A. Michel, Dr L. Schmid, Dr H. Schwarz, Professor Dr M. Seitz, Professor Dr A. So, PD Dr R. Theiler, Professor Dr A. Tyndall, PD Dr D. van Linthoudt and Professor Dr P. Villiger. Back


    References
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conflict of interest
 References
 

  1. Stucki G, Sangha O. Clinical quality management: putting the pieces together. Arthritis Care Res 1996;9:405–12.[Medline]
  2. Uitz E, Fransen J, Langenegger T, Stucki G. Clinical quality management in rheumatoid arthritis: putting theory into practice. Rheumatology 2000;39:542–9.[Abstract/Free Full Text]
  3. Fransen J, Stucki G, Twisk JWR et al. The effectiveness of a measurement–feedback system on outcome in rheumatoid arthritis: a controlled clinical trial. Ann Rheum Dis 2003;62:in press.
  4. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44:166–203.[ISI][Medline]
  5. Streiner DL, Norman GR. Health measurement scales. A practical guide to their development and use. Oxford: Oxford University Press, 1999.
  6. Di Palo MT. Rating satisfaction research: is it poor, fair, good, very good or excellent? Arthritis Care Res 1997;10:422–30.[ISI][Medline]
  7. Fransen J, Stucki G, Van Riel PLCM. The merits of monitoring: should we follow all our RA patients in daily clinical practice? Rheumatology 2002;41:601–4.[Free Full Text]
  8. Cabana MD, Rushton JL, Rush AJ. Implementing practice guidelines for depression: applying a new framework to an old problem. Gen Hosp Psychiatry 2002;24:35–42.[CrossRef][ISI][Medline]
Submitted 6 May 2002; Accepted 18 December 2002





This Article
Abstract
Full Text (PDF)
All Versions of this Article:
42/8/924    most recent
keg249v1
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Disclaimer
Request Permissions
Google Scholar
Articles by Fransen, J.
Articles by Michel, B. A.
PubMed
PubMed Citation
Articles by Fransen, J.
Articles by Michel, B. A.
Related Collections
Rheumatoid Arthritis