Relevant change in radiological progression in patients with hip osteoarthritis. II. Determination using an expert opinion approach

J. F. Maillefert1,4, M. Nguyen1, A. Gueguen2, L. Berdah3, M. Lequesne5, B. Mazières6, E. Vignon7 and M. Dougados1,

1 René Descartes University, Cochin Hospital, Institut de Rhumatologie, Paris,
2 INSERM U88, St Maurice Hospital, Paris,
3 NEGMA Laboratories, Toussus le Noble,
4 General Hospital, Dijon,
5 Leopold Bellan Hospital, Paris,
6 Rangueil Hospital, Toulouse and
7 Lyon-Sud Hospital, Pierre Bénite, France


    Abstract
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Aim. To determine the minimum clinically important difference (MCID) in joint space width (JSW) progression in patients with hip osteoarthritis (OA), based upon evaluation by a panel of clinical experts as a gold standard.

Methods. A sample of 298 patients with hip OA was selected from a multicentre, prospective, longitudinal, 3-yr follow-up study. A pelvic radiograph was obtained at entry and after 3 yr. For each film, the narrowest JSW was measured using a 0.1-mm graduated magnifying glass. The difference between baseline and 3-yr follow-up JSW was calculated. Two senior rheumatologists, who were experts in osteoarthritis, evaluated each pair of films and noted whether a clinically relevant deterioration in osteoarthritis stage occurred at 3 yr compared with baseline. Interobserver reliabilities were evaluated using the {kappa} coefficient and proportions of agreements. Then, for each measured difference in JSW (0.1 mm per 0.1 mm), the sensitivity and specificity for MCID, defined as the assessment of expert 1, expert 2 or a combination of both, were calculated. This allowed us to obtain, from graphic representations of the correct classification probabilities, the best measured JSW threshold, with the maximal true positive and the minimal false positive results.

Results. The mean measured change in JSW was -0.63±0.74 mm. Experts 1 and 2 considered the decrease in JSW to be clinically relevant in 122 (40.9%) and 100 pairs (33.6%) respectively. The proportion of agreements between the experts was 79.9%, with a {kappa} coefficient of 0.572. The best measured JSW threshold was -0.4 mm for expert 1, expert 2 and the combination of both; sensitivity and specificity were 0.75 and 0.8, 0.71 and 0.72, and 0.75 and 0.7 respectively.

Conclusion. This study suggests that a change of at least 0.4 mm in the radiological JSW could be considered clinically relevant. Other studies using other sets of patients and other methods are needed for validation.

KEY WORDS: Minimum clinically important difference, Hip osteoarthritis, Joint space width, Outcome measure.


    Introduction
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
The emergence of agents for the treatment of osteoarthritis (OA) that might be considered structure-modifying drugs has led to recommendations for methods to be used in studies evaluating these drugs [13]. In such trials, structural variables are usually considered as the primary outcome to be assessed and the measurement of joint space width (JSW) is currently accepted as the most sensitive technique. However, this evaluation results in a continuous variable and does not discriminate between patients with and those without relevant change.

Several approaches can be used to determine a cut-off point above which a change in imaging variables could be considered to be relevant [4]. Distribution-based models describe features of the distribution of the measure in the population. In particular, the smallest detectable difference (SDD) is based on measurement error and thus on reproducibility [5]. In other words, a cut-off determined using SDD is helpful in ensuring that the changes observed are true changes and are not random fluctuation that is related to variability in measurement procedures. Authority-based models are based on expert opinion, and hence on the intuitive clinician's global assessment, using gained experience and knowledge. Finally, predictive models are driven experimentally and are based on evidence.

In a previous study we determined the SDD, on the basis of intra-observer progression score measurement error, in the absolute progression of JSW in hip OA [6]. In the present study, we used an expert opinion approach to determine a cut-off point above which a change in JSW could be considered to be relevant in patients with hip OA.


    Methods
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Study design
A sample of 298 patients with hip OA was randomly selected from a multicentre, prospective, longitudinal, 3-yr follow-up study. The study protocol was approved by the ethics committee of the Hôpital Cochin, Paris, France.

Inclusion criteria
The inclusion criteria have been described previously [7]. Briefly, outpatients visiting a rheumatologist and fulfilling the American College of Rheumatology criteria for the diagnosis of hip OA [8] were enrolled in the study after written informed consent had been obtained. Other inclusion criteria were age between 50 and 75 yr and daily hip pain for at least 1 month during the last 3 months. Exclusion criteria were joint space width <1 mm at the narrowest point, radiographic medial or axial femoral head migration, secondary hip OA (defined as a history of hip fracture), inflammatory rheumatic disease, osteonecrosis of the femoral head and Paget's disease.

Structural evaluation
An anteroposterior weight-bearing radiograph of the pelvis with the lower limbs in 15° internal rotation was obtained at entry and once a year. All films were collected and analysed by one of us (ML) [9, 10]. Films were blinded for patient identity and for date. A randomization list was used to blind the chronology. The films of a single patient were placed side by side on a light box and the narrowest JSW (identical for all films) was selected and measured (interbone distance) using a 0.1-mm graduated magnifying glass. The difference between JSW measured at baseline and after 3 yr of follow-up was calculated. Evaluation of the intra-observer reliability using 30 pairs of films, assessed twice at an interval of 1 month, showed an intra-class coefficient of correlation of 0.840 (95% confidence interval 0.693–0.920).

Expert opinion
Two senior rheumatologists, who were experts in osteoarthritis (JFM, MN), evaluated each patient's pelvic radiographs at baseline and at 3 yr of follow-up and noted whether a clinically relevant deterioration of osteoarthritis stage had occurred. This assessment was not restricted to JSW at the narrowest point, but took into account the whole joint. It was performed with no reference to any scoring system, but was based only on the experience of the experts. Each pair of films was placed simultaneously on the light box. The viewing sessions were performed in the same room, using the same light box, without a magnifying glass. In order to evaluate intra-observer reliability, each observer assessed 30 pairs of films twice, with at least 1 week between the two sessions.

Statistical analysis
This was conducted in two steps. In the first step we evaluated the inter and intra-observer reliabilities, using the {kappa} coefficient and the observed proportions of agreements. The second step permitted us to propose a clinically relevant cut-off point by allowing us to transform the continuous variable (change in JSW after a 3-yr follow-up period) into a dichotomous variable: progression/no progression. After the experts' evaluation, the patients were classified as deteriorated/not deteriorated. Thereafter, all possible different thresholds were evaluated. For each change observed in measured JSW (from +0.8 to -3.7 mm), we calculated the sensitivity (the percentage of patients with a decrease in measured JSW above the threshold among patients with a clinically relevant change) and the specificity (the percentage of patients with a decrease in measured JSW below the threshold among patients with no clinically relevant change). The choice of cut-off was based on maximal sensitivity and specificity, using the graphic representation of correct classification probabilities. Three analyses were conducted: one for each expert and a third combining the opinions of the two experts. For this last analysis, a patient was considered as deteriorated only when both experts so classified the patient. For pairs of films, which were viewed twice by each observer, the results of the first viewing session were used in the analyses.

Descriptive analyses, {kappa} statistics and graphic representations of correct classification probabilities were obtained by the use of SPSS 10.0 for Windows (SPSS, Chicago, IL, USA).


    Results
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
The main characteristics of the 298 patients at baseline are summarized in Table 1Go.


View this table:
[in this window]
[in a new window]
 
TABLE 1.  Main characteristics of the 298 patients at baseline

 
During the 3 yr of follow-up, the change in measured JSW was -0.63±0.74 (mean±S.D.) (range -3.7 to 0.8). Experts 1 and 2 considered the decrease in JSW to be clinically relevant in 122 (40.9%) and 100 pairs (33.6%) respectively. The inter-observer reliability is shown in Table 2Go. The proportion of agreements was 79.9%, with a {kappa} coefficient of 0.572. Most of the disagreements between observers occurred when the decrease in JSW was <=0.9 mm, especially <=0.6 mm. In the evaluation of intra-observer reliabilities, the proportions of agreements were 86.7 and 83.3% for observers 1 and 2 respectively, with {kappa} coefficients of 0.73 and 0.56.


View this table:
[in this window]
[in a new window]
 
TABLE 2.  Interobserver reliability (no. of patients)

 
Graphic representations of sensitivity and specificity for the three analyses, i.e. expert 1, expert 2 and the combination of the two experts, are shown in Fig. 1Go. For all analyses, a cut-off point of 0.4 mm, i.e. a decrease of at least 0.4 mm, seemed to be the most relevant, with sensitivity and specificity of 0.75 and 0.8, 0.71 and 0.72, and 0.75 and 0.7 respectively.



View larger version (15K):
[in this window]
[in a new window]
 
FIG. 1.  Graphic representation of correct classification probabilities, based on experts' opinions (presence or absence of a clinically relevant deterioration in OA stage at 3 yr compared with baseline) as a gold standard and measured changes in JSW. For each measured difference in JSW (0.1 mm per 0.1 mm), the sensitivity and specificity for clinically relevant deterioration were obtained. This made it possible to obtain the best measured JSW threshold, with maximal true positive and minimal false positive results. (a) Correct classification probabilities obtained using the opinion of expert 1. The best threshold was -0.4 mm with sensitivity 0.75 (the decrease in measured JSW between baseline and 3 yr follow-up was >=0.4 mm in 75% of patients with clinically relevant deterioration) and specificity 0.8 (the decrease in measured JSW between baseline and the 3-yr follow-up was <0.4 mm in 80% of patients with no clinically relevant deterioration). (b) Correct classification probabilities obtained using the opinion of expert 2. The best threshold was -0.4 mm with sensitivity 0.71 (the decrease in measured JSW between baseline and the 3-yr follow-up was >=0.4 mm in 71% of patients with clinically relevant deterioration) and specificity 0.72 (the decrease in measured JSW between baseline and the 3-yr follow-up was <0.4 mm in 72% of patients with no clinically relevant deterioration). (c) Correct classification probabilities obtained using a combination of both experts' opinions. In this analysis, a patient was considered as deteriorated only when this was the opinion of both experts. The best threshold was -0.4 mm, with sensitivity 0.75 (the decrease in measured JSW between baseline and the 3-yr follow-up was >=0.4 mm in 75% of patients with a clinically relevant deterioration) and specificity 0.7 (the decrease in measured JSW between baseline and the 3-yr follow-up was <0.4 mm in 70% of patients with no clinically relevant deterioration).

 


    Discussion
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
This study determined a cut-off point above which a change in JSW could be considered as clinically relevant in patients with hip OA, on the basis of a clinical expert's opinion used as a gold standard. To our knowledge, the minimum clinically important difference (MCID) in JSW progression in patients with hip OA, determined using this approach, had not been available previously, although studies using a similar approach have been conducted in other diseases, such as rheumatoid arthritis [11]. This cut-off point could be used to differentiate patients with and without a structurally relevant change, both in therapeutic trials and in clinical practice.

Because of the characteristics of the population, it might be difficult to extrapolate the results of this study to a general population of patients with hip OA. This study focused on a particular subgroup of patients, all of whom had, for example, a painful active disease. However, such characteristics are commonly observed both in daily practice and in therapeutic trials. Finally, individuals with baseline values less than the threshold cannot be designated as deteriorated [12]. Thus, this threshold should not be used in trials examining structurally severe hip OA or, in daily practice, in monitoring patients with such severe disease.

An approach based on the judgement of experts was used. This approach has the advantage of probably representing clinically relevant goals of management, but the disadvantage of not being evidence-based [12]. One could object that, in therapeutic trials, instead of determining the percentages of patients responding or not to a drug using measurement of JSW and a cut-off point that is supposed to represent MCID, obtained using an expert panel's opinion, it might be easier to classify patients directly as responders or non-responders using an expert's or an expert panel's opinion. Similarly, in daily practice, it might be easier to classify patients as deteriorated or not on the basis of the physician's personal opinion. However, such approaches would be based purely upon subjective rating, whereas in this study the threshold was determined subjectively but was applied objectively. The results of the present study suggest that determining a threshold is a better approach. Although the sessions were performed by experienced physicians, the {kappa} coefficients and proportions of agreements were quite good, but not excellent. Additionally, in daily practice there might be greater variability as all practitioners are not experts in the field of OA.

The threshold was identical, and specificity and sensitivity were similar and satisfactory, when we used either of the two observers' opinions or a combination of both, suggesting that intrinsic validity was high. However, the threshold might have been different if we had used another standardized radiological procedure. Consequently, other studies on other sets of patients and other observers are needed for validation.

This work determined a cut-off point above which a change in JSW could be considered as clinically relevant in patients with hip OA, on the basis of the opinion of clinical experts as a gold standard. This cut-off point could be used to differentiate patients with and without a structurally relevant change both in therapeutic trials and in clinical practice. However, the determination of the MCID using the present approach might be influenced by several sources of variability, such as inclusion and exclusion criteria, the methods used, and investigators. Other studies using other sets of patients and other methods are needed for validation.


    Acknowledgments
 
This work was supported in part by NEGMA Laboratories.


    Notes
 
Correspondence to: M. Dougados, Institut de Rhumatologie, Cochin Hospital, 27 rue du Faubourg Saint Jacques, 75014 Paris, France. Back


    References
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 

  1. Lequesne M, Brandt K, Bellamy N et al. Guidelines for testing slow acting drugs in osteoarthritis. J Rheumatol1994;21(Suppl. 41):65–73.[ISI]
  2. Group for the Respect of Ethics and Excellence in Science (GREES). Recommendations for the registration of drugs used in the treatment of osteoarthritis. Ann Rheum Dis1996;55:552–7.[ISI][Medline]
  3. Altman RD, Brandt KD, Hochberg MC, Moskowitz RM, for the Task Force of the Osteoarthritis Research Society. Design and conduct of clinical trials in patients with osteoarthritis: recommendations from a task force of the Osteoarthritis Research Society. Osteoarthritis Cartilage1996;4:217–43.[ISI][Medline]
  4. Lassere MND, van der Heijde D, Johnson KR. Foundations of the minimal clinically important difference for imaging. J Rheumatol2001;28:890–1.[ISI][Medline]
  5. Lassere M, Boers M, van der Heijde D et al. Smallest detectable difference in radiological progression. J Rheumatol1999;26:731–9.[ISI][Medline]
  6. Dougados M, Gueguen A, Nguyen M et al. Radiological progression of hip osteoarthritis: definition, risk factors and correlations with clinical status. Ann Rheum Dis1996;55:356–62.[Abstract]
  7. Dougados M, Gueguen A, Nguyen M et al. Requirement for total hip arthroplasty: an outcome measure of hip osteoarthritis? J Rheumatol1999;26:855–61.[ISI][Medline]
  8. Altman R, Alarcon G, Appelrouth D et al. The American College of Rheumatology criteria for the classification and reporting of osteoarthritis of the hip. Arthritis Rheum1991;34:505–14.[ISI][Medline]
  9. Lequesne M. Quantitative measurement of joint space width during progression of osteoarthritis: chondrometry. In: Kuettner K, Goldberg V, eds. Osteoarthritis disorders. Rosemont: American Academy of Orthopedic Surgeons1995;30:427–44.
  10. Lequesne M, Cadet C, Auleley GR. New technique and reproducibility of the manual measurement of osteoarthritic hip joint space [abstract]. Osteoarthritis Cartilage2000;8:156.
  11. Bruynesteyn K, van der Heijde D, Boers M et al. Minimum clinically important difference in radiological progression of joint damage over 1 year in rheumatoid arthritis: preliminary results of a validation study with clinical experts. J Rheumatol2001;28:904–10.[ISI][Medline]
  12. Dougados M, Le Claire P, van der Heijde D, Bloch DA, Bellamy N, Altman RD. Special article: response criteria for clinical trials on osteoarthritis of the knee and hip. Osteoarthritis Cartilage2000;8:395–403.[ISI][Medline]
Submitted 15 December 2000; Accepted 20 July 2001





This Article
Abstract
Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Search for citing articles in:
ISI Web of Science (12)
Disclaimer
Request Permissions
Google Scholar
Articles by Maillefert, J. F.
Articles by Dougados, M.
PubMed
PubMed Citation
Articles by Maillefert, J. F.
Articles by Dougados, M.
Related Collections
Osteoarthritis and Cartilage