Are clinical trials in rheumatoid arthritis generalizable to routine practice? A re-evaluation of trial entry criteria

G. H. Kingsley1,3, B. Khoshaba1, C. M. Smith1, E. H. Choy1,2 and D. L. Scott1,2

Department of Rheumatology, 1 GKT School of Medicine, Weston Education Centre, Kings College, 10 Cutcombe Road, London SE5 9RS, 2 Kings College Hospital, Denmark Hill, London SE5 9RS and 3 University Hospital Lewisham, Lewisham High Street, London SE13 6LH, UK.

Correspondence to: G. H. Kingsley, Department of Rheumatology, GKT School of Medicine, Weston Education Centre, Kings College, Cutcombe Road, London SE5, UK. E-mails: gabrielle.kingsley{at}kcl.ac.uk and janice.jimenez{at}kcl.ac.uk


    Abstract
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 References
 
Objective. Trials of disease-modifying anti-rheumatic drugs (DMARDs) enrol active rheumatoid arthritis patients identified using standard criteria (three out of four of: ≥6 tender joints, ≥6 swollen joints, ESR ≥ 28 mm/h, ≥45 min morning stiffness). Concern has been expressed about generalizability, as many patients in routine practice have less active disease. Furthermore, these criteria do not map onto standard disease activity and treatment response measures. We examined how many routine patients were sufficiently active to meet trial recruitment criteria and whether alternative definitions of active disease were more appropriate.

Methods. We studied 504 patients in a cross-sectional study, 156 in a longitudinal study and 94 starting new DMARDs or biologics. Patients were classified as ‘trial active’ (met entry criteria), in remission or ‘intermediately active’ (between the two). We also evaluated the effect of amendments to criteria.

Results. Cross-sectionally only 38% patients were ‘trial active’, but longitudinally 68% were ‘trial active’ at least once. Thus, many clinic patients do have disease activity below the level required for trial entry, but over time most reach eligibility levels. More (62%) of the cohort starting new treatment were ‘trial active’, suggesting that recruitment criteria relate to clinical decisions. Criteria omitting morning stiffness and a disease activity score (DAS28) ≥5.4 replicated the classification given by current criteria.

Conclusions. Trial results can be generalized to routine practice because most clinic patients are ‘trial active’ when their therapy is changed and most become ‘trial active’ over time. As DAS-based criteria are simpler and relate directly to response measures, their use should be considered in future.

KEY WORDS: Rheumatoid arthritis, Clinical assessment, Clinical trial methodologies


    Introduction
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 References
 
Pressure on healthcare resources emphasizes the need for evidence-based medicine. An essential component is the randomized controlled trial (RCT) yet RCTs face major obstacles. Recruitment problems are universal [1] because of the time pressure on clinicians, complex trial regulations, a more questioning attitude by patients and better standard treatment. Generalizability [2, 3] is also a recurring concern since minorities and the elderly are often excluded [3] and trial patients have been shown to differ from routine clinic attendees, for example in terms of disease activity [2, 4, 5].

Since RCTs remain the best way to assess drug efficacy, it is crucial to optimize trial entry criteria ensuring generalizability yet facilitating recruitment. A common definition of patients who are active enough to enter trials (trial active) is that they should meet three out of four of the following: ≥6 tender joints, ≥6 swollen joints, erythrocyte sedimentation rate (ESR) ≥28 mm/h and ≥45 min early morning stiffness (EMS). The justification for these empirically adopted criteria is much less sound than the disease activity score (DAS) and ACR response criteria that are fully validated, internally consistent and, unlike the rheumatoid arthritis (RA) trial entry criteria, use the same components.

Sokka and Pincus [4, 5] have particularly criticised the disease activity aspect of current trial entry criteria. They showed that many routine clinic patients had insufficiently active disease to be eligible for RCTs of new disease-modifying anti-rheumatic drugs (DMARDs) and biologics and concluded these trials were therefore unsuitable as guides to practice. However, they did not look at variability of disease activity over time.

We investigated two facets of trial entry. First, we analysed whether the level of disease activity in trial patients was representative of that seen in routine clinic attendees at a single time point, over time and in patients changing therapy. Second, we examined whether simpler trial entry criteria, more consistent with DAS and response measures, could be developed by analysing the effect of different criteria on the proportion of patients eligible for trial entry.


    Patients and methods
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 References
 
Patient cohorts
We studied three groups of RA patients attending routine hospital follow-up clinics. All fulfilled ACR criteria [6]. They gave written consent according to the Declaration of Helsinki; the study was approved by Kings College Hospital Research Ethics Committee.

Group A, a cross-sectional cohort of 504 consecutive attendees assessed once, had a male/female (M/F) ratio of 109/395, mean age 56 yr (range 18–89 yr) and mean disease duration of 10 yr (range 1–53 yr). Three hundred and thirty-six (73%) were taking DMARDs including methotrexate in 237 (47%), sulphasalazine in 40 (8%), anti-tumour necrosis factor (TNF) in 14 (3%) and oral or recent i.m. steroids in 117 (23%).

Group B, a longitudinal cohort of 156 patients assessed four times or more by a single rheumatologist (EC), had (at the first visit) a M/F ratio of 38/118, mean age 59 yr (range 24–87 yr) and mean disease duration of 10 yr (range 1–46 yr). One hundred and twenty-seven (81%) were on DMARDs including methotrexate in 82 (52%), sulphasalazine in 9 (6%), anti-TNF in 5 (3%) and oral/recent i.m. steroids in 34 (22%).

Group C, a cross-sectional cohort of 94 consecutive patients starting new DMARDs or biologics, had a M/F ratio of 18/76, mean age 58 yr (range 18–88 yr) and mean disease duration of 12 yr (range 1–40 yr). Forty-two (45%) were starting methotrexate, 11 (12%) sulphasalazine, 8 (9%) anti-TNF and 33 other drugs; 25 (27%) were taking oral or had recent i.m. steroids.

Assessment and classification of patients and analysis of data
We recorded 28 joint counts for tender and swollen joints, EMS and ESR at all visits. Patients were classified as in remission, intermediately active (between remission and trial active) or trial active at each visit. We defined remission (using a variant of the ARA Remission Criteria which omits fatigue [7]) as EMS ≤ 15 min, ESR<30 mm/h (females) or <20 mm/h (males), no joint swelling, no joint tenderness and no joint pain. We defined trial active disease using accepted criteria (three out of four of tender joints ≥6, swollen joints ≥6, ESR ≥ 28 mm/h and EMS ≥ 45 min). We calculated DAS28 using the standard formula (with ESR).

The impact of changing trial entry criteria was assessed using two types of modification. First, using Cohort A (504 consecutive cases), we examined simplified versions of the existing criteria (and other single/composite measures), which replicated the level of disease activity defined by the current trial entry criteria. Second, using Cohort C (94 cases starting new treatment), we selected criteria replicating the level of disease activity in this cohort of patients considered by clinicians in routine practice to need a new DMARD. For each definition, the numbers of patients defined as trial active and, where appropriate, the sensitivity and specificity compared with current criteria, were calculated.


    Results
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 References
 
Applying trial entry criteria for disease activity in routine clinical practice
Disease activity at single time points
The 504 patients seen routinely (Cohort A) had an average of 5.1 swollen joints (95% CI 4.8, 5.4), 7.9 tender joints (95% CI 7.3, 8.6), an ESR of 34 mm/h (95% CI 31, 36) and 92 min EMS (95% CI 82, 102). Patients followed longitudinally (Cohort B) had similar initial clinical assessments: 4.7 swollen joints (95% CI 4.1, 5.4), 5.9 tender joints (95% CI 5.0, 6.9), an ESR of 31 mm/h (95% CI 27, 35) and 101 min EMS (95% CI 81, 120). As expected, patients starting new DMARDs or biologics (Cohort C) had higher disease activity: 7.5 swollen joints (95% CI 6.5, 8.4), 10.0 tender joints (95% CI 8.5, 11.6), an ESR of 49 mm/h (95% CI 42, 56) and 146 min EMS (95% CI 118, 173).

One hundred and eighty-nine (38%) patients in Cohort A had ‘trial active’ disease, 299 (59%) were intermediately active and 16 (3%) were in remission. Longitudinal Cohort B patients at their first visit were similar: 50 (32%) had ‘trial active’ disease, 98 (63%) were intermediately active and 8 (5%) were in remission. In contrast, almost twice as many (58; 62%) Cohort C patients had ‘trial active’ disease; however, a significant number still did not meet trial entry criteria with 36 (38%) intermediately active and none in remission. Whilst this could reflect a difference in the level of activity required for trial entry compared with that precipitating a change of drug in practice, it could also be that some patients were changing therapy for other reasons such as toxicity.

Disease activity during longitudinal follow-up of Cohort B (Table 1)
Only 43 patients (28%) stayed in their initial disease activity group and 10 (6%) spent periods in all three groups. At most visits, patients were intermediately active with only 28 (18%) in remission at any time point. Importantly, 68% (106) Cohort B patients, irrespective of initial activity classification, met trial entry criteria on at least one occasion.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Changes in disease activity group during follow-up in Longitudinal Cohort B (156 patients followed for four or more visits by a single rheumatologist)

 
Modification of existing trial entry criteria: with similar levels of disease activity
Reducing or omitting an existing criterion and leaving others unaltered (Table 2)
Removing EMS made little difference to classification (5% of patients changed) or sensitivity (100% to 92%); in contrast removing any other criterion made a major difference to both. Lesser changes in swollen/tender joints or ESR led to only minor classification changes.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Effects of different definitions of trial active RA in 504 cases in cross-sectional study (cross-sectional Cohort A)

 
Simplification by using a single criterion for disease activity (Table 2)
Using only one of the current criteria to assess activity altered classification in 16–31% of cases with concomitant major specificity and sensitivity changes confirming it is not appropriate to focus on one factor in a multisystem disease.

Simplification by using a composite measure (Table 2)
DAS is commonly used to assess activity and response in routine practice and RCTs but not for trial recruitment. DAS28 levels from 5.1 (common cut-off for high disease activity) to 5.5 were studied. DAS28 ≥5.4 best replicated current entry criteria, defining an identical proportion (38%) of patients as active; however, 68 (13%) patients changed classification, suggesting that the population identified is different. This could not be explained by the inclusion of EMS in current criteria since current criteria minus EMS still classified 13–15% of cases differently to DAS.

Modification of existing trial entry criteria: replicating clinicians’ routine practice
Fifty-eight patients (62%) in Cohort C were classified as ‘trial active’ by current criteria whereas criteria designed to replicate routine practice should classify most as active. Low-stringency criteria (three out of four of one tender joint, one swollen joint, ESR greater double equals 20 mm/h and EMS greater double equals 15 min) classified 90 (96%) cases as active. On substituting greater double equals3 tender and greater double equals3 swollen joints into these, 80 (85%) cases were defined as active; on substituting greater double equals4 tender and greater double equals4 swollen joints, this declined further to 72 (77%). In all instances EMS had minimal impact.

Using DAS-based criteria, DAS28 greater double equals3.1 (lowest level conventionally defined as active), classified 92 (98%) of these patients as active; increasing the level of DAS reduced the number of trial active patients.


    Discussion
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 References
 
This study investigated two aspects of trial entry. Firstly we examined whether trial patients are representative of routine clinic patients in terms of disease activity. Our cross-sectional studies showed that many patients seen on a single occasion did not have ‘trial active’ disease. This is in keeping with reports by Sokka and Pincus [4, 5] and an associated editorial by Kvein et al. [8]. However, these previous reports have presented an incomplete picture because they did not consider temporal changes in disease activity. We therefore extended research in this area by showing, in a longitudinal cohort followed for 2 yr, that most patients meet current trial entry criteria at least once. This indicates that patients entering RCTs for DMARDs and biologics represent a subset of routine patients (not a distinct population) selected at a point in their disease course when they are most active, the very time at which new DMARD therapy is likely to be considered in practice. Consistent with this, a much higher proportion of patients changing therapy met trial entry criteria. We conclude that RCTs using current criteria are able to establish treatment efficacy in a manner generalizable to routine practice.

The second aspect we investigated was whether alternative trial entry criteria are simpler, less restrictive and more consistent with standard disease activity and response measures. This is potentially possible as trials have employed entry criteria with varying numbers of tender or swollen joints [9], levels of ESR [10] and durations of morning stiffness [11], and have combined domains differently [12]. We recommend abandoning early morning stiffness in the definition of ‘trial active’ RA because it has little impact on classification and it is not one of the outcome assessments currently used in RCTs or clinical practice. The other assessments—swollen joint counts, tender joint counts and ESR—appear essential. However, they could be incorporated into one measure, namely DAS, that is widely used as a composite outcome measure. Using DAS is rational because it was derived using routine clinic patients with early RA changing DMARDs [13]. Furthermore using the same measure to assess suitability for entry and trial outcome is simple and logical. Using a single summary assessment, like the physicians’ global assessment, which has the attraction of simplicity, was tried by McConkey and colleagues but was not pursued due to variations between individual clinicians and centres [14]. We conclude there is a strong case for rationalizing the definition of ‘trial active’ RA by focusing on tender and swollen joint counts and ESR; these could be combined into one summary measure, the DAS.

There is continual debate on optimal outcome measures for RCTs [15] including radical suggestions like time on treatment [16] or changes in HAQ scores [17]. Similar debate is needed around trial entry criteria. Different criteria may be needed in trials undertaken for different purposes. Trials with entry criteria that identify patients with a high level of disease activity may be appropriate for establishing the efficacy of new drugs. However, criteria identifying a lower level of disease activity may be more relevant in defining how best to further improve moderately active disease with adjunctive therapy.

Finally, we appreciate that no single group can provide a definitive answer on optimal entry criteria for clinical effectiveness trials. Rather than providing definitive recommendations, our intention has been to highlight the need for further work to achieve a new international consensus. This will need to involve a wide range of clinicians with an interest in the field.


    Acknowledgments
 
We are grateful to the ARC (http://www.arc.org.uk) for supporting the programme of research in our unit and the Kings College Hospital Trustees for support for Ms Khoshaba. We would also like to acknowledge support to Kings College Hospital and University Hospital Lewisham from the UK National Health Service (NHS) Research and Development Programme.

We would also like to acknowledge the help of clinical colleagues at Kings College Hospital, University Hospital Lewisham and Queen Mary's Hospital Sidcup in allowing their patients to be recruited for this study.

The authors have declared no conflicts of interest.


    References
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 References
 

  1. Cooper C, Choy E, Arthritis Research Campaign's Clinical Trials Committee. The blossoming of evidence-based clinical rheumatology: The Arthritis Research Campaign's Clinical Trials Collaboration in association with the MRC Clinical Trials Unit, BSR and BOA. Rheumatology 2003;42:713–15.[Free Full Text]
  2. Pincus T, Stein CM. Why randomized controlled clinical trials do not depict accurately long-term outcomes in rheumatoid arthritis: some explanations and suggestions for future studies. Clin Exp Rheumatol 1997;15(Suppl 17):S27–S38.[Medline]
  3. Dieppe P, Bartlett C, Davey P, Doyal L, Ebrahim S. Balancing benefits and harms: the example of non-steroidal anti-inflammatory drugs. Br Med J 2004;329:31–4.[Free Full Text]
  4. Sokka T, Pincus T. Eligibility of patients in routine care for major clinical trials of anti-tumor necrosis factor alpha agents in rheumatoid arthritis. Arthritis Rheum 2003;48:313–18.[CrossRef][ISI][Medline]
  5. Sokka T, Pincus T. Most patients receiving routine care for rheumatoid arthritis in 2001 did not meet inclusion criteria for most recent clinical trials or American College Of Rheumatology criteria for remission. J Rheumatol 2003;30:1138–46.[ISI][Medline]
  6. Arnett FC, Edworthy SM, Bloch DA et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum 1988;31:315–24.[ISI][Medline]
  7. Molenaar ET, Voskuyl AE, Dinant HJ, Bezemer PD, Boers M, Dijkmans BA. Progression of radiologic damage in patients with rheumatoid arthritis in clinical remission. Arthritis Rheum 2004;50:36–42.[CrossRef][ISI][Medline]
  8. Kvien TK, Mikkelsen K, Nordvag BY. Results from controlled clinical trials: how relevant for clinical practice? J Rheumatol 2003;30:1135–7.[ISI][Medline]
  9. Bathon JM, Martin RW, Fleischmann RM et al. A comparison of etanercept and methotrexate in patients with early rheumatoid arthritis. N Engl J Med 2000;343:1586–93.[Abstract/Free Full Text]
  10. Ahern MJ, Harrison W, Hollingsworth P, Bradley J, Laing B, Bayliss C. A randomized double-blind trial of cyclosporin and azathioprine in refractory rheumatoid arthritis. Aust NZ J Med 1991;21:844–9.[ISI][Medline]
  11. Furst D, Felson D, Thoren G, Gendreau RM. Immunoadsorption for the treatment of rheumatoid arthritis: final results of a randomized trial. Ther Apheresis 2000;4:363–73.[CrossRef]
  12. Tugwell P, Bombardier C, Gent M et al. Low-dose cyclosporin versus placebo in patients with rheumatoid arthritis. Lancet 1990;335:1051–5.[CrossRef][ISI][Medline]
  13. van der Heijde DM, van't Hof M, van Riel PL, van de Putte LB. Development of a disease activity score based on judgment in clinical practice by rheumatologists. J Rheumatol 1993;20:579–81.[ISI][Medline]
  14. Grindulis KA, Calverley M, Constable TJ, Forster PJ, Ahmed ME, McConkey B. A comparison between clinical and laboratory tests in rheumatoid arthritis. Scand J Rheumatol 1983;12:285–8.[ISI][Medline]
  15. Anderson JJ, Bolognese JA, Felson DT. Comparison of rheumatoid arthritis clinical trial outcome measures: a simulation study. Arthritis Rheum 2003;48:3031–8.[CrossRef][ISI][Medline]
  16. Wolfe F, Michaud K, Stephenson B, Doyle J. Toward a definition and method of assessment of treatment failure and treatment effectiveness: the case of leflunomide versus methotrexate. J Rheumatol 2003;30:1725–32.[ISI][Medline]
  17. Tijhuis GJ, Zwinderman AH, Hazes JM, Van Den Hout WB, Breedveld FC, Vliet Vlieland TP. A randomized comparison of care provided by a clinical nurse specialist, an inpatient team, and a day patient team in rheumatoid arthritis. Arthritis Rheum 2002;47:525–31.[CrossRef][Medline]
Submitted 4 October 2004; revised version accepted 7 January 2005.



This Article
Abstract
Full Text (PDF)
All Versions of this Article:
44/5/629    most recent
keh565v1
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Search for citing articles in:
ISI Web of Science (1)
Disclaimer
Request Permissions
Google Scholar
Articles by Kingsley, G. H.
Articles by Scott, D. L.
PubMed
PubMed Citation
Articles by Kingsley, G. H.
Articles by Scott, D. L.
Related Collections
Rheumatoid Arthritis