Patient- versus physician-reported outcomes in rheumatoid arthritis patients treated with recombinant interleukin-1 receptor antagonist (anakinra) therapy
S. B. Cohen,
V. Strand1,
D. Aguilar2 and
J. J. Ofman2
Radiant Research, Dallas, Texas, 1 Stanford University Medical Center, Palo Alto, California and 2 Zynx Health, Los Angeles, California, USA.
Correspondence to: Stanley B. Cohen, Radiant Research, 5939 Harry Hines Boulevard, Suite 400, Dallas, Texas 75235, USA. E-mail: Stanleycohen{at}radiantresearch.com
 |
Abstract
|
---|
Objectives. To determine whether patient-reported outcomes may differentiate treatment response better than physician-reported outcomes for rheumatoid arthritis (RA) patients being treated with anakinra.
Methods. A meta-analysis was conducted using data obtained from three separate randomized controlled clinical trials (RCTs) (n = 1007). Outcomes from 6-month assessments were grouped into four categories: American College of Rheumatology (ACR) response criteria, patient-reported measures (patient-reported pain, patient global assessment, and assessment of physical function using the Health Assessment Questionnaire), physician-reported measures (tender and swollen joint counts and physician global assessment), and laboratory tests (C-reactive protein and erythrocyte sedimentation rate). Effect sizes were calculated using changes from baseline and pooled standard deviations for each of these types of outcome.
Results. Active treatment with anakinra was superior to placebo by ACR20 responses in all three RCTs. Effect sizes for patient-reported outcomes were greater than for physician-reported outcomes, and also greater than ACR20 in three of five anakinra cohorts. Across the RCTs, placebo responses were greater with physician-reported than with patient-reported outcomes. In the two studies evaluating patients with longer-standing disease, differences between pooled effect sizes for patient-reported and physician-reported outcomes were even more pronounced.
Conclusions. In three pivotal RCTs, active treatment with anakinra resulted in greater improvements in patient-reported than physician-reported outcomes compared with placebo. These observations confirm those previously reported from RCTs evaluating conventional DMARDs, demonstrating better discrimination of treatment effect with patient-reported outcomes.
KEY WORDS: Health status, Joints, Meta-analysis, Pain measurement, Randomized controlled trials, Recombinant interleukin-1 receptor antagonist, Rheumatoid arthritis, Treatment outcome.
 |
Introduction
|
---|
Rheumatoid arthritis (RA) is a chronic, inflammatory disease of unknown aetiology that affects 2.1 million American adults [1, 2]. Patients with RA typically experience a progressive decline in physical function, initially due to pain and inflammation and subsequently due to structural damage [3]. Recent advances in therapy have provided hope that long-term outcomes will improve for these patients.
Randomized controlled trials (RCTs) have previously demonstrated that treatment with methotrexate, leflunomide or sulphasalazine significantly improve signs and symptoms of active disease, radiographic progression, and physical function and health-related quality of life over 12 yr [48]. Improvement in signs and symptoms was demonstrated using American College of Rheumatology (ACR) response criteria. The ACR core set of outcome measures include physician-reported measures (swollen joint count, tender joint count and physician global assessment), patient-reported measures (global assessment of disease activity, pain and physical function) and laboratory parameters [C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR)] [9]. An ACR20 response is defined as 20% improvement in swollen and tender joints along with a >20% improvement in three of the five other outcome measures [9]. The ACR20 has been used by regulatory agencies to facilitate approval of new therapeutic agents for the treatment of RA based upon a significant response compared with placebo [10].
In recent years, improvements in patient-reported outcomes have been increasingly emphasized by clinicians and regulatory bodies because changes in these measures reflect changes most important to patients [5, 1014]. Moreover, recent research suggests that patient-reported measures may be more sensitive to differences between drug and placebo groups than physician-assessed measures [13]. For example, Goetzche found that patient assessment of change was among the most sensitive outcome variables in a meta-analysis of placebo-controlled randomized trials assessing non-steroidal anti-inflammatory drugs (NSAIDs) [15].
A recent analysis of two of the RCTs [4, 16] comparing leflunomide with methotrexate and sulphasalazine indicated that patient-reported outcomes may be less susceptible to the placebo response [1113]. Patient-reported outcomes were more likely to reflect treatment effects compared with physician-based measures.
Biological disease-modifying anti-rheumatic drugs (DMARDs) that inhibit tumour necrosis factor
and interleukin-1 have recently been approved for treatment of patients with moderate to severe RA.
Recent RCTs have demonstrated significant improvement in patient outcomes with these agents (etanercept, infliximab, adalimumab and anakinra) over 6 months to 2 yr of follow-up [1722].
Review of published data from these RCTs suggests that patient-reported outcomes may be more sensitive indicators of treatment effects than physician-reported outcomes. To further explore these observations, we performed a meta-analysis of three RCTs comparing treatment with anakinra and placebo in patients with active RA.
 |
Methods
|
---|
Six-month results from three RCTs of anakinra were included in the analysis. Data were obtained from published articles as well as from an Amgen (Thousand Oaks, CA, USA) Biologic License Application (BLA) that may not have previously been published. Only results for groups assigned to receive standard doses (12 mg/kg/day or fixed dosage 100 mg/day) of anakinra were included as they were similar or equivalent to the dose approved for treatment of active RA. Two reviewers abstracted data and resolved any discrepancies by discussion.
 |
Study designs
|
---|
Study 1 was a randomized, double-blind, and placebo-controlled 24-week study [19]. Patients with active RA (by ACR criteria) and experiencing symptoms for more than 6 months and less than 8 yr were included. All DMARD use was discontinued 6 weeks prior to enrollment, and corticosteroid doses remained constant throughout the study period. Patients were randomized to one of four groups: placebo, 30 mg anakinra, 75 mg anakinra and 150 mg anakinra. We included only results from the 75 and 150 mg anakinra and placebo groups (353 patients) as these dosages are comparable to the 100 mg/day dosage approved by the Food and Drug Administration (FDA).
Study 2 [20] was a 24-week multicentre, randomized, double-blind, placebo-controlled trial. Patients were included if they had been diagnosed with RA for more than 6 months and less than 12 yr. Eligible patients were required to have active RA and to have been on methotrexate for at least 6 consecutive months, including a minimum of 3 months at a stable weekly dose of 1025 mg immediately preceding the start of the study. With the exception of methotrexate, patients were required to discontinue all DMARDs at least 8 weeks prior to enrolment. Corticosteroid and NSAID doses remained unchanged throughout the study period. Patients were randomized to one of six study groups: placebo, 0.04 mg/kg anakinra, 0.1 mg/kg anakinra, 0.4 mg/kg anakinra, 1 mg/kg anakinra or 2 mg/kg anakinra daily subcutaneously. We included 6-month results from the 1 and 2 mg/kg anakinra and placebo groups (153 patients).
Study 3 [21] reported results for the first 24-weeks of a 2-yr multicentre, randomized, double-blind (first year only), placebo-controlled trial. Patients were required to have had active RA for at least 6 months and radiographic evidence of at least a single bone erosion in the feet, wrists or hands. Prior to randomization, participants were on stable methotrexate (1025 mg per week) for at least 6 months and maintained treatment while on study. Methotrexate use and any use of corticosteroids and/or NSAIDS could not change during the 4 weeks prior to study initiation. Patients were randomized to one of two study groups: 100 mg of anakinra or placebo (501 patients total).
In each RCT, the absolute response rate or least-squares mean changes and standard errors were abstracted. Mean changes for study 1 were adjusted for baseline values and study country, and the principle of LOCF (last observation carried forward) was used to address missing data. Mean changes for studies 2 and 3 were adjusted for baseline values and study centre. Additionally, for studies 2 and 3 a repeated measures mixed model analysis of covariance was used to assess treatment effect. For the mixed model approach, the data collected for all subjects prior to their withdrawal from the trial were included, and subjects without postbaseline data at week 24 had their baseline value imputed for their week-4 visit only. For visits subsequent to week 4, no imputation of absent data was carried out, and all missing data remained as such.
 |
Outcomes assessed
|
---|
All outcomes were assessed following 6 months of blinded protocol treatment. Outcome measures were placed into four categories: ACR response criteria, patient-reported measures, physician-reported measures, and laboratory tests.
Patient-reported measures included self-reports of pain, global assessment of disease activity, and physical function using the Health Assessment Questionnaire (HAQ). Study 1 utilized a visual analogue scale (VAS) to measure pain, with scores ranging from 0 to 1. In study 1, patient global assessment was determined using a five-point scale (04). For studies 2 and 3, pain and patient global assessment were measured on a 100-mm VAS. The measure of physical function, the HAQ, consists of 20 questions covering the following eight subscales: (1) dressing, (2) arising, (3) eating, (4) walking, (5) hygiene, (6) reach, (7) grip, and (8) common activities. Patients were asked to rate the difficulty of performing these activities on a four-point scale (03), with 0 indicating no difficulty and higher values representing greater impairment of physical function [23, 24].
Physician-reported outcomes included tender and swollen joint counts and physician global assessment. Each study assessed 68 diarthrodial joints for tenderness and 66 diarthrodial joints for swelling. Study 1 used a five-point scale (04) to measure physician global assessment. For studies 2 and 3, physician global assessment was measured on a 100-mm VAS.
Laboratory tests included CRP and ESR.
 |
Meta-analysis
|
---|
A general method for combining different measures of effect involves the calculation of effect size [2529]. An effect size is a standardized measure of the treatment effect. Effect sizes were calculated for the mean change for each outcome measure, using the difference in change from baseline to 6 months for treated and control groups as well as the pooled standard deviation. The effect size is defined as the difference between the changes in treatment and placebo groups divided by the pooled (treatment and placebo) standard deviation of the change. To calculate the effect sizes for proportions achieving ACR score targets (ACR20, ACR50 and ACR70), we took the natural logarithm (ln) of the odds ratio multiplied by the square root of 3 and divided by
[30].
2 statistics were calculated to test equality of proportions of patients achieving ACR criteria.
Effect sizes for each outcome measure were pooled within each outcome category (patient-reported measures, physician-reported measures and laboratory measures). Each active treatment arm (contrasted with its corresponding placebo group) contributed an effect size estimate for each outcome. Effect sizes were pooled within each outcome category and placebo using a random effects model (empirical Bayes method of Hedges and Olkin) using Fastpro software (Academic Press, USA) [29, 31]. Ninety-five per cent confidence intervals (95% CI) for all effect sizes were also obtained using the Fastpro statistical software.
 |
Results
|
---|
Study descriptions
Complete information on sample sizes, subject demographics and baseline disease status and a summary of baseline values for end-point measures are presented in Tables 1 and 2 for all three RCTs. (Note that data for study 2 includes both 3- and 6-month samplesonly the 6-month sample was used for outcome evaluation).
 |
ACR response
|
---|
Table 3 shows the percentage of patients achieving various levels of improvement in ACR response criteria. As previously reported, anakinra was superior to placebo, based on ACR20, ACR50, and ACR70 responses in all three RCTs at the specified dosages. Treatment was statistically superior to placebo in 3 of 5, 4 of 5 and 2 of 5 treatment-placebo comparisons for ACR20, ACR50, and ACR70 respectively (Table 3).
 |
Effect sizes
|
---|
Mean changes in individual components of the ACR response criteria are presented in Table 4. Table 5 lists effect size with 95% CIs by study outcome for each treatment-placebo comparison, and also lists pooled effect sizes with 95% CI by outcome category (laboratory, patient- and pooled physician-reported outcomes). Pooled effect sizes for patient-reported measures were greater than those for pooled physician-reported outcomes in all five treatmentplacebo comparisons, and were also greater than pooled laboratory measures in 4 of 5 treatment-placebo comparisons. Differences were more pronounced for the two combination studies evaluating patients with longer-standing disease. Additionally, effect sizes for patient-reported outcomes were greater than those for the ACR20 response score in 3 of 5 anakinra cohorts, though this was not the case for the larger study (study 3).
View this table:
[in this window]
[in a new window]
|
TABLE 5. Effect size with 95% CI for ACR response, patient-reported, physician-reported and laboratory outcomes by study arm
|
|
Table 6 presents the percentage change from baseline in patient-reported and physician-reported assessments for all three RCTs. The smallest percentage change from baseline in patients receiving placebo was noted for laboratory tests (ESR and CRP), and then for patient-reported outcomes of pain, patient global assessment and HAQ. Minimal improvements were seen in studies 1 and 2 in these individual components. A greater percentage response to placebo was seen for physician-reported outcomes in all three studies. Considerable improvement was seen in all studies for the anakinra-treated patients.
 |
Global assessments
|
---|
Tables 4 and 6 also present changes from baseline for patient and physician global assessments by study arm. Examination of global assessments offers an opportunity to compare results for conceptually similar and identically scaled outcomes assessed by physicians and patients. Greater improvements were noted by physicians than by patients in all three placebo groups, the differences being substantial in studies 2 and 3. Effect sizes were greater for the patient as opposed to the physician global assessments for all five treatment groups compared with placebo.
 |
Discussion
|
---|
As previously reported, results from these three RCTs demonstrated superiority of anakinra to placebo either as monotherapy or in combination with methotrexate, as determined by the ACR response criteria [1921]. Based on a previous report of greater sensitivity to treatment effect with the patient-reported outcomes of the ACR response criteria in RCTs involving conventional DMARDs [1113], we performed an analysis of effect sizes for the physician- and patient-reported components of the ACR response criteria in the anakinra RCTs. In each of these three RCTs (five treatmentplacebo comparisons), the pooled effect sizes for patient-reported outcomes were larger than those for physician-reported outcomes and exceeded the ACR20 response in 3 of the 5 anakinraplacebo comparisons. These findings suggest that patient-reported measures may be more sensitive to treatment effects than are physician-reported measures.
Additionally in these RCTs, physician-reported outcomes demonstrated greater improvement in placebo-treated patients over baseline than patient-reported outcomes. This difference is particularly striking for global assessments in studies 2 and 3 (Tables 4 and 6). This observation is consistent with results from similar analysis of two placebo-controlled trials comparing leflunomide with methotrexate and sulphasalazine for treatment of active RA [32].
Multiple hypotheses have been advanced to explain the observed improvements seen in placebo-treated patients with RA and other autoimmune diseases. Patient-reported assessments involve only one observer, the patient. Physician-reported assessments require two observers, the physician and the patient, which in theory could increase subjectivity in reporting. Physicians tend to be hopeful that their patients will respond to treatment and could succumb to expectation bias; similarly, patients may be more reticent in reporting disappointment or more willing to report improvement to their physician than to themselves. These observations are important, as they can inform us about future RCT results where placebo treatment is increasingly superimposed upon failed active therapy.
An alternative explanation to greater treatment effect with patient-reported outcomes is that anakinra may be more effective in improving patient-reported pain and physical function than improving swollen and tender joint counts in the populations of patients enrolled in these trials. This is unlikely, as noted in Table 6. The magnitude of the percentage response for anakinra-treated patients for physician-reported outcomes was numerically greater than the response for patient-reported outcomes in all three RCTS.
Some information relevant to this hypothesis can be gained by examining the results of the physician and patient global assessments within each of the three trials. Since these two assessments are measured on the same scale within each trial (though the scales differ across trials), direct comparison is possible. As Table 4 shows, standard errors for patient global assessments are very similar in magnitude to those for physician global assessments. Though this observation may not hold for other outcomes or in other trials, it does weigh against the hypothesis that differences in random error explain the observed differences in effect sizes.
When The Outcome Measures in Rheumatology Clinical Trials (OMERACT) Committee and the Ad Hoc Committee for the American College of Rheumatology developed the ACR response criteria, the lesser ability to detect true treatment response in RCTs with physician-reported outcomes was noted [9]. The committee examined 40 definitions of improvement for RA patients before the decision to recommend the present criteria. The candidate definitions for improvement that relied primarily on joint count outcomes had the highest frequency of false positives in a survey of rheumatologists as well as when applied to clinical trial results. As part of the process, the committee commented Compared with more comprehensive measures, definitions that depend on joint count generally do not discriminate as well between active drug-treated and placebo-treated patients and usually identified more placebo-treated patients as being improved.
Our analysis shows that the pooled effect size for the patient-reported outcomes was of similar magnitude or greater than the effect size for ACR20 response in all five treatmentplacebo comparisons. The results of this analysis do not suggest that patient-reported outcomes are superior to the composite ACR response criteria as the target outcome for RA clinical trials. These observations, however, do support the results of Pincus et al., who reported that patient-reported assessments were able to detect treatment effect as well as the ACR response criteria in their evaluation of the leflunomide data set, and suggest that clinical trials could be performed using only a questionnaire measuring patient-reported outcomes [12].
The results of this analysis demonstrate that in all instances the effect sizes for the pooled patient-reported outcomes are higher than the corresponding effect sizes for the pooled physician-reported outcomes. This apparent consistency must be interpreted with caution due to the small number of placebo/treatment comparisons examined. Five of five instances is not a statistically significant result using the two-sided 5% level sign test. Moreover, these five instances may not be independent of each other. There are only three clinical trials and thus only three placebo groups; in two of the trials two different doses were evaluated against the same placebo group. Therefore, these results are only suggestive of greater sensitivity to treatment effects of patient- vs physician-reported measures. Additional limitations of these analyses are that the confidence intervals may be underestimated because some of the outcome measures may be correlated with one another and the calculated effect sizes for different outcome measures may be related to the baseline scores.
This analysis suggests that the benefits of active treatment in RA patients with anakinra are more evident by patient-reported than physician-reported outcomes. This analysis confirms the previous observations from two RCTs evaluating conventional DMARD treatment in active RA, indicating that patient assessments of disease activity, pain and physical function are more sensitive to active treatment and better discriminate active from placebo therapy. Clinicians and researchers should consider assigning greater importance to patient-reported measures as indicators of treatment effectiveness.
J. Ofman and D. Aguilar were employed at the time of study by Zynx Health, a Cerner Company. Zynx Health, V. Strand and S. Cohen have received research grants and/or consulting payments from Amgen Inc., including funding for this study. Since conclusion of this study, J. Ofman became an associate of Amgen Inc., which manufactures and markets drugs used in the treatment of rheumatoid arthritis (Enbrel and Anakinra).
 |
Acknowledgments
|
---|
This study was supported by a research grant from Amgen, Thousand Oaks, CA, USA.
 |
References
|
---|
- Lawrence RC, Helmick CG, Arnett FC et al. Estimates of the prevalence of arthritis and selected musculoskeletal disorders in the United States. Arthritis Rheum 1998;41:77899.[CrossRef][ISI][Medline]
- Goronzy JJ, Weyand CM. Arthritis foundation. Rheumatoid arthritis: A. Epidemiology, Pathology, and Pathogenesis. Available at: http://www.arthritis.org
- Scott DL, Pugner K, Kaarela K et al. The links between joint damage and disability in rheumatoid arthritis. Rheumatology 2000;39:12232.[Abstract/Free Full Text]
- Strand V, Cohen S, Schiff M et al. Treatment of active rheumatoid arthritis with leflunomide compared with placebo and methotrexate. Leflunomide Rheumatoid Arthritis Investigators Group. Arch Intern Med 1999;159:254250.[Abstract/Free Full Text]
- Strand V, Tugwell P, Bombardier C et al. Function and health-related quality of life: results from a randomized controlled trial of leflunomide versus methotrexate or placebo in patients with active rheumatoid arthritis. Leflunomide Rheumatoid Arthritis Investigators Group. Arthritis Rheum 1999;42:18708.[CrossRef][ISI][Medline]
- Scott DL, Smolen JS, Kalden JR et al. Treatment of active rheumatoid arthritis with leflunomide: two year follow up of a double blind, placebo controlled trial versus sulfasalazine. Ann Rheum Dis 2001;60:91323.[Abstract/Free Full Text]
- Kalden JR, Scott DL, Smolen JS et al. Improved functional ability in patients with rheumatoid arthritislongterm treatment with leflunomide versus sulfasalazine. European Leflunomide Study Group. J Rheumatol 2001;28:198391.[ISI][Medline]
- Larsen A, Kvien TK, Schattenkirchner M et al. Slowing of disease progression in rheumatoid arthritis patients during long-term treatment with leflunomide or sulfasalazine. Scand J Rheumatol 2001;30:13542.[CrossRef][ISI][Medline]
- Felson DT, Anderson JJ, Boers M et al. American College of Rheumatology. Preliminary definition of improvement in rheumatoid arthritis. Arthritis Rheum 1995;38:72735.[ISI][Medline]
- The Food and Drug Administration, Center for Drug Evaluation and Research: Guidance for Industry. Clinical Development Programs for Drugs, Devices, and Biological Products for the Treatment of Rheumatoid Arthritis (RA). Available at: http://www.fda.gov/cder/guidance/1208fnl.htm#1). Accessed November 17, 2003.
- Tugwell P, Wells G, Strand V et al. Clinical improvement as reflected in measures of function and health-related quality of life following treatment with leflunomide compared with methotrexate in patients with rheumatoid arthritis: sensitivity and relative efficiency to detect a treatment effect in a twelve-month, placebo-controlled trial. Leflunomide Rheumatoid Arthritis Investigators Group. Arthritis Rheum 2000;43:50614.[CrossRef][ISI][Medline]
- Pincus T, Strand V, Koch G et al. An index of the three core data set patient questionnaire measures distinguishes efficacy of active treatment from that of placebo as effectively as the American College of Rheumatology 20% response criteria (ACR20) or the Disease Activity Score (DAS) in a rheumatoid arthritis clinical trial. Arthritis Rheum 2003;48:62530.[CrossRef][ISI][Medline]
- Strand V, Cannon G, Cohen S, Ware J, Crawford B, Doyle J. Correlation of health assessment questionnaire (HAQ) with Short Form 36 (SF-36): Comparison of leflunomide (LEF) with methotrexate (MTX) in patients with active rheumatoid arthritis (RA) [abstract]. In: ACR/ARHP 65th Annual Scientific Meeting, November 1115, 2001; San Francisco, CA.
- Scott DL, Strand V. The effects of disease-modifying anti-rheumatic drugs on the Health Assessment Questionnaire score. Lessons from the leflunomide clinical trials database. Rheumatology 2002;41:899909.[Abstract/Free Full Text]
- Gotzsche PC. Sensitivity of effect variables in rheumatoid arthritis: a meta-analysis of 130 placebo controlled NSAID trials. J Clin Epidemiol 1990;43:13138.[ISI][Medline]
- Cohen S, Cannon GW, Schiff M et al. Two-year, blinded, randomized, controlled trial of treatment of active rheumatoid arthritis with leflunomide compared with methotrexate. Utilization of Leflunomide in the Treatment of Rheumatoid Arthritis Trial Investigator Group. Arthritis Rheum 2001;44:198492.[CrossRef][ISI][Medline]
- Mathias SD, Colwell HH, Miller DP, Moreland LW, Buatti M, Wanke L. Health-related quality of life and functional status of patients with rheumatoid arthritis randomly assigned to receive etanercept or placebo. Clin Ther 2000;22:12839.[CrossRef][ISI][Medline]
- Maini R, St Clair EW, Breedveld F et al. Infliximab (chimeric anti-tumour necrosis factor alpha monoclonal antibody) versus placebo in rheumatoid arthritis patients receiving concomitant methotrexate: a randomised phase III trial. ATTRACT Study Group. Lancet 1999;354:19329.[CrossRef][ISI][Medline]
- Bresnihan B, Alvaro-Gracia JM, Cobby M et al. Treatment of rheumatoid arthritis with recombinant human interleukin-1 receptor antagonist. Arthritis Rheum 1998;41:2196204.[CrossRef][ISI][Medline]
- Cohen S, Hurd E, Cush J, Schiff M et al. Treatment of rheumatoid arthritis with anakinra, a recombinant human interleukin-1 receptor antagonist, in combination with methotrexate: results of a twenty-four-week, multicenter, randomized, double-blind, placebo-controlled trial. Arthritis Rheum 2002;46:61424.[CrossRef][ISI][Medline]
- Cohen SB, Moreland LW, Cush JJ et al. Anakinra (recombinant interleukin-1 receptor antagonist): A large, placebo controlled efficacy trial of anakinra in patients with erosive rheumatoid arthritis disease [abstract]. In: ACR/ARHP 65th Annual Scientific Meeting; November 1115, 2001, San Francisco, CA.
- Weinblatt ME, Keystone EC, Furst DE et al. Adalimumab, a fully human anti-tumor necrosis factor alpha monoclonal antibody, for the treatment of rheumatoid arthritis in patients taking concomitant methotrexate: the ARMADA trial. Arthritis Rheum 2003;48:3545.[CrossRef][ISI][Medline]
- Ramey DR, Fries JF, Singh G. The Health Assessment Questionnaire 1995 status and review. In: B. Spiker, ed. Quality of life and pharmacoeconomics in clinical trials, 2nd edn. Philadelphia: Lippincott-Raven, 1996.
- Fries JF, Spitz P, Kraines RG, Holman HR. Measurement of patient outcome in arthritis. Arthritis Rheum 1980;23:13745.[ISI][Medline]
- Cohen J. Statistical power analysis for the behavior sciences, 2nd edn. New York: Academic Press, 1977.
- Rosenthal R, Rubin D. Combining results from independent studies. Psychol Bull 1979;85:18593.[CrossRef][ISI]
- Glass G. Summarizing effect sizes. New directions for methodology of social and behavioral science: Quantitative assessment of research domains. San Francisco: Jossey-Bass, 1980.
- Rosenthal R. Meta-analytic procedures for social research. Beverly Hills: Sage Publications, 1984.
- Hedges LV, Olkin I. Statistical methods for meta-analysis. Orlando: Academic Press, 1985.
- Hasselblad V, Hedges LV. Meta-analysis of screening and diagnostic tests. Psychol Bull 1995;117:16778.[CrossRef][ISI][Medline]
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:17788.[CrossRef][ISI][Medline]
- Cohen S, Smolen J, Tugwell P, Crawford B. Patient-based functional measures of drug treatment in rheumatoid arthritis trials are not susceptible to placebo effect [abstract]. In: ACR/ARHP 64th Annual Scientific Meeting; October 29 November 2, 2000, Philadelphia, PA.
Submitted 28 August 2003;
revised version accepted 14 January 2004.