The Division of General Internal Medicine, University of Alberta and The Institute of Health Economics, Edmonton, Alberta, Canada.
Dr F McAlister, 2E3.24 WMC, University of Alberta Hospital, 8440 112 Street, Edmonton, Alberta, Canada T6G 2R7. E-mail: Finlay.McAlister{at}ualberta.ca
To practice evidence-based therapeutics, clinicians have to integrate measures of efficacy and safety from the literature with their patient's unique risks and values.1 To do this requires two assumptions. First, we assume that we can accurately estimate our patient's underlying baseline risk (or the expected event rate referred to by Furukawa and colleagues).2 This is no simple matter: while clinicians appear to be reasonably accurate in estimating the relative risks of different patients, even experienced clinicians perform poorly when estimating any one individual's absolute risk.3 Methods for estimating the expected event rate for a particular patient have recently been reviewed and will not be considered further in this commentary.1 The second common assumption in extrapolating from trials or meta-analyses to individual patients (who typically are at different risks from the average patient in these studies), is that the relative effects of therapy are similar for patients at different risks. The study by Furukawa and colleagues2 in this issue of the International Journal of Epidemiology adds to the emerging evidence supporting the validity of this assumption.
In discussing underlying or baseline risks in this commentary, I am not referring simply to the control event rate, but rather to patient characteristics (such as age, gender, disease aetiology, concomitant conditions, or disease status) present at baseline and known to impact prognosis for that particular disease. It is well recognized that any analyses relating treatment effects to control event rates in the same dataset will demonstrate a relation even if none exists (since the control event rate factors into both the expression for baseline risk and the expression for treatment effect).4
The best evidence for deciding whether treatment responsiveness differs across a spectrum of underlying risks would arise from individual patient data meta-analyses where the relative treatment effects in subgroups with widely varying risks can be directly compared. Although some such analyses have been conducted, they are few and far between and we are left to consider the second best approach to address this question: comparison of the relative treatment effects in different trials testing the same intervention (assuming that patients in different trials with the same condition will have different risk profiles). The problem is how to find these groups of related trialsthe easiest solution (employed by Furukawa and colleagues) is to review the reference lists of the rigorously conducted systematic reviews included in the Cochrane Collaboration database. I have described this approach as second best as selection bias may very well distort the findings (since a group of trials reporting widely divergent treatment effects are unlikely to be pooled to provide a single summary measure due to excessive heterogeneity). However, at this time it is the best we have.
Schmid et al.5 provided the first such systematic approach by examining the relationship between effect measures and baseline risk in 115 meta-analyses (using a hierarchical model to account for the functional correlation between observed rates discussed earlier and random error in measurement of the control rate). They found that while the risk difference was significantly related to underlying risk in 31% of cases, in 87% of cases the relative risk (RR) (and in 86% of cases the odds ratio [OR]) did not vary significantly with the control event rate. Fukurawa and colleagues extend this work in a separate dataset by demonstrating high rates of concordance for both the OR and the RR in 1843 comparisons between individual trials and the summary effect measures derived from pooling all other trials in that topic area.2 In particular, they found that the concordance rates were high for both OR and RR even when control event rates differed substantially (up to threefold) between trials. They found a qualitative discrepancy (where an individual trial reported an RR [or OR] in the opposite direction from the summary RR [or OR] for the other trials) in only 4% of cases but could not discern any features of these trials that suggested why such a discrepancy may occur. Thus, while both studies suggest that relative effect measures are constant across the usual spectrum of underlying risks in the vast majority of cases, neither study has advanced our understanding of when this assumption is unlikely to hold.
Sackett has hypothesized that relative treatment effects will be constant over the usual range of underlying risks for risk factor interventions designed to slow the progress of disease, but that they will rise with increasing baseline risk for interventions designed to reverse the consequences of a disease process.6 This latter situation would seem to apply particularly when the intervention has both positive and negative effects on the outcome of interest (for example, surgical procedures to prevent certain outcomes in the long-term usually expose patients to an increased risk of these same outcomes in the immediate peri-operative period). Thus, we would expect the relative risk reduction (RRR) associated with treatments such as angiotensin converting enzyme inhibitors in heart failure, beta-blockers in myocardial infarction, or thiazides for hypertension to be similar in patients with different underlying risksindeed this is exactly what is seen.79 On the other hand, we would expect the RRR associated with interventions such as carotid endarterectomy or coronary artery bypass grafting to be higher in patients at higher riskagain, exactly what is seen.10,11 However, a word of caution: the validity of this hypothesis depends on the outcomes examined and is unlikely to hold for combined endpoints where the risk factor intervention impacts on only one of these endpoints.12 For example, consider the example of cholesterol lowering agents. Although these drugs produce a consistent RRR in coronary events' and cardiac mortality across different risk strata, their RRR for the combined endpoint of all-cause mortality will vary and indeed be greater in high-risk patients (such as those with established coronary disease) in whom a greater proportion of all deaths will be cardiac.13
Nevertheless, on the basis of the studies by Drs Schmid and Furukawa, it now seems reasonable to accept the assumption that relative treatment effects are consistent across the spectrum of underlying risks ... usually.
Acknowledgments
FM is a Population Health Investigator of the Alberta Heritage Foundation for Medical Research. The author thanks D Sackett, W Taylor, R Roberts, B Haynes, PJ Devereaux, A Laupacis, S Walter, S Yusuf, J Sinclair, S Connolly, T Louis, and B Djulbegovic for their active participation and thoughtful insights in an e-mail discussion group on this issue and M Egger for helpful editorial comments.
References
1
McAlister FA, Straus SE, Guyatt GH, Haynes RB, for the Evidence-Based Medicine Working Group. Users' Guides to the Medical Literature. XX. Integrating research evidence with the care of the individual patient. JAMA 2000;283:282936.
2
Furukawa TA, Guyatt GH, Griffith LE. Can we individualize the number needed to treat? An empirical study of summary effect measures in meta-analyses. Int J Epidemiol 2002;31:7276.
3
Grover SA, Lowensteyn I, Esrey KL et al. Do doctors accurately assess coronary risk in their patients? Preliminary results of the coronary health assessment study. Br Med J 1995;310:97578.
4
Sharp SJ, Thompson SG, Altman DG. The relation between treatment benefit and underlying risk in meta-analysis. Br Med J 1996;313:73538.
5 Schmid CH, Lau J, McIntosh MW, Cappelleri JC. An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Statist Med 1998;17:192342.[CrossRef][ISI]
6 Sackett DL. Why RCTs fail but neednt. II. Failure to apply physiological statistics, or the only formula a clinician-trialist is ever likely to need (or understand!). CMAJ 2001;(in press).
7 Flather MD, Yusuf S, Kober L et al. Long-term ACE-inhibitor therapy in patients with heart failure or left-ventricular dysfunction: a systematic overview of data from individual patients. Lancet 2000;355:157581.[CrossRef][ISI][Medline]
8 The Beta-Blocker Pooling Project Research Group. The Beta-Blocker Pooling Project (BBPP): subgroup findings from randomized trials in post infarction patients. Eur Heart J 1988;9:816.[Abstract]
9 Collins R, Peto R, MacMahon S et al. Blood pressure, stroke, and coronary heart disease. Part 2, short-term reductions in blood pressure: overview of randomised drug trials in their epidemiological context. Lancet 1990;335:82738.[ISI][Medline]
10 Rothwell PM. Can overall results of clinical trials be applied to all patients? Lancet 1995;345:161619.[CrossRef][ISI][Medline]
11 Yusuf S, Zucker D, Peduzzi P et al. Effect of coronary artery bypass graft surgery on survival: overview of 10-year results from randomised trials by the Coronary Artery Bypass Graft Surgery Trialists' Collaboration. Lancet 1994;344:56370.[ISI][Medline]
12
Davey Smith G, Egger M. Who benefits from medical interventions? Treating low risk patients can be a high risk strategy. Br Med J 1994;308:7274.
13 Davey Smith G, Song F, Sheldon TA. Cholesterol lowering and mortality: the importance of considering initial level of risk. Br Med J 1993;306:136773.[ISI][Medline]