Department of Social Medicine, University of Bristol, UK. E-mail: d.a.lawlor{at}bristol.ac.uk
Under its definition for the word hindsight the Oxford English Dictionary includes the following statement hindsight is always better than foresight (http://dictionary.oed.com/), and the slogan of a private survey and evaluation company, ingeniously called Hindsight, is remember hindsight is always 20/20! (http://www.hndsight.com/). We have the benefit of the hindsight from randomized controlled trials (RCT) when we comment on this meta-analysis of observational studies, but whether the conflicting results between the trial and observational evidence on the association between hormone replacement therapy (HRT) use and coronary heart disease (CHD) will lead to 20/20 vision remains to be seen.
The disparity between findings from observational studies and RCT of the effects of HRT on CHD,14 has created considerable debate among researchers, practitioners and postmenopausal women. The authors of the meta-analysis reprinted in this issue of the International Journal of Epidemiology concluded that the pooled estimate of effect from the best quality observational studies (internally controlled prospective and angiographic studies) inferred a relative reduction of 50% with ever use of HRT and stated that overall, the bulk of the evidence strongly supports a protective effect of estrogens that is unlikely to be explained by confounding factors.4 By contrast, recent randomized trials among both women with established CHD and healthy women have found HRT to be associated with slightly increased risk of CHD or null effects.1,2 For example, the large Women's Health Initiative (WHI) randomized trial found that the hazards ratio for CHD associated with being allocated to combined HRT was 1.29 (95% CI: 1.02, 1.63), after 5.2 years of follow-up.1
These marked differences between observational findings and trials are important for two reasons. First, and foremost, is the clinical impact. As another commentator on the same subject remarked:
Does HRT decrease or increase the risk of heart disease? At least every woman, every gynecologist and every primary care doctor want to know the "correct" answer.5
Second, is the broader implication for observational epidemiology. Prior to the publication of the WHI it was suggested that well conducted observational studies produced similar estimates of treatment effects as RCT, and that the notion of a hierarchy of evidence with the RCT on top could not be supported.6,7 The differing results between observational studies and RCT in the association between HRT and CHD throw this idea into question and may signify the death of observational epidemiology.8 It is important, therefore, to determine why the results from the trials and observational studies are so different.
A number of explanations have been suggested for these disparities. Whilst some have suggested that the results of the trials were biased because of contamination, and in the case of the WHI, early termination of the arm assessing the effect of combined HRT, the consistency across a number of trials of a null effect make these explanations unlikely. More plausible explanations are that women who participated in the trials were importantly different from those who participated in the observational studies, or that the observational study results were confounded.
![]() |
Changing goalposts |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In one of their original publications, prior to the trial evidence, investigators from the Nurses' Health Study showed that the protective effect of HRT was stronger among women who were at highest risk.9 In June 1990, the FDA Advisory Committee were considering a request by drug companies to approve a label change that would permit prevention of heart disease to be included as an indication for hormone replacement use. Elizabeth Barrett-Connor told the committee that the label change should not be agreed without trial evidence. However, Meir Stampfer told the committee I believe that the data are quite substantial in showing a protective effect of estrogens for heart disease, and I believe that it is a cause and effect relationship. The label change was approved (http://www.curedisease.com/internal_medicine.pdf, page 1516; last accessed 19 May 2004).
Following publication of the Heart and Estrogen/progestin Replacement Study (HERS) (a secondary prevention trial), in 1998,2 showing a 52% increased risk of CHD in the first year of use, the Nurse's Health Study investigators re-analysed their data examining effects of short-term use of HRT among 2489 women (drawn from the total sample of 121 700) with a prior history of cardiovascular disease and found a higher rate of recurrent events, the opposite of what they had claimed earlier, but now fitting with the trial data.10 Co-incident with publication of the WHI1 a re-analysis of observational data demonstrated an almost identical increased relative risk (1.28) for a total cardiovascular incidence endpoint (CHD and stroke) as was found in the trial.11
These shifting goalposts, however, fail to explain why earlier claims from observational studies were so strongly supportive of a protective effect of HRT.
![]() |
Were trial participants importantly different from women studied in observational studies? |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
One of the commonest reasons for women seeking HRT at the time that most observational studies included in this review were conducted was that they had unpleasant menopausal symptoms; in the UK this continues to be one of the commonest reasons for use of HRT.13,14 A potentially important difference between trial participants and women included in observational studies is that the latter will have included many women with symptomatic menopauses who want treatment whereas the former are, by definition, women who are prepared to take a 50:50 chance of being allocated to HRT. It is plausible therefore that if menopausal symptoms are a good indicator of relative or absolute oestrogen deficiency, and that this is associated with increased CHD risk, HRT may be protective against CHD in such women but not in those who are asymptomatic. However, menopausal symptoms are affected by a wide range of social, psychological and cultural factors and are not necessarily a good reflection of hormonal status. Further, the role of oestrogen as an important determinant of CHD is unclear.15,16
![]() |
Were observational studies confounded? |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
![]() |
Why were the observational and RCT results consistent for other outcomes? |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Although results have been inconsistent, on the whole observational studies have not found an association between HRT with stroke.3 Clearly, if studies are not finding a protective effect with respect to stroke then one cannot begin to argue about the role of residual confounding in explaining this (non-existent) association. However, the fact that observational studies tended to find no consistent effect of HRT on stroke whilst in the same studies a protective effect against CHD is found requires further consideration, given the similarities between these two conditions with respect to risk factor profiles.3,26 Interestingly, in their discussion of the paper by Thompson et al., which used a combined endpoint of myocardial infarction and stroke, Stampfer and Colditz hint at finding this combination problematic and indeed decrease the weight of this study in their pooled estimate because of the inclusion of strokes in the overall outcome. However, they make no comment about why they felt including strokes with CHD might underestimate the overall effect of HRT on their main outcome come of interestCHD.
An important difficulty with respect to epidemiological studies of stroke is the ability to distinguish between stroke sub-types,27 and any differential associations between HRT and stroke sub-types may obscure the real picture. Indeed, in the WHI trial, exactly this differential pattern of increased risk of ischaemic stroke (i.e. a similar association to that found for CHD in this trial, as one would expect from their similar pathophysiology) and decreased risk of haemorrhagic stroke was found,28 suggesting that this is a plausible explanation. Attempts to assess the association of HRT with stroke sub-types in observational studies have yielded inconsistent results between studies for both sub-types.27 Heterogeneity between studies may in part reflect the differing extent of misclassification of stroke sub-type in these studies since routine death certificate classifications tend to be inaccurate and clinical diagnoses will only have reasonable accuracy where there has been widespread use of scans.27 In the WHI trial strokes were classified as ischaemic or haemorrhagic based on brain imaging.
![]() |
Should we call it a day for observational epidemiology? |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Hulley S, Grady D, Bush T et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA 1998;280:60513.
3 Grady D, Rubin SM, Petitti DB et al. Hormone therapy to prevent disease and prolong life in postmenopausal women [see comments]. Ann Intern Med 1992;117:101637.[ISI][Medline]
4 Stampfer MJ, Colditz GA. Estrogen replacement therapy and coronary heart disease: a quantitative assessment of the epidemiologic evidence. Prev Med 1991;20:4763. (Reprinted Int J Epidemiol 2004;33:44553).[ISI][Medline]
5 Michels KB. Hormone replacement therapy in epidemiologic studies and randomized clinical trials- are we checkmate? Epidemiology 2003;14:35.[CrossRef][ISI][Medline]
6 Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. New Engl J Med 2000;342:187886.
7 Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. New Engl J Med 2000;342:188792.
8 Davey Smith G, Ebrahim S. Epidemiologyis it time to call it a day? Int J Epidemiol 2001;30:111.
9 Grodstein F, Stampfer MJ, Colditz GA et al. Postmenopausal hormone therapy and mortality. New Engl J Med 1997;336:176975.
10 Grodstein F, Manson JE, Stampfer MJ. Postmenopausal hormone use and secondary prevention of coronary events in the nurses' health study. a prospective, observational study. Ann Intern Med 2001;135:18.
11 Humphrey LL, Chan BK, Sox HC. Postmenopausal hormone replacement therapy and the primary prevention of cardiovascular disease. Ann Intern Med 2002;137:27384.
12 The WHI Steering Committee and Writing Group. Risks of postmenopausal hormone replacement therapy. JAMA 2002;288:282324.[ISI]
13 Griffiths F, Convery B. Women's use of hormone replacement therapy for relief of menopausal symptoms, for prevention of osteoporosis, and after hysterectomy. Br J Gen Pract 1995;45:35558.[ISI][Medline]
14 Griffiths F, Jones K. The use of hormone replacement therapy; results of a community survey. Fam Pract 1995;12:16365.[Abstract]
15 Barrett-Connor E. Sex differences in coronary heart disease. Why are women so superior? The 1995 Ancel Keys Lecture. Circulation 1997;95:25264.
16 Lawlor DA, Ebrahim S, Davey Smith G. Sex matters: secular and geographic trends in sex differences in ischaemic heart disease mortality. BMJ 2001;323:54145.
17 Barrett-Connor E. Hormone replacement therapy. BMJ 1998;317:45761.
18 Davey Smith G, Ben-Shlomo Y, Lynch J. Life course approaches to inequalities in coronary heart disease risk. In: Stansfeld S, Marmot M (eds). Stress and the Heart. London: BMJ books, 2002, pp. 2049.
19 Grodstein F, Manson JE. Relationship between hormone replacement therapy, socioeconomic status, and coronary heart disease. JAMA 2003;289:4445.
20 Krieger N. Postmenopausal hormone therapy. New Engl J Med 2003;348:236364.
21 Lawlor DA, Davey Smith G, Ebrahim S. The association of socioeconomic position from across the life course with HRT use: an explanation for the discrepancy between observational and RCT evidence on HRT and CHD? Am J Public Health 2004 (In press)
22 Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet 2004: (In press)
23 Barbone F, Filiberti R, Franceschi S et al. Socioeconomic status, migration and the risk of breast cancer in Italy. Int J Epidemiol 1996;25:47987.[Abstract]
24 Claussen B, Davey Smith G, Thelle D. Impact of childhood and adulthood socioeconomic position on cause specific mortality: the Oslo Mortality Study. J Epidemiol Community Health 2003;57:4045.
25 Dedman DJ, Gunnell D, Davey Smith G, Frankel S. Childhood housing conditions and later mortality in the Boyd Orr cohort. J Epidemiol Community Health 2001;55:1015.
26 Farahmand BY, Persson PG, Michaelsson K et al. Socioeconomic status, marital status and hip fracture risk: a population-based case-control study. Osteoporosis Int 2000;11:80308.[CrossRef][ISI][Medline]
27 Lawlor DA, Davey Smith G, Leon DA, Sterne JA, Ebrahim S. Secular trends in mortality by stroke subtype in the 20th century: a retrospective analysis. Lancet 2002;360:181823.[CrossRef][ISI][Medline]
28 Wassertheil-Smoller S, Hendrix SL, Limacher M et al. Effect of estrogen plus progestin on stroke in postmenopausal women: the Women's Health Initiative: a randomized trial. JAMA 2003;289:267384.
29 Paganini-Hill A. Hormone replacement therapy and stroke: risk, protection or no effect? Maturitas 2001;38:24361.[CrossRef][ISI][Medline]
30 Davey Smith G, Ebrahim S. Data dredging, bias, or confounding. BMJ 2002;325:143738.
31 Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol 1996;25:110716.[Abstract]
32 Weiss NS. Can the specificity of an association be rehabilitated as a basis for supporting a causal hypothesis? Epidemiology 2002;13:68.[CrossRef][ISI][Medline]
33 Petitti DB, Perlman JA, Sidney S. Postmenopausal estrogen use and heart disease. New Engl J Med 1986;315:13132.[ISI][Medline]
34 Davey Smith G, Ebrahim S. Mendelian randomization: can genetic epidemiology contribute to understanding environmental determinants of disease? Int J Epidemiol 2003;32:122.[CrossRef][ISI][Medline]
35 Lawlor DA, Davey Smith G, Bruckdorfer KR, Kundu D, Ebrahim S. Those confounded vitamins: what can we learn from the differences between observational versus randomized trial evidence? Lancet 2004 (In press).
36 Davey Smith G, Ebrahim S. Mendelian randomization: prospects, potentials, and limitations. Int J Epidemiol 2004;33:3042.