1 Institute of Clinical Pharmacology, Charité, Humboldt-University, Berlin and Institute of Pharmacoepidemiology and Technology Assessment, Schumanstrasse 20/21, D-10117 Berlin, Germany and 2 Department of Epidemiology and Biostatistics,McGill University, Pharmacoepidemiology Research Unit, Division of Clinical Epidemiology, Royal Victoria Hospital, Montréal, Québec, Canada H3A 1A1
3 To whom correspondence should be addressed. e-mail: samy.suissa{at}clinepi.mcgill.ca
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Key words: acute myocardial infarction/detection bias/healthy user bias/observational studies/Womens Health Initiative study
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In 2002, the Womens Health Initiative (WHI) study, a randomized trial of 16 000 post-menopausal women planned for 8 year follow-up, was halted early because of an increase in the risk of breast cancer observed in the group of women treated with HRT (Rossouw et al., 2002). It was also observed that the risk of CHD was increased with the use of HRT with a rate ratio of 1.29 (95% CI 1.021.63) relative to placebo. Such an increased risk, albeit small, was not noted in most previous observational studies. This discrepancy between several well-conducted observational studies and this large randomized trial merits investigation.
Here we describe and discuss particular methodological aspects of the observational and randomized studies that may explain these important and systematic discrepancies.
![]() |
Methodological issues in the WHI trial |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Detection bias
Detection bias could be related to unblinding of clinic gynaecologists or unblinding of the women themselves. The blinding was prematurely broken for 3444 (40.5%) HRT users and 548 (6.8%) placebo users, mostly to manage persistent vaginal bleeding. It is uncertain whether the study succeeded in maintaining blinding of other clinical staff. Even if the gynaecologists concealed the unblinded womens treatment allocation from other clinical staff, the women themselves might have disclosed it when seeking medical advice. It appears only too possible that a woman with new onset of vaginal bleeding after taking the study medication will have guessed that they were on active hormone treatment. These women may have mentioned their bleeding problems or assumptions about treatment allocation to other clinical study staff, thereby creating a potential for detection bias.
Detection bias is of less concern for hard clinical outcomes that are easily clinically diagnosed, but it could be of concern for softer outcomes that may remain undetected in a substantial number of cases. Several cohort studies have shown that 2244% of incident myocardial infarctions (MIs) remain clinically unrecognized at the time they occur (Sheifer et al., 2001). There could be potential for detection bias of acute MI even in a randomized trial, if unblinding occurred and electrocardiograms (ECGs) were recorded preferably in HRT users.
In the WHI study, the reported percentage of silent MIs of 2.7% is unusually low. (Rossouw et al., 2002). Silent MIs, despite their name, are not generally silent. Approximately half of the affected patients experience non-specific symptoms that at the time may not be recognized to be a consequence of acute MI (Sheifer et al., 2001
). The acute MI will then remain clinically undiagnosed. The patients state of alertness may affect pain perception, and anxieties about a possibly increased risk of acute MI may increase the request for additional ECG investigations when non-specific symptoms are experienced. These additional ECG recordings will then increase the number of acute MIs and decrease the number of otherwise silent MIs. Detection bias could occur if the request for an additional ECG was related to the known post-menopausal hormone use from unblinding.
It is likely that ECGs were collected preferably in HRT users as a result of several events that occurred during the time course of the WHI study. In August 1998, results from the Heart and Estrogen/progestin Replacement Study (HERS) were widely publicized. This secondary prevention study, conducted to demonstrate the cardiovascular benefit of HRT treatment, reported the unexpected finding of a small increase in risk of acute MI in HRT-treated women during the first study year and received widespread media attention (Hulley et al., 1998). In spring 2000 and 2001, all WHI study participants received a warning letter from the investigators that informed them about a slight increase in risk of acute MI and other acute cardiovascular events in hormone users during the first study year. It is conceivable that these publications and letters caused a lot of insecurities among study participants and particularly among those women who knew that they were on HRT treatment due to new onset of vaginal bleeding after study medication. These women may have more frequently requested additional ECGs from their physicians, even in the case of less typical angina pectoris symptoms, and thus have had otherwise clinically unrecognized MIs diagnosed more often.
In the WHI study, ECGs were only collected at baseline and at 3 and 6 years after randomization. Silent MIs, that due to increased alertness in HRT users may have been diagnosed more often as acute MI in the hormone-treated group, may have remained undiagnosed in a substantial number of placebo users, since no ECGs were recorded when the study was halted early. To decrease the potential for detection bias, it would have been necessary to record an ECG in all study participants when the study was stopped. This, to our knowledge, has not been the case. However, even if an ECG had been recorded at this point in time, it would not completely abolish the potential for detection bias, since clinically unrecognized non-Q-wave MI would still be missed.
We assessed the potential impact of this detection bias by assuming that the rate of acute MI detection differed according to the blinding status of exposure. Table I shows three hypothetical stratifications for the overall crude rate ratio of 1.28. Taking the figures of 2244% of unrecognized MI published in the literature, the first calculation assumes that the chance of detecting an acute MI in blinded subjects is only 80% of the chance of detection in unblinded subjects, and hence unblinded subjects are around 1.2 times more likely to be diagnosed than the blinded study subjects. As a result, the rate ratio would be reduced to 1.19. The second one assumes that unblinded subjects are 1.5 times more likely to be diagnosed, resulting in a rate ratio reduced to 1.10. The third stratification assumes that unblinded subjects are 1.8 times more likely to be diagnosed, resulting in a rate ratio of 1.02. Thus, varying degrees of differential detection levels for the diagnosis of an acute MI could reduce the rate ratio to unity.
|
![]() |
Methodological issues in the observational studies |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The risk factor profile of HRT-using women is less consistently beneficial when we consider baseline data from the Nurses Health Study which reported a risk reduction of 40% for acute MI in hormone users (Stampfer et al., 1991
). In this study, current hormone users were in fact more often suffering from hypertension, appeared to have more familial risk factors of CHD, and they had higher serum cholesterol levels than never users. Current HRT users were, on the other hand, less often smokers, had a lower prevalence of diabetes, and a lower body weight. Yet other studies did not confirm a relevant difference in cardiovascular risk factor profile between HRT users and non-users (MacLennan et al., 1998
).
It has also been suggested that women complying with HRT treatment have a more favourable prognosis with respect to acute coronary outcomes, and the term compliance bias has been used to describe this possibility (Petitti, 1994; Barrett-Connor and Grady, 1998
). Patients who were compliant with placebo treatment in both the Coronary Drug Project (1980
) and the Beta-Blocker Heart Attack Trial (Horwitz et al., 1990
) had a risk reduction of acute coronary outcomes of the same magnitude that was observed for HRT treatment in the epidemiological studies (Petitti, 1994
). Compliance with HRT treatment could thus be a surrogate parameter for otherwise not easily measurable factors, such as, for example, a healthier lifestyle, a better diet, more physical exercise, less smoking, better compliance with other protective medications and/or better adherence to other protective types of behaviour. Both healthy user bias and compliance bias are therefore interrelated, although adjustment for coronary risk factors did not take away the beneficial effect observed with adherence to medication (Barrett-Connor and Grady, 1998
).
Choice of the reference group
All observational studies compared users of HRT, ever, current or past, with never users. The issue of whether women who never used HRT are comparable with women who do has been the object of much debate. In particular, it has been observed that women who use HRT are generally more health conscious and have lower cardiac risk factors than those who do not (see previous section). Thus, such confounding factors need to be accounted for, and were in some studies. Nevertheless, if this confounding is real, many unmeasured factors would remain unadjusted for and have been proposed as responsible for some residual bias that exaggerates the benefit.
The reference group of never users may in fact be so different that adjustment may be futile, particularly since many unmeasurable risk factors that might modify the risk of cardiovascular disease could not be accounted for in the analysis. An alternative to this approach, geared to the control of confounding, is to question the choice of the reference group. A possible reference group that may be similar to current users of HRT is past users. These women decided to initiate HRT and to stop for some reason. These reasons include either the cessation of menopausal symptoms, concern about safety from published reports or side effects such as bleeding that were deemed undesirable by the patient. However, the fact that these women initiated HRT may in fact be suggestive of greater homogeneity on the cardiovascular risk profile with the corresponding women who initiate and continued to use HRT. Choosing a reference group that on conceptual grounds appears to be similar also avoids the problem of controlling for factors that may be in the causal pathway between exposure and outcome.
We identified all studies that separated HRT exposure into current and past use, focusing on the primary CHD prevention studies. We found nine such studies (Hernandez-Avila et al., 1990; Rosenberg et al., 1993
; Psaty et al., 1994
; Heckbert et al., 1997
; Sidney et al., 1997
; Kaplan et al., 1998
; Grodstein et al., 2000
; Petitti et al., 2000
; Varas-Lorenzo et al., 2000
) and estimated the rate ratio of current HRT use relative to past use. Table II compares this corrected rate ratio with the conventional rate ratio of current HRT use relative to never use. The weighted average of the rate ratio relative to never use is 0.67 (0.630.73), while the corrected rate ratio relative to past use is 0.82 (0.740.92). Thus, the use of this reference group of past users decreases the rate reduction from 33 to 18%. We also assessed the combined effect of duration of HRT use from these same nine studies. Table III shows the rate ratios by duration of HRT use, in comparison with the findings of the WHI study. Relative to past users, current users of HRT for >5 years have similar rate ratios of 0.79 and 0.78, although the CI from the WHI study is wider. For <1 year of use, the rate ratios are 0.99 and 1.78, with wide overlapping CIs.
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Our methodological considerations suggest that the finding of an increased risk of acute MI for HRT in the WHI study could be due to bias despite the initial randomization. RCTs are usually considered as the gold standard in clinical research where most concerns about bias do not apply. However, with increasing endeavours to reproduce findings from long-term observational research in randomized studies, we may be facing the limits of randomized research. Indeed, we have to ask ourselves whether the advantage of randomization can be maintained over many years and whether study subjects and physicians can be kept blinded. If this is not the case, opportunities for bias have to be considered in the same way as we require it for observational research. In the WHI study, unblinding of physicians and presumably patients occurred in a substantial proportion of study participants, with an important differential between the HRT and placebo groups. Detection bias could therefore be an alternative explanation for the unexpected small increase in risk of acute MI.
On the other hand, it is possible that the protective effect of HRT on acute MI has been overestimated in previous observational studies. Both healthy user bias and compliance bias would tend to exaggerate the protective effect. It is practically impossible to quantify the impact of these difficult to measure lifestyle factors and risk profiles on the risk estimates in the epidemiology studies. If current HRT users differ systematically in their health habits and risk profiles from non-users, it may be more appropriate to compare them with past HRT users instead of non-users, since similar health habits may be assumed in both groups of HRT users. By changing the comparison group in this way in our analyses, the protective effect of HRT use on acute MI became less pronounced. This could indicate that the protective effect of HRT on acute MI may have been overestimated in the observational studies due to a healthy user bias.
The discussion about healthy user and compliance bias becomes more complex when we consider fractures as another outcome: several authors have suggested that the beneficial effects of estrogens on the skeletal system were also exaggerated in the observational studies due to a healthier lifestyle of HRT users (Hochberg, 2000). It is of interest in this respect that the results of the observational studies and the WHI randomized trial were in concordance for fractures: in a meta-analysis of observational studies (Grady et al., 1992
), the risk of hip fractures was summarized as 0.75 for estrogen versus no estrogen use and, in the WHI trial, the risk was 0.66 (95% CI 0.450.98) for estrogenprogestin use versus placebo, respectively. The agreement in the effects of HRT on hip fractures between the observational studies and the WHI study could have two interpretations: first that the impact of healthy lifestyles on the risk estimates is less than assumed, which could then equally hold for other end-points, such as, for example, acute MI; or, alternatively, that other risk factors are of more importance for fractures than the discussed healthy user bias.
Grodstein recently emphasized the remarkable consistency between the WHI trial and the observational studies also for other end-points, such as, for example, colorectal cancer, breast cancer, stroke and pulmonary embolism, suggesting that there is little confounding in studies of the relationship of hormone therapy to these end-points (Grodstein et al., 2003). A possible explanation for the similar results for these other end-points and the discrepancy in the results for the risk of acute coronary events could be related to different risk functions. For acute coronary outcomes, the WHI study shows that the risk increases predominantly on initiation of therapy and decreases with longer hormone therapy, with a rate ratio of 0.78 (NS) for >5 years of hormone use (Table III). For the other end-points, the effects are usually observed later in the course of hormone therapy. Early clinical events, as Grodstein pointed out, might not be captured in cohort studies, since these studies do not usually include new users of hormone therapy. However, a protective effect has also been reported in a considerable number of casecontrol studies where early clinical events will not escape identification. Other factors contributing to the observed discrepancy could be related to clinical differences between the observational studies and the WHI randomized trial. The observational studies mostly included women on unopposed estrogen therapy (due to secular trends in hormone therapy), whereas a combined regimen with medroxyprogesterone acetate (MPA) was administered in the halted arm of the WHI study. Several studies have shown that addition of MPA to estrogen diminishes the elevation in high-density lipoprotein observed for the use of estrogen alone (Writing Group for the PEPI Trial, 1995
). There have been only sparse data on the effect of combined HRT on acute MI. The few data that indicated a protective effect on cardiovascular outcomes also for combined hormone therapy mostly included women on a cyclical combined hormone regimen, i.e. the addition of progestins to estrogens for 1014 days per month, whereas a continuous combined regimen was used in the WHI study (Grodstein et al., 2003
).
Another clinical factor concerns the choice of the estrogenprogestin dose in women of advanced age in the WHI study. Clinicians tend to reduce the dose of estrogen and progestin in elderly women to account for a reduction in hepatic and renal clearance (Genazzani and Gambacciani, 2002). There was no dose reduction undertaken in elderly women in the WHI study. Findings from the Nurses Health Study have suggested that estrogens at a dose of 0.3 mg/day exhibit an equally protective effect on acute coronary events as estrogens at a dose of 0.625 mg/day (Grodstein et al., 2000
). The doses administered in the elderly women in the WHI study may therefore have been too high relative to their dose requirements. As a consequence, we may need more data on the effects of different hormone regimens and lower hormone doses on CHD.
In conclusion, it appears too early to conclude that the opposite effect of post-menopausal hormones on acute coronary outcomes found between the observational studies and the WHI randomized trial is solely a consequence of bias in the observational studies. We are only just beginning to understand that randomized trials might be equally subject to bias, particularly if they are conducted over many years to study long-term outcomes that were believed could only be investigated in an observational setting. In the WHI study, detection bias could be an alternative explanation for the small unexpected risk increase.
![]() |
Acknowledgements |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Alvarez A, Hermenegildo C, Issekutz AC, Esplugues JW and Sanz MJ (2002) Estrogens inhibit angiotensin II-induced leukocyteendothelial cell interactions in vivo via rapid endothelial nitric oxide synthase and cyclooxygenase activation. Circ Res 91,11421150.
Barrett-Connor E and Grady D (1998) Hormone replacement therapy, heart disease, and other considerations. Annu Rev Public Health 19,5572.[CrossRef][ISI][Medline]
Benson K and Hartz AJ (2000) A comparison of observational studies and randomized, controlled trials. N Engl J Med 342,18781886.
Clarkson TB, Anthony MS and Klein KP (1996) Hormone replacement therapy and coronary artery atherosclerosis: the monkey model. Br J Obstet Gynecol 103 Suppl 13,5357.[ISI][Medline]
Concato J, Shah N and Horwitz RI (2000) Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 342,18871892.
Coronary Drug Project (1980) Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med 303,10381041.[Abstract]
Genazzani AR and Gambacciani M (2002) A personal initiative for womens health: to challenge the Womens Health Initiative. Gynecol Endocrinol 16,255257.[ISI][Medline]
Grady D, Rubin SM, Petitti DB, Fox CS, Black D, Etttinger B, Eraster VL and Cummings SR (1992) Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med 117,10161037.[ISI][Medline]
Grodstein F, Stampfer MJ, Colditz GA, Willett WC, Manson JE, Joffe M, Rosner B, Fuchs C, Hankinson SE, Hunter DJ et al. (1997) Postmenopausal hormone therapy and mortality. N Engl J Med 336,17691775.
Grodstein F, Manson JE, Colditz GA, Willett WC, Speizer FE and Stampfer MJ (2000) A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann Intern Med 133,933941.
Grodstein F, Clarkson TB and Manson JE (2003) Understanding the divergent data on postmenopausal hormone therapy. N Engl J Med 348,645650.
Guetta V and Cannon RO, III (1996) Cardiovascular effects of estrogen and lipid-lowering therapies in postmenopausal women. Circulation 93,19281937.
Heckbert SR, Weiss NS, Koepsell TD, Lemaitre RN, Smith NL, Siscovick DS, Lin D and Psaty BM (1997) Duration of estrogen replacement therapy in relation to the risk of incident myocardial infarction in postmenopausal women. Arch Intern Med 157,13301336.[Abstract]
Hernandez-Avila M, Walker AM and Jick H (1990) Use of replacement estrogens and the risk of myocardial infarction. Epidemiology 1,128133.[Medline]
Hochberg M (2000) Preventing fractures in postmenopausal women with osteoporosis. A review of recent controlled trials of antiresorptive agents. Drugs Aging 17,317330.[ISI][Medline]
Horwitz RI, Viscoli CM, Berkman L, Donaldson RM, Horwitz SM, Murray CJ, Ransohoff DF and Sindelar J (1990) Treatment adherence and risk of death after a myocardial infarction. Lancet 336,542545.[ISI][Medline]
Hulley S, Grady D, Bush T, Furberg C, Herrington D, Riggs B and Vittinghoff E (1998) Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. J Am Med Assoc 280,605613.
Kaplan RC, Heckbert SR, Weiss NS, Wahl PW, Smith NL, Newton KM and Psaty BM (1998) Postmenopausal estrogens and risk of myocardial infarction in diabetic women. Diabetes Care 21,11171121.[Abstract]
Karas RH (2002) Animal models of the cardiovascular effects of exogenous hormones. Am J Cardiol 90,22F25F.[CrossRef][ISI][Medline]
MacLennan AH, Wilson DH and Taylor AW (1998) Hormone replacement therapy: prevalence, compliance and the healthy women notion. Climacteric,1,4249.[Medline]
Meade TW and Vickers MR (1999) HRT and cardiovascular disease. J Epidemiol Biostat 4,165190.[Medline]
Petitti DB (1994) Coronary heart disease and estrogen replacement therapy. Can compliance bias explain the results of observational studies? Ann Epidemiol 4,115118.[Medline]
Petitti DB, Sidney S and Quesenberry CP, Jr (2000) Hormone replacement therapy and the risk of myocardial infarction in women with coronary risk factors. Epidemiology 11,603606.[CrossRef][ISI][Medline]
Psaty BM, Heckbert SR, Atkins D, Lemaitre R, Koepsell TD, Wahl PW, Siscovick DS and Wagner EH (1994) The risk of myocardial infarction associated with the combined use of estrogens and progestins in postmenopausal women. Arch Intern Med 154,13331339.[Abstract]
Rosenberg L, Palmer JR and Shapiro S (1993) A casecontrol study of myocardial infarction in relation to use of estrogen supplements. Am J Epidemiol 137,5463.[Abstract]
Ross RK, Paganini-Hill A, Mack TM, Arthur M and Henderson BE (1981) Menopausal oestrogen therapy and protection from death from ischaemic heart disease. Lancet 1,858860.[Medline]
Rossouw JE, Finnegan LP, Harlan WR, Pinn VW, Clifford C and McGowan JA (1995) The evolution of the Womens Health Initiative: perspectives from the NIH. J Am Med Womens Assoc 50,5055.[Medline]
Rossouw JE, Anderson GL, Prentice RL, Lacroix AZ, Kooperberg C, Stefanick ML, Jackson RD, Beresford SA, Howard BV, Johnsonl KC et al. (2002) Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Womens Health Initiative randomized controlled trial. J Am Med Assoc 288,321333.
Sanada M, Higashi Y, Nakagawa K, Tsuda M, Kodama J, Kimura M, Chayama K and Ohama K (2002) Hormone replacement effects on endothelial function measured in the forearm resistance artery in normocholesterolemic and hypercholesterolemic postmenopausal women. J Clin Endocrinol Metab 87,46344641.
Sheifer SE, Manolio TA and Gersh BJ (2001) Unrecognized myocardial infarction. Ann Intern Med 135,801811.
Sidney S, Petitti DB and Quesenberry CP, Jr (1997) Myocardial infarction and the use of estrogen and estrogenprogestogen in postmenopausal women. Ann Intern Med 127,501508.
Smith SC, Jr, Blair SN, Criqui MH, Fletcher GF, Fuster V, Gersh BJ, Gotto AM, Gould KL, Greenland P and Grundy SM (1995) Preventing heart attack and death in patients with coronary disease. Circulation 92,24.[Medline]
Stampfer MJ, Colditz GA, Willett WC, Manson JE, Rosner B, Speizer FE and Hennekens CH (1991) Postmenopausal estrogen therapy and cardiovascular disease. Ten-year follow-up from the Nurses Health Study. N Engl J Med 325,756762.[Abstract]
Stork S, von Schacky C and Angerer P (2002) The effect of 17beta-estradiol on endothelial and inflammatory markers in postmenopausal women: a randomized, controlled trial. Atherosclerosis 165,301307.[CrossRef][ISI][Medline]
Tikkanen MJ (1993) Mechanisms of cardiovascular protection by postmenopausal hormone replacement therapy. Cardiovasc Risk Factors 3,138143.
Varas-Lorenzo C, Garcia-Rodriguez LA, Perez-Gutthann S and Duque-Oliart A (2000) Hormone replacement therapy and incidence of acute myocardial infarction. A population-based nested casecontrol study. Circulation 101,25722578.
Wassertheil-Smoller S, Anderson G, Psaty BM, Black HR, Manson J, Wong N, Francis J, Grimm R, Kotchen T, Langer R et al. (2000) Hypertension and its treatment in postmenopausal women: baseline data from the Womens Health Initiative. Hypertension 36,780789.
Wolf PH, Madans JH, Finucane FF, Higgins M and Kleinman JC (1991) Reduction of cardiovascular disease-related mortality among postmenopausal women who use hormones: evidence from a national cohort. Am J Obstet Gynecol 164,489494.[ISI][Medline]
Writing Group for the PEPI Trial (1995) Effects of estrogen or estrogen/progestin regimens on heart disease risk factors in postmenopausal women. The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial. J Am Med Assoc 273,199208.[Abstract]