Commentary: Meta-analysis of Individual Participants Data in Genetic Epidemiology
John P. A. Ioannidis1,2,
Philip S. Rosenberg3,
James J. Goedert4 and
Thomas R. OBrien4
1 Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, and Ioannina Biomedical Research Institute, Foundation for Research and TechnologyHellas, Ioannina, Greece.
2 Department of Medicine, Tufts University School of Medicine, Boston, MA.
3 Biostatistics Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, MD.
4 Viral Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, MD.
Received for publication October 9, 2001; accepted for publication March 14, 2002.
bias, epidemiology; genetics; meta-analysis
Abbreviations:
Abbreviations: AIDS, acquired immunodeficiency syndrome; CCR2, C-C chemokine receptor 2; CCR5, C-C chemokine receptor 5; CDC, Centers for Disease Control and Prevention; D32, 32-base pair deletion; HIV-1, human immunodeficiency virus type 1; MIPD, meta-analysis of individual participants data; MPL, meta-analysis of the published literature; SDF-1, stromal cell-derived factor 1.
 |
INTRODUCTION
|
---|
With more than 30,000 human genes and several genetic markers per gene, the potential for identifying genetic associations for various diseases is enormous (15). Genetic effects are often modest: Many subjects must be studied, and single epidemiologic studies are unlikely to be definitive. Meta-analysis may help meet the challenge of synthesizing data from studies of genetic epidemiology (6). Several meta-analyses of published literature (MPL) have already appeared in the field (7). Meta-analysis of epidemiologic studies (8) is controversial. Critics have focused on variability in study designs, poor data quality, insufficient confounder adjustment, publication bias, and spuriously narrow confidence intervals (912). Use of the raw information from individual subjects instead of published data avoids some of these problems. Meta-analysis of individual patient data has been applied successfully in randomized trials (13, 14). The equivalent design in observational epidemiology (15) may be called meta-analysis of individual participants data (MIPD), since observational studies may also include healthy subjects.
In 1997, we conducted an MPL of host genetic effects in human immunodeficiency virus type 1 (HIV-1) disease progression. This topic was becoming prominent in the biomedical literature (1618), and varying conclusions of investigators generated extensive controversy (1922). Considerable bench research was being conducted to evaluate these epidemiologic associations (23, 24). We followed the MPL with an MIPD on the same topic. This project became one of the first applications of MIPD in human genetics (25) and, to our knowledge, the first involving time-to-event data. Our work provides an opportunity to evaluate the potential advantages and difficulties of conducting a large-scale MIPD in genetic epidemiology. Here we present some insights gained in the process.
 |
CONDUCT OF THE MPL
|
---|
The MPL examined the effect of a 32-base pair deletion (
32) in the C-C chemokine receptor 5 gene (CCR5) and the 64I allele in the C-C chemokine receptor 2b gene (CCR2 641), on the progression of HIV-1-infected persons to acquired immune deficiency syndrome (AIDS). These chemokine receptors are coreceptors for HIV-1 cell entry. We used published data from 10 cohorts that included 3,034 subjects with known CCR5 genotype and 2,383 subjects with known CCR2 genotype. Published data were identified from MEDLINE, AIDSLINE, and abstracts from major meetings. The MPL was reported in Nature Medicine in May 1998 (26).
 |
CONDUCT OF THE MIPD
|
---|
For the MIPD, we contacted all of the primary investigators who had published or presented data (as per the MPL search) and asked them to participate and to refer other groups who might be working in this field. An open invitation for the MIPD was also included in the MPL publication (26). A provisional protocol was distributed, commented upon, and finalized with suggestions received from coinvestigators. Data cleaning, standardization of databases, and clarification of data queries proceeded in parallel with data collection. Statistical programming was developed and refined to allow automated performance of calculations and graphic display of results. While the analyses of the CCR5 and CCR2 alleles were being completed, we decided to extend the meta-analysis to include the stromal cell-derived factor 1 (SDF-1) 3'A allele for which inconsistent published data had accrued in the interim. SDF-1 is the natural ligand for CXCR4, a major HIV-1 coreceptor. The extension of the meta-analysis to include SDF-1 3'A was justified, since this allele was debated as being potentially even more important for HIV disease progression than either CCR5
32 or CCR2 64I. Interim analyses and manuscript drafts were shared with coinvestigators for comments and revisions.
 |
ADVANTAGES OF MIPD (table 1)
|
---|
Data
Completeness of information
Some investigators may publish limited data, may take a long time to publish, or may not publish at all. Selective publication of "positive" studies causes publication bias (27, 28). An advantage of MIPD is that extended databases from published studies (e.g., genotype data for additional alleles or subjects) and data from unpublished studies can be included. Our MIPD included additional data from two cohorts that had originally published information on a limited number of patients and unpublished data from three cohorts.
Standardization of information
In the MIPD, we were able to use standardized data with a priori definitions applied consistently across participating studies, whereas the MPL was limited by inconsistent treatment of key variables across the original publications. Standardization is important for categorizing participants and for defining outcomes and the eligible follow-up period.
The MIPD used consistent definitions to categorize eligible participants as seroconverters or seroprevalent subjects. Seroconverters enter a study before HIV-1 infection, while seroprevalent subjects are already infected with HIV-1 at entry into the study. Patients with rapid disease progression are less likely to survive long enough for recruitment in a seroprevalent cohort. Besides affecting generalizability, this bias influences the magnitude of the observed effect of genetic markers that affect disease progression differentially among rapid and slow progressors.
The MIPD focused on three clinical outcomes: progression to AIDS as defined by the Centers for Disease Control and Prevention (CDC) in 1987, progression to death, and progression to death after AIDS had developed. Although many of the published reports had used the 1987 CDC definition, other AIDS definitions were sometimes used, including the 1993 CDC definition, the 1993 European definition, and specific decreases in CD4+ lymphocyte counts. For some published reports, the definition of AIDS was unclear. For mortality, deaths unrelated to AIDS were included in some published reports, but not in others.
For the synthesis of results of prospective studies, a consistent definition of the follow-up period is highly desirable. In studies of natural history, the use of effective antiretroviral therapy may substantially change disease progression (29). Earlier, more aggressive therapy may be given to patients with signs of worse prognosis. Furthermore, specific alleles may modify the therapeutic effect (30). Studies included in the MPL handled the follow-up period inconsistently. In the MIPD, we censored follow-up on January 1, 1996, which represents the earliest date of general availability of effective treatment for HIV-1 infection.
Analysis
Time-to-event analyses
An MIPD permits greater flexibility in the statistical analysis than an MPL. Synthesis of time-to-event information is problematic without access to data on individual participants (figure 1). The MIPD applied Cox models (31) separately to each study and synthesized study-specific hazard ratios with fixed and random effects general variance models using the Q statistic for heterogeneity (32, 33). MIPDs of randomized trials have typically used an extension of the nonparametric Mantel-Haenszel-Peto method (34). This method is biased when the allocation ratio in the compared groups deviates substantially from one (35), a frequent situation in genetic epidemiology. For example, ratios were 5.3:1, 4.9:1, and 18.5:1 for CCR5
32, CCR2 64I, and SDF-1'A/A, respectively, in subjects of European descent. In the MIPD, proportionality of hazards could be tested in each study and in subgroups of interest. Moreover, one could use methods that do not require proportional hazards, for example, splines (36) and Poisson regression, that would be impossible in MPL.

View larger version (33K):
[in this window]
[in a new window]
|
FIGURE 1. Kaplan-Meier plots for progression to acquired immunodeficiency syndrome (AIDS) (according to the 1987 Centers for Disease Control and Prevention definition) for human immunodeficiency virus type 1-infected seroconverters with and without the C-C chemokine receptor 5 32-base pair deletion (CCR5 32) allele in nine different cohorts. Only wild-type C-C chemokine receptor 2 participants are considered in the analysis. Also shown are the relative hazards (RH) for the analysis of each cohort by Cox proportional hazards models along with 95 percent confidence intervals in parentheses. Although carriers of CCR5 32 (CCR5 +/ 32) tend to progress to AIDS more slowly than do noncarriers (CCR5 +/+) in most cohorts, the difference is formally significant only in one cohort. Furthermore, in that cohort, the database included in the meta-analysis is substantially larger than the database included in the original publication of its results. Using random effects calculations, the summary relative hazards showed a statistically significant 24 percent reduction in the risk of AIDS (25). ACHM, Amsterdam Cohort of Homosexual Men; AIVD, Amsterdam Cohort of Intravenous Drug Users; DCG, DC Gay Cohort; MACS, Multicenter AIDS Cohort Study; MHCS, Multicenter Hemophilia Cohort Study; SEROCO, French Seroconverter Cohort; SFCC, San Francisco City Cohort; SFMHS, San Francisco Mens Health Study; SHCS, Swiss HIV Cohort Study.
|
|
Adjusted/multivariate analyses
A common problem in synthesizing published data is adjustment for different covariates, especially when strong confounders are defined or accounted for inconsistently across studies. Another issue is how to adjust consistently for parameters that may be intermediates in the pathway between the genetic marker and the clinical outcome. For example, the MIPD also collected information on CD4+ lymphocyte count and HIV-1 RNA levels, two strong predictors of disease progression whose values vary substantially depending upon when they are measured in the disease course. The MIPD used consistent definitions and time windows for these measurements.
Linkage disequilibrium
Linkage disequilibrium is the nonrandom association of alleles that lie close together on a chromosome. Loci in linkage disequilibrium pose analytic challenges in studies of genetic epidemiology. For example, the CCR5
32 allele is in strong negative linkage disequilibrium with the CCR2 64I allele. A comparison of CCR2 64I carriers against noncarriers may be biased because participants with the CCR5
32 allele preferentially cluster in the latter group. Bias could be substantial if CCR5
32 indeed confers a better prognosis. Most original studies published analyses of the effects of CCR2 64I without accounting for CCR5
32. In the MIPD, we were able to evaluate the effects of CCR2 64I among participants without CCR5
32.
Alternative genetic models and effects of multiple genes
In modeling genetic effects, heterozygotes may be collapsed with either group of homozygotes (to test either a dominant or recessive model), or genotypes may be treated as three separate categories (to test a codominant model). For rare alleles, the least common group of homozygotes may defy meaningful analysis even when the total number of subjects in a single study is large. Furthermore, published studies may not report data on all genotypes. Therefore, an MIPD may be needed to fully test alternative genetic models.
Although inherited diseases resulting from a single genetic change have been the focus of most genetic studies to date, most genetic effects probably result from interactions between multiple genes (37). The concurrent evaluation of multiple genes may require a larger sample size than may be easily accrued in a single study. Analyses of single studies for such genetic effects may suffer from type II error, while published significant results may represent type I errors. The increased statistical power and analytic flexibility of an MIPD may enhance the ability to evaluate alternative genetic models and multiple genes. For example, the MIPD included 86 participants with both CCR5
32 and CCR2 64I and 50 CCR2 64I homozygotes (representing 1.7 and 1.0 percent of the total, respectively), and the analysis showed that the genetic effects were consistent with a dose-response relation.
Population subgroups
Confounding by race or ethnicity in studies of genetic epidemiology (termed "population stratification" by geneticists) has received considerable attention (3840). The frequency of certain alleles or the magnitude of their genetic effect may differ by race. For example, we evaluated CCR5
32 specifically in participants of European descent only, since this allele is almost totally restricted to such participants. Similarly, the effects of CCR2 64I and SDF-1 3'A were evaluated among participants of European and African descent, since the allele frequencies differ between these racial groups. In contrast to an original study (41), we found no evidence of effect modification by race. Given the variability in racial categorization and inconsistent reporting of racial subgroups across studies, reliable adjustment for population stratification may be difficult in an MPL.
Interpretation
Heterogeneity
Observed heterogeneity between various studies may result from bias, chance, or genuine diversity (42). An MIPD removes much of the bias component by eliminating discrepancies in data definitions and analytic approaches, as described above, and reduces the opportunity for chance findings by increasing statistical precision. In addition, an MIPD may permit the investigation of sources of heterogeneity at the individual level within patient populations of specific studies.
Assessing sampling bias
Not all participants in a cohort may be genotyped for specific alleles, and this may create sampling bias. For two large cohorts, we identified evidence of such selection biases acting in opposite directions (favoring selection of rapid or slow progressors). Such information reflects the generalizability of single studies as well as the entire MIPD.
Other
Establishing collaborations
Large genetic epidemiology studies may require a collaborative, international approach. The MIPD provides a mechanism for all investigators working in a common field to contribute and exchange information. New alleles and disease associations may be proposed and targeted by ongoing MIPDs. For example, recently, several other alleles have been proposed to affect HIV-1 disease progression (4346). Field experts may decide early on that no single study has sufficient statistical power to answer the questions; thus, they form a consortium to address the question. Alternatively, researchers may act independently at the onset and then collaborate in an MIPD to achieve a consensus. Theoretically, an international meta-analysis may be used to decide whether genotyping should be performed in additional studies. If updated on a regular basis as new data appear (a cumulative meta-analysis approach (47, 48)), an MIPD may determine when there is enough evidence on the effect or lack thereof of a specific allele to establish consensus.
 |
DISADVANTAGES OF MIPD (table 1)
|
---|
Data
While an MIPD may retrieve unpublished data, not all published data may be made available for synthesis. One cohort with published data withdrew due to disagreement on the analysis. However, we were able to compare the results from that study with those of the meta-analysis and conclude that the exclusion was unlikely to have altered our conclusions.
Interpretation
Consensus about the mode of analysis, the results, and their interpretation must be attained through all stages of the MIPD. This may not be simple in a field where controversy exists. A serious potential problem with studies that involve active collaboration and pooled results is that investigators may withdraw at any time. Even if the withdrawal does not change the overall results, reanalysis of the remaining data may delay publication and increase the cost of the MIPD.
Resources
The time and effort required to perform an MIPD must be seriously considered. Our MIPD used a professional data-management company contracted by the National Cancer Institute. A total of 2,088 hours of data management were used in the MIPD (as of May 2001). The four coordinating investigators each invested between 5 and 20 percent of their full-time effort during the project. More than 1,000 e-mails were exchanged between the coordinating investigators and with the data managers. International coordination was also demanding and included several general mailings and extensive telephone, fax, and electronic mail communications. MPL and MIPD are compared in scope in table 2. While more conclusive answers were obtained with the MIPD, extensive resources were required.
View this table:
[in this window]
[in a new window]
|
TABLE 2. Comparison of the scope of meta-analysis of published literature and meta-analysis of data from individual participants
|
|
 |
FINAL COMMENT
|
---|
Many of the discussed advantages and disadvantages of the MIPD would probably be found in a comparison of MPL and MIPD for nearly any research question. An MIPD benefits from more complete and better standardized information, greater analytic flexibility, and less chance of bias. For studies of genetic epidemiology specifically, advantages of the MIPD include consistent treatment of loci in linkage disequilibrium, better ability to evaluate the effects of multiple genes or different genetic models, and better evaluation of population stratification. Despite the advantages, an MIPD is not indicated in all cases. Given the large number of putative genetic associations, the time and resources for an MIPD cannot be justified routinely. MIPD has several advantages when time-to-event analyses are involved and when the effects under study are easily affected by inconsistencies in definitions and analytic approach across studies. The advantages are uncertain when postulated effects are large and obvious and when definitions and analyses are highly consistent across studies. In selected situations, an MIPD may target additional questions that individual studies or an MPL cannot fully address. An MIPD is a costly and time-consuming enterprise and should not be undertaken lightly. Additional empiric evidence on this method may give us a more complete understanding of its optimal role in epidemiologic studies of human genetics.
View this table:
[in this window]
[in a new window]
|
TABLE 1. Advantages and disadvantages of meta-analysis of individual participants data compared with meta-analysis of published literature in genetic epidemiology.
|
|
 |
NOTES
|
---|
Correspondence to Dr. John P. A. Ioannidis, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece (e-mail: jioannid{at}cc.uoi.gr). 
 |
REFERENCES
|
---|
- International Human Genome Sequencing Consortium. Initial sequencing and analysis of the human genome. Nature 2001;409:860921.[ISI][Medline]
- The International SNP Map Working Group. A map of human genome variation containing 1.42 million single nucleotide polymorphisms. Nature 2001;409:92833.[ISI][Medline]
- Risch N, Merikangas K. The future of genetic studies of complex human diseases. Science 1996;273:151617.
- Lander ES. The new genomics: global views of biology. Science 1996;274:5369.
- Gray IC, Campbell DA, Spurr NK. Single nucleotide polymorphisms as tools in human genetics. Hum Mol Genet 2000;9:24038.[Abstract/Free Full Text]
- Khoury MJ, Little J. Human genome epidemiologic reviews: the beginning of something HuGE. Am J Epidemiol 2000;151:23.[ISI][Medline]
- Ioannidis JPA, Ntzani E, Trikalinos TA, et al. Replication validity of genetic association studies. Nat Genet 2001;29:3069.[ISI][Medline]
- Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA 2000;283:200812.[Abstract/Free Full Text]
- Shapiro S. Meta-analysis/shmeta-analysis. Am J Epidemiol 1994;140:7718.[ISI][Medline]
- MacArthur C, Foran PJ, Bailar JC 3rd. Qualitative assessment of studies included in a meta-analysis: DES and the risk of pregnancy loss. J Clin Epidemiol 1995;48:73947.[ISI][Medline]
- Petitti DB. Of babies and bathwater. Am J Epidemiol 1994;140:77982.[ISI][Medline]
- Blettner M, Sauerbrei W, Schlehofer B, et al. Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 1999;28:19.[Abstract]
- Stewart LA, Parmar MKB. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet 1993;341:41822.[ISI][Medline]
- Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med 1995;14:205779.[ISI][Medline]
- Steinberg KK, Smith SJ, Stroup DF, et al. Comparison of effect estimates from a meta-analysis of summary data from published studies and from a meta-analysis using individual patient data for ovarian cancer studies. Am J Epidemiol 1997;145:91725.[Abstract]
- Liu R, Paxton WA, Choe S, et al. Homozygous defect in HIV-1 coreceptor accounts for resistance to some multiply-exposed individuals to HIV-1 infection. Cell 1996;86:36777.[ISI][Medline]
- Samson M, Libert F, Doranz BJ, et al. Resistance to HIV-1 infection in Caucasian individuals bearing mutant alleles of the CCR5 chemokine receptor gene. Nature 1996;382:7225.[ISI][Medline]
- Dean M, Carrington M, Winkler C, et al. Genetic restriction of HIV-1 infection by a deletion allele of the CKR5 structural gene. Science 1996;273:185662.[Abstract/Free Full Text]
- Smith MW, Dean M, Carrington M, et al. Contrasting genetic influence of CCR2 and CCR5 variants on HIV-1 infection and disease progression: Hemophilia Growth and Development Study (HGDS), Multicenter AIDS Cohort Study (MACS), Multicenter Hemophilia Cohort Study (MHCS), San Francisco City Cohort (SFCC) Study, ALIVE Study. Science 1997;277:95965.[Abstract/Free Full Text]
- Michael NL, Louie LG, Rohrbaugh AL, et al. The role of CCR5 and CCR2 polymorphisms in HIV-1 transmission and disease progression. Nat Med 1997;3:11602.[ISI][Medline]
- Wrong kind of chemokine research. (Editorial). Nat Med 1997;3:1051.[Medline]
- . Mummidi S, Ahuja SS, Gonzalez E, et al. Genealogy of the CCR5 locus and chemokine system gene variants associated with altered rate of HIV-1 disease progression. Nat Med 1998;4:78693.[ISI][Medline]
- Lee B, Doranz BJ, Rana S, et al. Influence of the CCR2-V64I polymorphism on human immunodeficiency virus type 1 coreceptor activity and on chemokine receptor function of CCR2b, CCR3, CCR5, and CXCR4. J Virol 1998;72:74508.[Abstract/Free Full Text]
- Mariani R, Wong S, Mulder LC, et al. CCR2-64I polymorphism is not associated with altered CCR5 expression or coreceptor function. J Virol 1999;73:24509.[Abstract/Free Full Text]
- Ioannidis JP, Rosenberg PS, Goedert JJ, et al. Effects of CCR5 D32, CCR2 64I, and SDF-1 3'A polymorphisms of HIV-1 disease progression: an international meta-analysis of individual patient data. Ann Intern Med 2001;135:78295.[Abstract/Free Full Text]
- Ioannidis JPA, OBrien TR, Rosenberg PS, et al. Genetic effects on HIV disease progression. (Letter). Nat Med 1998;4:536.
- Easterbrook P, Berlin JA, Gopalan R, et al. Publication bias in clinical research. Lancet 1991;337:86772.[ISI][Medline]
- Ioannidis JPA. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998;279:2816.[Abstract/Free Full Text]
- Palella FJ, Delaney KM, Moorman AC, et al. Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection. N Engl J Med 1998;338:85360.[Abstract/Free Full Text]
- OBrien TR, McDermott DH, Ioannidis JPA, et al. Effect of chemokine receptor gene polymorphisms on the response to potent antiretroviral therapy. AIDS 2000;14:8216.
- Cox DR. Regression models and life tables. J R Stat Soc B 1972;34:187220.[ISI]
- Petitti D. Meta-analysis, decision analysis, and cost-effectiveness analysis: methods for quantitative synthesis in medicine. 2nd ed. New York, NY: Oxford University Press, 1999.
- Trikalinos TA, Ioannidis JP. Predictive modeling and heterogeneity of baseline risk in meta-analysis of individual patient data. J Clin Epidemiol 2001;54:24552.[ISI][Medline]
- Early Breast Cancer Trialists Collaborative Group. Effect of radiotherapy and surgery in early breast cancer: an overview of the randomized trials. N Engl J Med 1995;333:144455.[Abstract/Free Full Text]
- . Greenland S, Salvan A. Bias in the one step method for pooling study results. Stat Med 1990;9:24752.[ISI][Medline]
- Rosenberg PS. Hazard function estimation using B-splines. Biometrics 1995;51:87487.[ISI][Medline]
- Dean M. Strategies for gene discovery. In: OBrien TR, ed. Chemokine receptors and AIDS. New York, NY: Marcel Dekker (in press).
- Little J, Bradley L, Bray MS, et al. Reporting, appraising, and integrating data on genotype prevalence and gene-disease associations. Am J Epidemiol (in press).
- Knowler WC, Williams RC, Pettitt DJ, et al. Gm3;5,13,14 and type 2 diabetes mellitus: an association in American Indians with genetic admixture. Am J Hum Genet 1988;43:5206.[ISI][Medline]
- Khoury M, Beaty TH, Cohen BL. Fundamentals of genetic epidemiology. New York, NY: Oxford University Press, 1993.
- Gonzalez E, Bamshad M, Sato N, et al. Race-specific HIV-1 disease-modifying effects associated with CCR5 haplotypes. Proc Natl Acad Sci U S A 1999;96:120049.[Abstract/Free Full Text]
42. Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet 1998;351:1237.[ISI][Medline]
- Martin MP, Dean M, Smith MW, et al. Genetic acceleration of AIDS progression by a promoter variant of CCR5. Science 1998;282:190711.[Abstract/Free Full Text]
- Alexander L, Weiskopf E, Greenough TC, et al. Unusual polymorphisms in human immunodeficiency virus type 1 associated with nonprogressive infection. J Virol 2000;74:436176.[Abstract/Free Full Text]
- McDermott DH, Beecroft MJ, Kleeberger CA, et al. Chemokine RANTES promoter polymorphism affects risk of both HIV infection and disease progression in the Multicenter AIDS Cohort Study. AIDS 2000;14:26718.[ISI][Medline]
- Faure S, Meyer L, Costagliola D, et al. Rapid progression to AIDS in HIV+ individuals with a structural variant of the chemokine receptor CX3CR1. Science 2000;287:22747.[Abstract/Free Full Text]
- Lau J, Antman EM, Jimenez-Silva J, et al. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 1992;327:24854.[Abstract]
- Ioannidis JPA, Lau J. Evolution of treatment effects over time: empirical evidence from recursive cumulative meta-analyses. Proc Natl Acad Sci U S A 2001;98:8316.[Abstract/Free Full Text]