Modeling Treatment Effects on Binary Outcomes with Grouped-Treatment Variables and Individual Covariates

S. Claiborne Johnston1, Tanya Henneman2, Charles E. McCulloch3 and Mark van der Laan2

1 Department of Neurology, University of California, San Francisco, San Francisco, CA.
2 Division of Biostatistics, School of Public Health, University of California, Berkeley, Berkeley, CA.
3 Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, CA.

Received for publication May 2, 2001; accepted for publication May 24, 2002.


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
During evaluation of treatment effects in observational studies, confounding is a constant threat because it is always possible that patients with a better prognosis, not adequately characterized by measured covariates, are chosen for a specific therapy. Ecologic analyses may avoid confounding that would be present in analysis at the individual level because variations in regional or hospital practice may be unrelated to prognosis. The authors used simulated data with an excluded confounder to evaluate the reliability and limitations of the grouped-treatment approach, a method of incorporating an ecologic measure of treatment assignment into an individual-level multivariable model, similar to the instrumental variable approach. Estimates based on the grouped-treatment approach were closer to the true value than those of standard individual-level multivariable analysis in every simulation. Furthermore, confidence intervals based on the grouped-treatment approach achieved approximately their nominal coverage, whereas those based on individual-level analyses did not. The grouped-treatment approach appears to be more reliable than standard individual-level analysis in situations where the grouped-treatment variable is unassociated with the outcome except via the actual treatment assignment and measured covariates.

confounding factors (epidemiology); ecology; instrumental variable; selection bias; simulation

Abbreviations: Abbreviation: OR, odds ratio.


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
The validity of any observational study is limited by the possibility that confounders are inadequately measured or adjusted. This is a particularly difficult problem for observational studies of treatment effects because the decision to treat is often based on prognostic factors. This form of bias, termed confounding by indication, is a constant threat to validity in observational studies (14).

Ecologic studies may bypass confounding at the individual level. Even when confounding obscures a true underlying benefit of a therapy at the individual level, grouping the unit of analysis can produce more reliable results (57). For example, hospitals that use a more effective treatment more frequently should have better overall outcomes for patients treated there compared with another hospital that does not offer the treatment. Regardless of the pretreatment prognosis of those selected for the more effective therapy, outcomes in this group will be better than if the therapy were not available. Variability in hospital or regional practice provides a form of pseudo randomization.

On the basis of these concepts, we previously compared traditional surgical treatment of intracranial aneurysms with a new, minimally invasive technique in a study involving 70 academic medical centers (8, 9). Recognizing that utilization differences among hospitals were likely primarily due to practice variability rather than differences in patient prognosis (5, 6), we used the proportion of patients treated by the newer technique at a given hospital (the probability of receiving the newer treatment among patients in the analysis treated at the hospital) as the treatment variable and did not include in the model the actual treatment each patient received. Rather than discarding useful information by aggregating all data by hospital, we constructed models at the individual level: They predicted an individual’s probability of in-hospital death. Because demographic variables could influence both the proportion of cases treated by the new technique at a hospital and the outcome of cases, we included these in the model. The final model was constructed like a standard multivariable analysis except that the actual treatment assignment for a patient was substituted by a grouped-treatment variable: the proportion of cases treated by the new technique at the hospital of treatment. More generally, a grouped-treatment variable is defined as the probability of an individual’s receiving the treatment of interest at a specified level of aggregation. It can be calculated by determining the proportion of cases treated by the therapy of interest at any specified unit of aggregation, such as a ward, clinic, hospital, region, state, or country.

For the grouped-treatment approach to produce valid estimates of the treatment effect, several assumptions are required. The most important assumption, which we term the basic instrumental variable assumption, is that unmeasured confounders do not produce an association between pretreatment prognosis and the grouped-treatment variable. In other words, conditional on the value of the actual treatment and the modeled covariates, the grouped-treatment variable and the outcome are statistically independent. This assumption may be difficult to satisfy. For example, hospitals that use the new treatment for intracranial aneurysms may receive higher risk cases by referral or may offer other technologies and expertise that could contribute to outcome differences. Thus, the grouped-treatment approach is also limited by the possibility of unmeasured confounding.

The utility of aggregating treatment assignments in avoiding confounding has been recognized for some time by economists, who have termed the approach the instrumental variable method (10, 11), but it has been used infrequently by epidemiologists and health services researchers (12). Economists and epidemiologists tend to construct multivariable models differently, and economists rarely study dichotomous outcomes (10, 13, 14), which are frequently of interest in health studies. The grouped-treatment approach attempts to preserve a model structure familiar to health researchers. Several authors have described the mathematical basis for the grouped-treatment approach (15), most frequently in the context of illustrating the impact of omitted covariates (1618). However, the potential utility of the method as a means for evaluating treatment effects in the presence of unmeasured confounders has not been the subject of prior reports. Furthermore, the validity of the method has not been illustrated with terms and data familiar to epidemiologists. In this study, we subject the method to testing with simulated data sets in order to test its validity and determine its limitations.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Marginal and conditional estimates
In a multivariable model with a dichotomous outcome, omitting a covariate that is associated with the outcome but not with the treatment assignment does not produce confounding, but it does reduce the apparent effect size of the treatment. Standard multivariable models that include all covariates produce the conditional treatment effect, which is the expected impact of treatment in an individual while holding all other variables constant. If covariates that are associated with outcome are excluded from the multivariable model, the marginal treatment effect is produced—marginal with respect to the excluded covariates. With a logistic model, the magnitude of the marginal treatment effect is smaller than that of the conditional treatment effect because the association between treatment and outcome is weakened when a covariate is missing from the model (19): Less variability in outcome is accounted for by the treatment because more is due to a missing covariate. The logit curves based on averaging the outcome probabilities are closely approximated by a different logistic curve with an attenuated slope (20). However, if a covariate is not available to include in an analysis, it may not be available to consider in making treatment decisions, so the overall impact of treatment in a population would be more accurately reflected by the marginal treatment effect. Randomized trials produce an estimate of the marginal treatment effect for the studied population unless an adjusted analysis is reported. As in a randomized trial, we assumed that all patients could receive either of the treatments being compared and that treatment of one patient did not influence the outcome of another, which can occur when other management strategies or patient selection is affected by outcomes of previously treated patients (21). Also similar to randomized trials, the parameter estimated by the grouped-treatment analysis is expected to be marginal with respect to unmeasured confounders.

Simulations
For each set of conditions tested, 1,000 patient cohorts with N individuals were simulated using S Plus (version 2000; Mathsoft Corporation, Seattle, Washington) software. We modeled a dichotomous treatment Ti, meant to represent a new therapy, received at hospital Hi, where i was unique for each individual 1, 2, 3, ..., N. Individuals were distributed evenly among 50 hospitals, defined by Hj = 1, 2, 3, ..., 50, with varying utilization of the treatment, as defined below. A confounder Ci and a covariate Bi were randomly and independently assigned a value of 0 or 1 to each patient with 50 percent probability. The confounder was related to both outcome and treatment according to the odds ratios (ORs) ORCO and ORCT, respectively. The covariate was not related to treatment but was related to outcome by ORBO. Outcome Oi was binary, representing dead (Oi = 1) or alive (Oi = 0). The new therapy was modeled to be effective with ORTO = 0.75; this small treatment effect was modeled so that confounding would bias results to produce an apparent detrimental effect in some individual-level models.

To model practice variability among hospitals, the probability of receiving treatment was dependent on the hospital. For most simulations, treatment was assigned using the following algorithm:

Logit [probability of treatment] = (0.04)H + [ln(ORCT)]C 0.9, (1)

where ln is the natural logarithm; this algorithm was chosen to ensure a large effect from confounding while allowing a wide range of treatment probabilities across the different institutions. Outcome was assigned to model the independent effects of the covariate B, the confounder C, and the treatment assignment, as follows:

Logit [probability of outcome] = [ln(ORBO)]B + [ln(ORCO)]C + [ln(ORTO)]T + 0.05. (2)

These equations ensure that there is no nonrandom association between the confounder and the grouped-treatment variable except through the actual treatment assignment.

In a previous analysis, we used generalized estimating equations to account for clustering of observations within institutions (9). To simplify these simulations and their interpretations, we assume here that there is no clustering of observations beyond that produced by treatment assignment. Therefore, logistic regression can be used instead of generalized estimating equations.

To test the reliability of the approach with different distributions of treatment probability among the hospitals, we generated additional cohorts: a uniform distribution, modeling a wide distribution of treatment frequency (probability of treatment = (0.01)H + 0.2 x C), and a split distribution, modeling frequent utilization of the treatment at only half the hospitals (probability of treatment determined by the uniform distribution, ranging from 0 percent to 10 percent at half the hospitals and from 35 percent to 60 percent at the other half). These distributions were chosen to reflect the type of variability in clinical practice that may be encountered in epidemiologic and health services research.

A total of 1,000 cohorts, corresponding to replicates, each with a sample size of 5,000, were generated according to a Bernoulli distribution for each set of conditions tested using the probabilities given by equations 1 and 2. An estimate of the treatment effect and its 95 percent confidence interval (by the Wald method) were calculated for each cohort using three different logistic models:

• Model 1 (standard, individual-level analysis with known covariate and confounder):

Logit [probability of outcome] = ß1 T + ß2 B + ß3 C + K,

where ß1 represents the conditional treatment effect when there are no unmeasured confounders. K is a constant.

• Model 2 (standard, individual-level analysis with known covariate and unmeasured confounder):

Logit [probability of outcome] = ß4 T + ß5 B + K,

where ß4 represents a confounded estimate of the conditional treatment effect.

• Model 3 (unmeasured confounder and grouped-treatment variable (grouped-treatment estimator)). A grouped-treatment variable (GT) was determined as the proportion of patients treated by the new therapy at a given hospital:

Logit [probability of outcome] = ß6 GT + ß7 B + K,

where ß6 represents an unconfounded estimate of the treatment effect because there is no association between GT and outcome independent of treatment assignment T. The magnitude of ß6 is an estimate of the marginal treatment effect—ß4 from model 2 with ORCT set to 1—because the condition in which all patients are treated by the new procedure (GT = 1) differs from the condition in which no patient is so treated (GT = 0) by the coefficient ß1. If the grouped-treatment analysis is unbiased, then ß6 should be equal to ß4 from model 2 with ORCT set to 1, which was estimated by simulation and is defined as the "true" value being estimated.

For each of the models, calculated coefficients from the logistic models listed above and the limits of the Wald 95 percent confidence interval were averaged for the 1,000 replicates. These values were converted to odds ratios. The confidence interval for each replicate was evaluated to test inclusion of the true underlying treatment effect: The calculated confidence interval would be expected to include the modeled marginal treatment effect (ß4 from model 2 with ORCT set to 1) in 95 percent of the replicates if it were determined accurately. Therefore, we tallied the proportion of replicates in which the calculated confidence interval included the actual marginal treatment effect, termed the percent coverage, to characterize the behavior of confidence intervals.

A number of variables were varied to determine the limitations of the grouped-treatment method as an approach to estimating the marginal treatment effect. Ranges of these variables were chosen to illustrate a variety of conditions that could occur in clinical practice and to test the method when confounding was severe enough to reverse the direction of the apparent treatment effect.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
For the initial simulations, the grouped-treatment variable—the proportion of cases receiving treatment T at the hospital of treatment—varied between 42 percent and 84 percent. This distribution might reflect practice variability for an established therapy with unclear indications. Because we modeled a small protective effect for treatment (model 1; ORTO = 0.75), the expected variability in outcome by hospital was small (66–68 percent dying).

The accuracy of the grouped-treatment estimator (model 3) was tested over a range of association between the confounder and the treatment probability (ORCT) and compared with the standard, individual-level estimate with the confounder "unknown" and absent from the equation (model 2) (table 1). In these simulations, 5,000 patients and 50 hospitals were modeled, and all other associations were held constant. The underlying conditional treatment effect was small and protective (model 1; ORTO = 0.75). The prognosis was adversely affected by the confounder (ORCO = 2.7) and the covariate (ORBO = 2). Under these conditions, the underlying unconfounded marginal treatment effect is 0.76, the individual-level estimate (model 2) when ORCT = 1, and does not vary in the table. The association between the grouped-treatment variable and the actual treatment assignment was relatively weak in these cohorts (R = 0.26–0.29). The grouped-treatment estimates are generally closer to the unconfounded marginal treatment effect than are the standard individual-level estimates with the confounder unknown. Bias was present at the extremes, which were smaller than in the standard individual-level model and appeared to be in the direction of the underlying confounding. The confidence interval of the grouped-treatment estimates included the underlying marginal treatment effect in approximately 95 percent of replicates, as represented by the percent coverage, which suggests that the reliability of the estimate was accurately represented. This was in sharp contrast to the percent coverages for the individual-level estimator.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Treatment effect estimates varying the association between the confounder and the probability of treatment*
 
With a similar distribution of the grouped-treatment variable, the strength of the association between confounder and outcome (ORCO) was varied (table 2). Again, the population was set at 5,000 patients distributed among 50 hospitals with ORBO = 2 and ORTO = 0.75. Those with the confounder were modeled to receive treatment more frequently (ORCT = 2), and the association between the confounder and the outcome was varied from a strong protective effect (ORCO = 0.05) to a strong detrimental effect (ORCO = 20.1). The unconfounded marginal estimate of treatment effect (model 2 with ORCT = 1) varied for each level of ORCO and is given in the second column of table 2; values include random variation because they were calculated. Compared with the unconfounded marginal estimates, the grouped-treatment approach generally produced less biased estimates than did the standard individual-level model (model 2). Again, the 95 percent confidence interval for the grouped-treatment estimator included the underlying treatment effect in approximately 95 percent of replicates, whereas the confidence interval of the individual-level estimator was unreliable.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Treatment effect estimates varying the association between the confounder and the outcome*
 
To determine the sensitivity of the grouped-treatment approach to sample size, we varied the modeled population size from 500 to 7,000 distributed evenly among 50 hospitals (table 3). All other variables were held constant (ORTO = 0.75, ORBO = 2, ORCO = 2.7, ORCT = 2). Under these conditions, the underlying unconfounded marginal treatment effect (model 2 with ORCT = 1) is 0.76. Because the confounder is associated with greater probability of treatment and is detrimental, incomplete adjustment would be anticipated to produce an odds ratio closer to one. The grouped-treatment approach produced less biased estimates even with only 500 patients or 10 per hospital. Bias was generally reduced with increasing sample size for the grouped-treatment approach but not for the individual-level model. The 95 percent confidence interval for both estimates narrowed with increasing sample size; however, the confidence interval accurately reflected the reliability of the estimate for the grouped-treatment approach but not the standard individual-level approach.


View this table:
[in this window]
[in a new window]
 
TABLE 3. Treatment effect estimates varying the number of observations*
 
We varied the independent covariate broadly (ORBO = 0.05–20.1) to ensure that the grouped-treatment approach produced reliable estimates when a known confounder was included in the model (table 4). Other associations were held constant, as in table 3, with 5,000 observations modeled. The individual-level estimates were biased and systematically skewed toward the direction of confounding. The grouped-treatment estimates accurately represented the marginal treatment effect (OR = 0.76); confidence intervals were broad but well behaved.


View this table:
[in this window]
[in a new window]
 
TABLE 4. Treatment effect estimates varying the association between the independent risk factor and the outcome*
 
To model different forms of practice variability among hospitals, we varied the distribution of treatment probability (table 5). Regardless of the direction of confounding and the form of the distribution of treatment probabilities across hospitals, the grouped-treatment estimates were less biased representations of the marginal treatment effect compared with individual-level estimates. Confidence interval coverage was again fully in accord with the nominal 95 percent.


View this table:
[in this window]
[in a new window]
 
TABLE 5. Treatment effect estimates varying the association between the confounder and the outcome in three representatives of practice variability*
 

    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
These simulation results suggest that, if specific assumptions are satisfied, incorporating grouped-treatment variables into models with individual-level covariates and binary outcomes produces more reliable estimates of the treatment effect than do standard individual-level models when a confounder is not included in the models. In a wide variety of situations, the coefficients produced by the grouped-treatment approach are reasonably unbiased estimates of the treatment effect. The estimated treatment effect is the odds ratio comparing the condition that all patients were treated by the therapy of interest with the condition that none of the population was thus treated after adjustment for known covariates. This would correspond with the expected results for an institution that converted completely from use of one procedure to another. Of course, it may not be possible to treat all patients with both procedures, and these findings would not apply to those who are not candidates for both (22), similar to results of randomized trials. In addition, the relative benefit of the better procedure may not be as great in a subpopulation of patients. This possibility can be evaluated by testing nonlinear forms of the grouped-treatment variable. For example, categorizing the grouped-treatment variable or fitting splines may reveal a plateau in the benefit of a procedure at high rates of use.

In our models, the grouped-treatment variable and the outcome are statistically independent conditional on the value of the actual treatment. This is equivalent to the basic instrumental variable assumption (10, 22). Practically, it corresponds with the assumption that the proportion of cases treated by a given procedure at an institution is not independently associated with the pretreatment prognosis of patients. In other words, high-risk patients are not being referred to certain hospitals because of those hospitals’ utilization of a specific procedure. For example, sicker patients with intracranial aneurysms do not preferentially seek treatment at those hospitals offering a new therapy. This assumption may be difficult to satisfy in practice and may limit the utility of the grouped-treatment approach.

If there are known confounders that are clustered at certain hospitals, such as ethnicity or older age, and these are also associated with outcome, including these variables in the model would be expected to eliminate the confounded association between the grouped-treatment variable and the outcome. For example, hospitals that treat wealthier patients may offer a new therapy more frequently. If wealth is associated with the outcome and there is no measure of wealth available, the grouped-treatment variable would be associated with outcome and the basic instrumental variable assumption would not be fulfilled. However, if an accurate measure for a patient’s wealth were included in the model, the assumption could be fulfilled. We found that adjustment for a covariate in the model with the grouped-treatment variable was possible, at least when the covariate was equally distributed among hospitals. Further analysis is required to confirm that adjustment for an unequally distributed covariate is practical and to demonstrate the impact of clustering of observations by institution due to unmeasured covariates that are not related to treatment assignment. In addition, adjustment for unequally distributed covariates that are measured only at the group level may be practical, but this requires further modeling and assumptions. Because unmeasured group-level covariates may be present, it may be advisable to use generalized estimating equations or random effects models to account for clustering (9).

As long as the basic instrumental variable assumption is fulfilled, the grouped-treatment estimator appears to provide a less biased assessment of the impact of treatment than does an estimator based on a standard individual-level model that does not include the confounder. Therefore, at a minimum the grouped-treatment estimator could provide a test of residual uncontrolled confounding in the standard analysis: A large difference between estimates based on the individual-level treatment variable and based on the grouped-treatment variable might suggest the presence of unmeasured confounding. A specific statistical test is available to compare results from standard and instrumental variable models (23), but a difference that is not statistically significant may still be important from a clinical or policy perspective, so a subjective comparison of the estimates may be more useful. In this respect, the grouped-treatment approach may have value as a sensitivity analysis.

The grouped-treatment approach is likely to provide more accurate estimates of the treatment effect if the correlation between the grouped-treatment variable and the actual treatment assignment is strong relative to the correlation between the grouped-treatment variable and the unmeasured confounder (24). The correlation between the grouped-treatment variable and actual treatment assignment increases with greater practice variability; in this instance, the likelihood of receiving treatment varies greatly from institution to institution, and this variation is independent of unmeasured confounders. As the correlation decreases, the grouped-treatment estimator is expected to become more biased in the direction of the standard, confounded estimator. Although we modeled three distributions of treatment utilization among the hospitals, the correlation between the actual treatment assignment and the grouped-treatment variable varied little (R = 0.19–0.28). The grouped-treatment approach may not perform as well when practice variability is very small.

It is possible to show that the grouped-treatment estimate is unbiased with large samples when there is no underlying treatment effect—equivalent to the null hypothesis in many studies. Therefore, the approach should be reliable for hypothesis testing when the existence, rather than the magnitude, of a treatment effect is of primary interest. Additional theoretical research will be required to define the limits of the method and to develop more generalized models to accommodate nonnormal, continuous outcomes.

The confidence intervals for the grouped-treatment estimate were wider than those for the standard estimate. This was expected because of the loss of precision in estimating treatment by using the imperfect surrogate, the proportion of patients receiving the treatment at a given hospital. The confidence interval of the grouped-treatment estimate included the actual unconfounded marginal estimate in approximately 95 percent of replicates over a range of conditions. This was not the case for the individual-level model with the unmeasured confounder. Therefore, the grouped-treatment approach appears to produce confidence intervals with nominal coverage behavior.

In the example motivating these simulations and in our previous studies, we have defined the grouped-treatment variable as the proportion of cases receiving a certain procedure at the hospital of treatment. Other constructs for grouping the treatment are possible, such as geographic region or relative distance to a hospital with particular characteristics (12), and are valid as long as the grouped-treatment variable is independent of outcome, conditional on known covariates and the actual treatment assignment. Grouping based on the hospital of treatment has certain advantages. First, it is an intuitive unit so that the assumptions can be evaluated more directly. Second, the hospital of treatment is a convenient variable that is frequently available in large data sets. Third, practice variability between hospitals tends to exist whenever there is disagreement about the indications for a given treatment (5, 6), so it is relatively easy to demonstrate correlation between the grouped-treatment variable and actual treatment assignment. Fourth, if there are other hospital characteristics associated with the availability of a given treatment, such as greater hospital treatment volume, these can be evaluated in the models if measurable. For example, in our previous study of intracranial aneurysm treatment, we showed that the newer procedure was associated with lower risk of in-hospital death and that the effect was independent of hospital volume and offering other specialized procedures (8). If other hospital characteristics are not measurable, the results can still be interpreted at the hospital level: If hospitals that use the treatment more frequently have better outcomes, regardless of whether the treatment or other hospital characteristics are responsible, these hospitals may be good choices for referral of patients. However, the possibility that utilization of the treatment of interest may be associated with other group-level characteristics must be carefully considered. Such an association provides another potential source of unmeasured confounding.

Although the grouped-treatment approach produces biased estimates of the underlying marginal treatment effect, this bias appears to be smaller than that in standard models with only individual-level variables. In many instances, the bias in the grouped-treatment estimate appears quite small, and its confidence interval has good properties. Therefore, the grouped-treatment approach may supplement standard multivariable analyses of large data sets when the grouped-treatment variable has no independent association with the outcome because of unmeasured confounders and when the treatments being compared are generally options for the individuals included in the cohort.


    ACKNOWLEDGMENTS
 
The authors thank Drs. Ira Tager, John Colford, and Alan Hubbard for important discussions in formulating the method.


    NOTES
 
Reprint requests to Dr. S. Claiborne Johnston, Department of Neurology, Box 0114, University of California, San Francisco, 505 Parnassus Ave., San Francisco, CA 94143-0114 (email: clay.johnston{at}ucsfmedctr.org). Back


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Byar DP. Why data bases should not replace randomized clinical trials. Biometrics 1980;36:337–42.[ISI][Medline]
  2. Green SB, Byar DP. Using observational data from registries to compare treatments: the fallacy of omnimetrics. Stat Med 1984;3:361–73.[ISI][Medline]
  3. Grobbee DE, Hoes AW. Confounding and indication for treatment in evaluation of drug treatment for hypertension. BMJ 1997;315:1151–4.[Free Full Text]
  4. Miettinen OS. The need for randomization in the study of intended effects. Stat Med 1983;2:267–71.[Medline]
  5. Wen SW, Kramer MS. Uses of ecologic studies in the assessment of intended treatment effects. J Clin Epidemiol 1999;52:7–12.[ISI][Medline]
  6. McPherson K. The Cochrane Lecture. The best and the enemy of the good: randomised controlled trials, uncertainty, and assessing the role of patient choice in medical decision making. J Epidemiol Community Health 1994;48:6–15.[Abstract]
  7. Detels R, Munoz A, McFarlane G, et al. Effectiveness of potent antiretroviral therapy on time to AIDS and death in men with known HIV infection duration. Multicenter AIDS Cohort Study Investigators. JAMA 1998;280:1497–503.[Abstract/Free Full Text]
  8. Johnston SC. Effect of endovascular services and hospital volume on cerebral aneurysm treatment outcomes. Stroke 2000;31:111–17.[Abstract/Free Full Text]
  9. Johnston SC. Combining ecological and individual variables to reduce confounding by indication: case study, subarachnoid hemorrhage treatment. J Clin Epidemiol 2000;53:1236–41.[ISI][Medline]
  10. Kennedy P. A guide to econometrics. Cambridge, MA: MIT Press, 1998.
  11. Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol 2000;29:722–9.[Abstract/Free Full Text]
  12. McClellan M, McNeil BJ, Newhouse JP. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA 1994;272:859–66.[Abstract]
  13. Stefanski LA, Buzas JS. Instrumental variable estimation in binary regression measurement error models. J Am Stat Assoc 1995;90:541–50.[ISI]
  14. Amemiya T. Advanced econometrics. Cambridge, MA: Harvard University Press, 1985.
  15. Sheppard L, Prentice RL, Rossing MA. Design considerations for estimation of exposure effects on disease risk, using aggregate data studies. Stat Med 1996;15:1849–58.[ISI][Medline]
  16. Chao WH, Palta M, Young T. Effect of omitted confounders on the analysis of correlated binary data. Biometrics 1997;53:678–89.[ISI][Medline]
  17. Neuhaus JM, Kalbfleisch JD. Between- and within-cluster covariate effects in the analysis of clustered data. Biometrics 1998;54:638–45.[ISI][Medline]
  18. Hauck WW, Neuhaus JM, Kalbfleisch JD, et al. A consequence of omitted covariates when estimating odds ratios. J Clin Epidemiol 1991;44:77–81.[ISI][Medline]
  19. Hauck WW, Anderson S, Marcus SM. Should we adjust for covariates in nonlinear regression analyses of randomized trials? Control Clin Trials 1998;19:249–56.[ISI][Medline]
  20. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics 1988;44:1049–60.[ISI][Medline]
  21. Joseph KS. The evolution of clinical practice and time trends in drug effects. J Clin Epidemiol 1994;47:593–8.[ISI][Medline]
  22. Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. J Am Stat Assoc 1996;91:444–55.[ISI]
  23. Hausman JA. Specification tests in econometrics. Econometrica 1978;46:1251–71.[ISI]
  24. Bound J, Jaeger DA, Baker RM. Problems with instrumental variables estimation when the correlation between the instruments and the endogenous explanatory variable is weak. J Am Stat Assoc 1995;90:443–50.[ISI]