Swiss Federal Office of Public Health Division of Epidemiology and Infectious Diseases 3003 Berne, Switzerland
Greenland et al. (1) offer a critical discussion of problems arising in the conditional logistic regression model when information in the data is weak relative to background knowledge. They warn against an uncritical use of this model and recommend that "analysts need to inspect their data in detail in order to alert themselves to the possible danger [of statistical bias]..." (1, p. 537). We fully agree on this point but would like to add two remarks to their investigation.
Statistical inference, the methodology used most widely to tackle inference problems in the empirical sciences, is hampered by two major difficulties:
The paper by Greenland et al. (1) provides an instructive illustration of both problems.
With regard to point 1, the authors focus on the conditional logistic regression model. This limitation is of particular concern because it ignores model uncertainty. Hierarchical models are mentioned as an alternative, but no estimates are presented. These models might be better suited to incorporate the right degree of matching (see reference 3) and might yield more reasonable estimates.
With regard to point 2, the authors present various frequentist estimates and, finally, Bayesian estimates for two fairly vague prior distributions reflecting background knowledge. The reader gets the impression that there is considerable uncertainty about the underlying odds ratios, but even more uncertainty about what strategy to follow when analyzing sparse data problems similar to the ones presented. While the first impression is correct, the second one is homemade by the statistical community. The prevalent school (frequentism) offers various methods to obtain point estimates, standard errors, and uncertainty bounds. All of these methods seem fairly reasonable, but there exists no rational way to decide which one to use.
We think that these fundamental problems necessitate a more general and coherent approach to data analysis. An approach that addresses these problems should proceed along the following lines: 1) the analysis of different plausible models to cope with model uncertainty. Often, a best model can be singled out when information in the data is strong. Otherwise, a fully Bayesian approach using model probabilities should be taken; 2) the implementation of the Bayesian methodology, which offers the advantage of being able to incorporate relevant background knowledge and avoiding those rather arbitrary decisions about the type of method for computing point and interval estimates.
In public health, well-calibrated risk assessments are crucial, and decision-making is often the final goal. An analysis that enables the scientist to fuse all of the relevant information is certainly advantageous and is therefore desirable. Epidemiologists might benefit by adopting the Bayesian approach that combines their subject-matter knowledge with empirical data.
REFERENCES
Department of Epidemiology UCLA School of Public Health Department of Statistics UCLA College of Letters and Science Los Angeles, CA 900951772
I thank Neuenschwander and Zwahlen (1) for their perceptive comments. The issues they raise are too deep to address here, but I have published a number of articles that discuss them. Those articles are based on works by others, most of all those by Leamer (2
) and Good (3
). The work by Leamer (2
, chap. 5) is especially relevant in showing how models and modeling procedures correspond to prior distributions and in demonstrating the profound logical and scientific flaws inherent in still-standard approaches (e.g., stepwise regression). A few key points from that book are summarized elsewhere (4
, 5
).
One view of modeling derived from theory and empirical studies (2, 3
, 6
, 7
) is that a model is a smoothing device to estimate various nonparametrically defined parameters such as population risk ratios. Models whose link functions (8
) are canonical (like the logistic) or close to it work much better in this role than do other models when the data set is small or sparse. To compensate for this technical limitation on the choice of link function, one should attempt to keep the linear predictor flexible (e.g., by entering products, powers, or splines). One then, however, faces the problem of balancing the number of terms in the linear predictor against the limits of data.
Hierarchical regression (mixed or multilevel modeling) allows one to enter many more terms in the predictor than is possible with ordinary methods; it does so by using smooth, contextual constraints that limit the effective degrees of freedom of the model (9). These constraints reduce the small-sample and sparse-data artifacts that plague both unconditional and conditional maximum likelihood (9
, 10
). The additional terms also allow better uncertainty assessments than do standard model-selection techniques (9
, 11
).
Because hierarchical regression has both Bayesian and frequentist rationales (12), it offers an operationalization of the Bayesian-frequentist fusion advocated by statisticians who recognize the fallacies in mechanical or monolithic approaches to data analysis (3
, 13
15
). An alternative approach popular among some Bayesians is to average results over a finite set of models (16
); this strategy may be viewed as a crude approximation to the smooth averaging attainable with in hierarchical regression (15
, 17
). Regardless of what methods they use, I believe that epidemiologists (and, indeed, all scientists) should understand Bayesian probability logic (18
20
), for the harsh light such logic casts on questionable statistical methods (such as the use of "noninformative" or "reference" priors in Bayesian statistics) will encourage analyses that are more sound scientifically than the current norm.
REFERENCES