Commentary: Population versus individual level causal effects

Felix Elwert and Christopher Winship

Department of Sociology, Harvard University, Cambridge, MA 02138, USA.

Felix Elwert, Department of Sociology, Harvard University, Cambridge, MA 02138, USA. E-mail: elwert{at}fas.harvard.edu

We congratulate Maldonado and Greenland1 (MG henceforward) on an interesting and provocative paper. Aiming at epidemiological applications, MG identify the causal effect of changing a distribution of exposures to a target population on the population's outcome distribution. Instead of applying a particular treatment to an individual, MG apply a distribution of treatments (the exposure distribution) to a population. By raising the unit of analysis from the individual to the population, MG depart in important respects from the standard model of counterfactual causal inference. Comparing MG's model to the standard model we make two points: First, MG's conceptualization of causal effects on the population level is valuable if the stable unit-treatment assumption (SUTVA) does not hold at lower levels, but the data requirements are steep. Second, as they mention, we emphasize that MG's population level estimates generally cannot be interpreted as estimates of average causal effects (ACE) in the standard individual-level approach.

Individual-level causal effects

We remind the reader of the standard individual-level presentation of the counterfactual model of causal inference, also known as the Rubin Model.2–4 Here, a particular treatment, t, is applied to a unit of analysis, i, (e.g. a person). The causal effect of t on i, {delta}i, is defined as the difference between the outcome of the unit under treatment, Yi(t), and the outcome of the same unit under control, Yi(c),

((I))

The ‘fundamental problem of causal inference’2 is that Yi(t) and Yi(c) cannot be directly observed together, because every unit of analysis is placed either in treatment or in control condition, but not in both at the same time. Therefore direct estimation of causal effects is impossible. As in MG, the solution is to substitute for the counterfactual observation another unit of analysis, j, which resembles i in all causally relevant respects other than treatment status.

Typically, we are not interested in the causal effect for a specific individual, but rather the average causal effect, ACE, in the study population:

((II))

In completely randomized experiments, the standard estimator for this parameter subtracts the mean outcome of the units in the treatment group from the mean outcome of the units in the control group:

((III))

This approach assumes that there is no interaction between units and that all treated units in the study receive identical treatments. Rubin terms this the ‘stable unit-treatment value assumption’ (SUTVA).5,6

The key virtue of randomization is to create balanced treatment and control groups that resemble each other across all causally relevant variables except treatment status. Techniques such as matching on propensity scores are available to achieve balance even in non-randomized observational studies.7,8

Population-level causal effects: utility and data requirements

MG's framework applies exposure distributions to target populations. Consequently, their unit of analysis is the population. This approach has merit, particularly when SUTVA does not hold within the population. Such situations occur frequently, e.g. in educational research where student test scores may be affected by tutoring their classmates received. Here one would want to use classes for units of analysis, rather than students.

Note, however, that the higher the unit of analysis, the more challenging the data requirements due to comparability of units of analysis, and identity of treatments.

The counterfactual model relies on the comparison of units of analysis that resemble each other in all causally relevant aspects except treatment status. To continue our educational example on the population (classroom) level, it would be necessary to find comparable classes, rather than comparable students. If SUTVA does not hold, this would not only involve comparable student populations, but also comparable dependencies between students within classes in order to ensure comparable peer effects.

The standard model further assumes that all units in the treatment group receive identical treatments. (Note that in a population level analogy to the standard individual-level model, a treatment group contains multiple target populations as units of analysis, each of which contains multiple individuals. Comparing a single target population to a single substitute would amount to working with a sample of N = 2.) If the treatment in question is an exposure distribution, as MG stipulate, identity of treatments across units (i.e. target populations) becomes much harder to assert. It depends on two aspects: (1) the exposure distribution's marginal distribution, which records the relative frequency of exposure levels within a target population; and (2) the mapping of distinct exposures from the exposure distribution onto individuals within a target population. If the population is heterogeneous in its members, different mappings of the same exposure distribution will induce different outcomes. Thus, to assure identity of treatments, both the marginal exposure distribution and its mapping have to be held constant across target populations in the treatment group. Due to these challenges, it seems advisable to choose the smallest unit for which SUTVA still holds as unit of analysis.

Dissimilarity of population-level causal contrasts and average causal effects

MG remark that ‘not all population causal contrasts can be interpreted as averages of individual causal effects of exposure’ (p.1039 in their paper). We would like to go further and argue that MG's population-level estimates will hardly ever represent average individual-level causal effects, because their approach generally does not sustain the conditions of a standard individual-level counterfactual analysis.

An example of the causal effect of smoking on lung cancer may convey the guiding intuition. Consider a population of 1000 men. Of these, 40% are highly susceptible to smoking-induced lung cancer and smoke, and 60% are minimally susceptible to cancer and do not smoke. The rate of lung cancer in this population is 40%. We want to estimate the effect of a change in the exposure distribution from 40% to 60% ever-smokers (similar to MG's example on p.1039). We identify a perfect substitute population of 1000 other men, 600 of whom smoke. However, all of these smokers are only minimally susceptible to lung cancer. In this population the cancer rate is 1%. MG's measure of causal contrast would indicate that increasing the exposure to smoking has decreased the incidence of lung cancer, even though each individual member would suffer an increased risk of cancer by taking up smoking. The reason is that different individuals smoke in the two populations.

This result makes sense in MG's approach, because it accurately identifies the population-level causal effect of having changed both the exposure distribution's marginal distribution and its mapping onto the target population. In the individual-level approach this result would be impossible, because the ACE cannot be negative if all {delta}i are positive. MG's population level estimates and the standard individual-level ACE are not equivalent.

References

1 Maldonado G, Greenland S. Estimating causal effects. Int J Epidemiol 2001;30:1035–42.[Abstract/Free Full Text]

2 Holland P. Statistics and causal inference. J Am Statist Assoc 1986;81: 945–70.[ISI]

3 Reiter J. Using statistics to determine causal relationships. American Mathematical Monthly 2000;107:24–32.[ISI]

4 Winship C, Morgan C. The estimation of causal effects from observational data. Annu Rev Sociol 1999;25:659–707.[CrossRef][ISI]

5 Rubin DB. Bayesian inference for causal effects: the role of randomization. Ann Stat 1978;7:34–58.

6 Little RJ, Rubin DB. Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches. Annu Rev Public Health 2000;21:21–45.

7 Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70: 41–55.[ISI]

8 Rubin DB, Thomas N. Combining propensity score matching with additional adjustments for prognostic covariates. J Am Statist Assoc 2000;95:573–85.[ISI]