1 Department of Medicine, University of Pennsylvania School of Medicine, Philadelphia, PA 19104-6021
2 Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania School of Medicine, Philadelphia, PA 19104-6021
3 Center for Education and Research on Therapeutics, University of Pennsylvania School of Medicine, Philadelphia, PA 19104-6021
4 Center for Bioethics, University of Pennsylvania School of Medicine, Philadelphia, PA 19104-6021
5 Johnson & Johnson Pharmaceutical Research & Development, LLC, Titusville, NJ 08560
We were gratified to see in the January 15, 2005, issue of the Journal further discussion of the ethics of conducting clinical trials that are underpowered to reliably detect clinically important treatment differences (1, 2
). Unfortunately, Bacchetti et al.'s central argument, that "smaller sample sizes have a more favorable ethical balance [than larger ones]" (1
, p. 106), is based on conceptions of both social and individual benefits that we think are invalid.
First, the authors (1) argue that the net social benefit produced by a trial is a fixed quantity that should be divided by the number of study participants. However, if the trial is successful (e.g., if a study proves a new treatment effective for a certain condition), then each person afflicted by the condition benefits. The resultant net social benefit is thus the product of the value of the new treatment per patient and the number of those afflicted (including the study participants), rather than the quotient. If a clinically significant treatment effect failed to achieve statistical significance because the study was underpowered, no social benefit would be realized. Although Bacchetti et al. are correct that marginal contributions to statistical power decrease with increasing numbers of participants, the fact that each additional participant still increases the probability that this benefit is obtained at all means that each will augment the expected value of the trial (the product of the net social benefit and the probability that it is obtained).
Second, because people commonly participate in research for altruistic reasons (24
), and because additional participants increase the probability that a social benefit is obtained, each participant's expected individual benefit increases with larger sample sizes. If a new treatment is proven effective, each participant's altruistic motives are rewarded in full; if it is not, and the study was underpowered, then none are rewarded at all. On the other hand, if an adequately powered trial determines that a clinically important benefit is unlikely (recognizing the impossibility of "proving" the null hypothesis), then altruistic motives are still rewarded. Assuming, as Bacchetti et al. (1
) do, that the average burden per participant is constant across all possible sample sizes results in an improved risk-benefit ratio for individual research participants as the sample size increases.
Finally, Bacchetti et al. inaccurately characterize prior arguments about underpowered trials, including our own, as assuming that "a study's projected value is determined only by its power" (1, p. 105). We explicitly acknowledged 3 years ago that power is merely one determinant of a trial's validity, and hence its potential value (3
). Because "the purpose of conducting clinical investigations of a drug is to distinguish the effect of a drug from other influences, such as spontaneous change in the course of the disease, placebo effect, or biased observation" (5
), we believe that investigators must document adequate power prior to exposing patients to the risks and burdens of research. However, value also is influenced by the choice of endpoints, the choice of comparators, the generalizability of the results, and the nature and severity of the disease under study. Acceptance of Bacchetti et al.'s arguments encourages the conduct of not only underpowered trials but also trials among homogeneous patient samples because such studies tend to reduce the variability in outcome measures, thereby decreasing the number of subjects required for any given level of power. Because such studies would have limited generalizability, they would also be of limited value.
Accepting Bacchetti et al.'s conclusion that "ethics committees ... need not consider whether a study is too small" (1, p. 108) would not only represent a major step backwards in the protection of human research subjects but also encourage the conduct of studies less likely to improve public health. These would be unfortunate consequences under any circumstances, particularly if spurred by flawed reasoning.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
References |
---|
![]() ![]() |
---|