Reply

Andy Vail2 and Liz Gardener1

R & D Support Unit, Hope Hospital, Stott Lane, Salford M6 8HD and 1 Salford Royal Hospitals NHS Trust, Salford, UK

2 To whom correspondence should be addressed. e-mail: andy.vail{at}man.ac.uk

We welcome Dr Dickey’s remarks (Dickey, 2003Go) and agree with a great many of them. Most importantly, we agree with the concluding paragraph: it is important that all clinically valuable information is analysed and reported, that comparisons should be reported regardless of statistical significance, and that clinical knowledge is essential to the design and interpretation of clinical studies.

Our concern (Vail and Gardener, 2003Go) is with statistical errors that preclude useful interpretation of results by clinicians. We described several such errors, of which Dickey highlights two.

First, the ‘unit of analysis’ refers to whether success rates (for implantation, pregnancy, live birth, etc.) are reported per woman, or per cycle or other denominator. Here we share Dickey’s concern that too many questions are ‘answered so simplistically as to be of questionable usefulness’. Where women are randomized, either they should be the unit of analysis or statistical adjustment is necessary to account for the lack of independence between analysed units. Such statistical adjustment is not available in much mainstream software and is beyond the usual postgraduate medical syllabus. We would advocate a simple, valid analysis using women as the denominator rather than a simple invalid analysis using cycle as the denominator. It is not our view that trials should be limited to only one cycle per woman. Clinical practice should determine trial design.

Dickey raises valid issues in epidemiological comparisons. There is a long-recognized need to adjust crude rates per person for ‘case-mix’ (Spiegelhalter, 1999Go). If reasons for drop-out are unrelated to treatment then analysis is straightforward using standard ‘time-to-event’ analyses such as Kaplan–Meier. Where drop-out is related to treatment the simplest strategy is to assume failure for these cases. It is not true that use of an inappropriate denominator such as ‘per cycle’, thus introducing a unit of analysis error, resolves these issues.

Secondly, Dickey highlights the difficulty of selecting outcome measures. To avoid problems of multiple hypothesis testing, trialists should pre-specify a primary analysis and consider other analyses as secondary. In the case of subfertility treatments the obvious primary outcome for clinicians and patients clearly involves live birth, although we are not in a position to enter the argument of singleton versus multiple births. Trialists may prefer to power their trials to detect differences in a ‘primary’ outcome more immediately influenced by the intervention under study. Where these are interim outcomes on the path to live birth, such as fertilization, implantation or pregnancy, we make two points. The first is that it is only worth reporting analyses that are statistically valid with respect to the unit of analysis. The second is that women should be followed until birth outcome is ascertained (either positive or negative). Gains that appear worthwhile in interim outcomes may be diluted to such an extent by the factors that Dickey describes as to have no practical benefit. Worse, interventions may have later repercussions if, for example, facilitation of implantation allows a non-viable embryo to reach the pregnancy stage.


    References
 Top
 References
 
Dickey, R.P. (2003) Clinical as well as statistical knowledge is needed when determining how subfertility trials are analyzed. Hum. Reprod., in press.

Spiegelhalter, D.J. (1999) Surgical audit: statistical lessons from Nightingale and Codman. J. Roy. Statist. Soc. A, 162, 45–58.[CrossRef][ISI]

Vail, A. and Gardener, E. (2003) Common statistical errors in the design and analysis of subfertility trials. Hum. Reprod., 18, 1000–1004.[Abstract/Free Full Text]





This Article
Extract
FREE Full Text (PDF )
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Request Permissions
Google Scholar
Articles by Vail, A.
Articles by Gardener, L.
PubMed
PubMed Citation
Articles by Vail, A.
Articles by Gardener, L.