The IVF league tables: time for a reality check

Khaldoun Sharif1 and Masoud Afnan

Assisted Conception Unit, Birmingham Women’s Hospital, Birmingham B15 2TG, UK

1 To whom correspondence should be addressed. e-mail: khaldoun.sharif@bham-womens.thenhs.com


    Abstract
 Top
 Abstract

 Discussion
 References
 
In many countries the reported results of the different IVF clinics are published nationally by the relevant regulatory bodies, such as the Human Fertilisation and Embryology Authority in the UK. The published format suggests that clinics are being compared on a valid basis, and the involvement of the regulatory bodies apparently gives this comparison the official seal of authenticity. However, clinics can control many factors in patient selection, management and reporting, and these can inflate the reported results. Consequently, patients, instead of being appropriately guided, can be misled by these so-called ‘IVF league tables’. The IVF community, including users, providers and regulatory bodies, need to recognize these factors and to develop a way forward for comparing clinics results on an equal and a valid basis. In this paper we discuss the factors that could affect the reported IVF results, and suggest possible options for valid comparison.

Key words: IVF/league tables/results


   
 Top
 Abstract

 Discussion
 References
 
"The power of accurate observation is often viewed as cynicism by those who lack it." G.B.Shaw

The main aim of an infertility service is to achieve a successful pregnancy in infertile couples, who would logically prefer to go to units with better ‘results’. These results can be divided (for the purpose of this discussion) into real, actual, and reported. ‘Real’ results refer to the live birth rate in all infertile patients seen at the unit. ‘Actual’ results refer to those in patients actually treated in the unit, i.e. patients seen minus patients excluded from treatment for various reasons. ‘Reported’ results are those that the unit gives out to its prospective patients and reports to statutory or voluntary regulatory bodies, such as the Human Fertilisation and Embryology Authority (HFEA) in the UK. These ‘reported’ results then get published in the regulatory body’s official documents and on its world-wide web home pages and thus assume a sort of official seal of authenticity. Additionally, as all units’ results get published in the same document, they assume a dimension of comparability. Patients usually believe they are comparing like with like. This is further emphasized by the way these results are reported. For example, the HFEA reports the results of IVF and ICSI for the UK as a whole as well as for individual units. These results are reported in three uniform formats: live birth per started cycle, live birth per oocyte collection, and live birth per embryo transfer. Each of these is reported per women <38 years of age and for all ages, further suggesting comparability.

But are these results comparable, or could they be skewed such that particular units have higher ‘reported’ results but not necessarily higher ‘actual’ or ‘real’ results? Favourable patient selection to improve the results has already been suggested as a direct result of publishing these so called ‘league tables’ (Anonymous, 1999Go). Our paper is an attempt to explore these issues further, and suggest ways forward to improve the comparability of the results.

We have presented in Table I some patient selection criteria that could be used to make the reported IVF results higher than the actual or real results and the rationale of their use. In Table II we have done the same for some reporting practices and cycle management issues. In addition, after each item, we have addressed the question ‘how to find out’ if the unit is following such practices.


View this table:
[in this window]
[in a new window]
 
Table I . Examples of patient selection criteria that can alter the reported results.
 

View this table:
[in this window]
[in a new window]
 
Table II . Reporting and management practices that can alter the reported results
 

    Discussion
 Top
 Abstract

 Discussion
 References
 
The HFEA publishes the reported results of each UK IVF clinic in its ‘Patients’ Guide’ (HFEA, 2000Go). The American Society for Reproductive Medicine also publishes similar results for the USA (SART, 1998Go) and other countries are following suit (Anonymous, 1999Go). One of the stated aims of publishing these results is to guide the patients to choose their clinic. As their main aim is to ‘have a baby’, not surprisingly the main point that directs patients’ choice is the chances of success in each clinic. However, as evident from Tables I and II, the ‘reported’ results do not necessarily reflect the ‘real’ or ‘actual’ results. It is therefore imperative that the HFEA and similar regulatory bodies try to publish comparable results as much as possible.

One way of doing so is to publish the results for various age groups, which is currently being done (e.g. <38 years of age and all age groups in the UK). As the reporting system becomes more sophisticated, other factors that impact the reported results should also be published. Our paper is an attempt to draw the attention (of the regulatory bodies, the IVF community, and the patients) to some of these factors, many of which have been proven to significantly affect the results (Templeton et al., 1996Go; Sharif et al., 1998Go)

Also at play here is the ‘double jeopardy’ effect. This means that patients not accepted in a certain clinic (because they have lower probability of success) will often go to another clinic, which may not operate similar selection policy. This will artificially further widen the difference between the results of the two clinics. In fact, when all these methods are applied together, they could have a large scale ‘domino effect’ on the results of the unit employing them, which may not be easily apparent to some one looking at the results in their currently published format.

We have also tried, in the ‘how to find out’ sections of the tables to draw attention to how these factors could be ascertained for each clinic. Perhaps the ultimate check would be through the regulatory bodies requiring each clinic to report their exact patient selection criteria. In other aspects, such as the reporting of cancelled cycles, a clearly agreed, and adhered to, standard is required.

A possible way forward to aid comparison is the development of an agreed ‘standard patient group and outcome’. For example, this group could be women <35 years of age with normal basal FSH levels undergoing their first cycle of treatment. Similarly the outcome could be the live birth rate per cycle started—the latter defined as the start of any medications. This would reduce the impact of ‘patient-mix’ on the comparability of results between different centres.

Publishing IVF results may often seem like a thankless task for regulatory bodies, as they keep being criticised on a statistical basis (Marshal and Spiegelhalter, 1998Go) as well as on the notion that they may encourage certain adverse practices (Anonymous, 1999Go). Yet they remain, in our opinion and many of our patients’, very valuable. Probably we will never make them perfect, but our paper is an attempt to make them better.


    References
 Top
 Abstract

 Discussion
 References
 
Anonymous (1999). Information versus choice in infertility treatment. Lancet, 353, 1895.

HFEA (2000). The Patients’ Guide to IVF Clinics. Human Fertilisation and Embryology Authority, London.

Marshal, E.C. and Spiegelhalter, D.J. (1998). Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. Br. Med. J., 316, 1701–1704.[Abstract/Free Full Text]

SART (1998). Assisted reproductive technology in the United States and Canada: 1995 results generated from the American Society for Reproductive Medicine/ Society for Assisted Reproductive Technology Registry. Fertil. Steril., 69, 389–398.[CrossRef][ISI][Medline]

Sharif, K., Elgendy, M., Lashen, H. and Afnan, M. (1998). Age and basal follicle stimulating hormone as predictors of in vitro fertilisation outcome. Br. J. Obstet. Gynecol., 105, 107–112.[ISI][Medline]

Templeton, A., Morris, J. and Parslow, W. (1996). Factors that affect outcome of in-vitro fertilisation treatment. Lancet, 348, 1402–1406.[CrossRef][ISI][Medline]