Clinical Operational Research Unit, University College London,
1 St Albans City Hospital, St Albans,
2 Royal Hampshire County Hospital, Winchester,
3 Broomfield Hospital, Chelmsford,
4 Robert Jones and Agnes Hunt Orthopaedic Hospital, Oswestry,
5 Rheumatology Research Unit, University of Leeds, Leeds,
6 Harrogate Hospital, Harrogate,
7 Grimsby Hospital, Grimsby,
8 Basingstoke District Hospital, Basingstoke,
9 Medway Hospital, Gillingham and
10 Royal Hallamshire Hospital, Sheffield, UK
Correspondence to:
S. Gallivan, Clinical Operational Research Unit, 4 Taviton Street, London WC1H 0BT, UK.
Survival analysis using methods due to Kaplan and Meier [1] is the recommended statistical technique for use in cancer trials [2]. It is applied by analysing the distribution of patient survival times following their recruitment to a study. The analysis expresses these in terms of the proportion of patients still alive up to a given time following recruitment. In graphical terms, a plot of the proportion of patients surviving against time has a characteristic decline (often exponential), the steepness of the curve indicating the efficacy of the treatment being investigated. The more shallow the survival curve, the more effective the treatment. KaplanMeier analysis can be used to test the statistical significance of differences between the survival curves associated with two different treatments.
In recent years, such survival analysis has found widespread application in the field of rheumatology [37], being used to examine how long patients continue to tolerate a particular drug before either side-effects or lack of efficacy cause them to be switched to some other therapy. Although intuitively this seems analogous to oncology trial analysis, little research seems to have been done to confirm that survival analysis is a valid method to examine drug survivorship. In spite of this, survival analysis is becoming not only accepted within rheumatology, but dogmatically insisted upon. Indeed the authors have experienced harsh criticism from referees for not using it.
Of course, as a well-developed and investigated statistical method, the mathematical logic underlying KaplanMeier analysis is impeccable and we certainly would not dispute its value when applied in appropriate circumstances. However, just because a statistical technique has been shown to be mathematically sound and applicable to one clinical study does not make it universally applicable to all clinical studies.
Frustrated by numerous referees insisting on KaplanMeier analysis and concerned about the applicability of the method to drug survival, we have carried out some simple tests of the assumptions underlying the method. Sadly, it does not fare well.
In order to test the method, we used data for 606 patients recruited as part of a follow-up study investigating the long-term progression of patients initially recruited with early rheumatoid arthritis (RA). This forms part of a programme of research known as the Early Rheumatoid Arthritis Study (ERAS). Our tests of KaplanMeier methods concern a subgroup of the full ERAS patient group comprising 606 patients who completed at least 2 yr follow-up and were prescribed sulphasalazine as their first disease-modifying anti-rheumatic drug (DMARD). We have focused attention on patients whose first DMARD was sulphasalazine since it was the drug most commonly prescribed as a first DMARD within the ERAS cohort.
We have investigated three assumptions that are fundamental to the validity of KaplanMeier survival analysis in relation to drug survivorship:
In order to examine if these assumptions are valid, we divided data for our cohort into two groups, according to the criterion being examined. Thus, to examine whether drug survival depends on when patients were recruited to the study (assumption A), the 606 patients were divided into two groups of 303 patients each, the former group having earlier recruitment dates than the latter. Survival curves were plotted to compare the drug survival characteristics of sulphasalazine between the two groups. Similar methods were used to examine assumptions (B) and (C). The results of this process are summarized in Fig. 1. In each case, the drug survival characteristics were significantly different between the two groups (P<0.01, log rank test for each case). It should be noted that these tests are not wholly independent. However, correction for multiple testing would have little effect on our conclusions.
|
This, of course, should be disturbing news for advocates of KaplanMeier drug survival analysis. Of course we cannot infer whether the same is true for other follow-up studies, which may be some solace. But against this, the onus should now be on those who use such drug survival analysis to show that it is valid, given the evidence that there is one case where it is not.
References