Department of Psychiatry and Behavioural Science, University College London, and Honorary Consultant Psychiatrist, North East London Mental Health NHS Trust, Warley Hospital, Mascalls Park, Brentwood, Essex CM14 5HQ, UK
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To re-evaluate the evidence comparing clozapine with conventional antipsychotics and to investigate sources of heterogeneity.
Method Individual studies were inspected with assessment of clinical relevance of results. Meta-regression analysis was performed to investigate sources of heterogeneity.
Results Ten trials were examined. Recent large-scale studies have not found a substantial advantage for clozapine, especially in terms of a clinically relevant effect. Meta-regression showed that shorter study duration, financial support from a drug company and higher baseline symptom score consistently predicted greater advantage of clozapine.
Conclusions It may be inappropriate to combine studies in meta-analysis, given the degree of heterogeneity between their findings. The benefits of clozapine compared with conventional treatment may not be substantial.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Individual study results were tabulated and examined. The difference in symptom scores between the clozapine group and the comparison group at the end of treatment as a percentage of the post-treatment score in the control group was calculated. This was done in order to compare results with the 20% difference that is commonly said to represent clinically significant improvement in individuals in terms of symptom ratings in treatment-resistant cases (Rosenheck et al, 1997; Wahlbeck et al, 1999).
Meta-analysis was conducted to examine heterogeneity between studies, which is assessed by testing the weighted variation of individual study results about the mean effect. The outcomes of individual trials were converted to standardised mean differences (SMDs) to allow the results of studies using different outcome measures to be combined. The SMD is usually calculated as the difference between the mean of the experimental group and the mean of the control group divided by the combined standard deviation (Hedges & Olkin, 1985). The level of symptoms at the end of the study or change in symptoms over the course of the study was defined as the main outcome of interest, since this is the principal objective of clozapine therapy in treatment resistance. All studies used either the Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1962) or the Positive and Negative Syndrome Scale (PANSS; Kay et al, 1987) to rate symptoms. Intention-to-treat data were used if possible. In one study (Rosenheck et al, 1997) standard deviations were not available for the intention-to-treat data and so the standard deviations obtained from the analysis of treatment completers were used instead.
In the study by Essock et al (1996) it was difficult to decide which data to use as the basis for calculating SMD, since this was a naturalistic study and a large proportion of people in the control group were prescribed clozapine at some point during the study. Intention-to-treat data were available for change in BPRS score at the end of the study from a later publication by the same authors, who noted that the results were similar when the analysis was performed with crossovers excluded (Essock et al, 2000). The intention-to-treat data were therefore used as the primary basis for calculating SMDs, but sensitivity analysis was conducted using post-treatment scores with crossovers excluded. In this case standard deviation was calculated from the t-value provided (Essock et al, 1996) and the number of patients in each group was provided on request by the authors (N. Covell, personal communication, 2002). Sensitivity analysis was also conducted using non-intention-to-treat data from the study by Kane et al (2001), for reasons explained below.
Meta-regression analysis was then conducted to investigate possible sources of heterogeneity. This consists of a weighted regression analysis using the individual study SMDs as the data points. The following trial characteristics were investigated to see whether they predicted outcome in terms of the SMD:
These factors were considered a priori to be potential predictors of outcome. Duration was considered to be a source of heterogeneity in the Cochrane review (Wahlbeck et al, 2000a). There has also been some suggestion that initial severity (Umbricht et al, 2002) and financial support (Wahlbeck et al, 2000a) might predict outcome. Size of study was examined as a proxy for study quality and because it may indicate publication bias (Sterne et al, 2001). Year of publication was examined to assess whether there was an effect of the initial enthusiasm for a novel treatment. The effect of a pre-trial treatment period was examined because of the possibility that this could introduce selection bias by excluding participants who respond to a new trial of a standard antipsychotic. Dose ratio was examined because of suggestions that the use of overly high dosages of comparator drugs have contributed to clozapine's apparent superiority.
A univariate analysis was conducted to explore the associations between outcome and the individual hypothesised explanatory variables. The reduction of the tau-squared value (that is, the residual variance between study results) was noted for each analysis. The analysis was repeated using the data-set with non-intention-to-treat data for the studies by Essock et al (1996) and Kane et al (2001). Multivariate analysis was considered but it was decided that there were too few trials for it to produce reliable results.
All meta-analysis was conducted using the STATA version 7.0 statistical package (Stata, 2001).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Table 2 shows the results of individual studies according to the outcome measures used in each. It is clear that there is great variation between study results. The study by Kane et al (1988) is the only sizeable trial to have found a substantial and unequivocal difference between clozapine and standard neuroleptics. The largest trial conducted found only small differences in symptom scores and improvement rates in the intention-to-treat analysis, although the former are reported to be statistically significant because of the large sample size (Rosenheck et al, 1997). The large naturalistic study by Essock et al (1996) is difficult to interpret because 66% of patients in the control group were given clozapine at some time during the course of the trial. There were no differences on symptom scores at the end of the trial, both in intention-to-treat analysis and analysis excluding crossovers. Survival analysis of improvement rates in the two groups showed almost identical survival curves, according to the authors. Slight differences favouring clozapine at the end of the study were based on small numbers and were not felt to represent real differences (N. Covell, personal communication, 2002).
|
Only two studies reported data on rates of re-hospitalisation or discharge. Essock et al (1996) reported that there was no difference in discharge rates between patients assigned to clozapine and those assigned to usual care, but patients in the clozapine group who were discharged were less likely to be readmitted. Rosenheck et al (1997) found no difference in the proportion of patients who were readmitted during the study but found that overall the group of patients assigned to clozapine spent 24 fewer days in psychiatric in-patient beds. Neither of the out-patient trials presented data on admission rates.
Table 2 also shows the difference in improvement ratings on the main outcome measure expressed as a percentage of the post-treatment score for the control group. This difference was greatest in the two earliest trials, but falls far short of the 20% clinically relevant difference in the more recent trials. Exceptions are the small trial in children (Kumra et al, 1996), the study conducted in Taiwan (Hong et al, 1997) and the latest study of out-patients (Kane et al, 2001). The last is the only long-term study to find that clozapine was substantially superior to a typical anti-psychotic. It is worth noting therefore that 50% of patients in the haloperidol group were withdrawn because of lack of efficacy. It is difficult to understand this finding in a group of patients who were considered stable enough to live in the community. The fact that they had to be withdrawn from the trial suggests that their condition had deteriorated more than would be expected in the normal course of events. Information on how these patients fared subsequently is necessary to address this possibility. It is also unusual that the rate of withdrawal because of the lack of efficacy was chosen as the main measure of outcome. Considering only patients who remained in the trial, final total BPRS scores were similar for groups allocated to haloperidol (n=11, mean 40.2, s.d.=12.2) and to clozapine (n=23, mean 37.5, s.d.=9.0). Rates of withdrawal were also very variable across trials, ranging from 9.5% to 43% in the clozapine groups and from 8 to 72% in the control groups.
Meta-analysis
Figure 1 shows that
heterogeneity between study results was substantial and the statistical test
for heterogeneity was highly significant (Q=38.2, d.f.=8, P < 0.001).
Overall meta-analysis using intention-to-treat data and a fixed effects model
produced an overall effect of 0.38 standard deviations (95% CI
0.270.50) in favour of clozapine over the standard antipsychotic. A
random effects model yielded a similar result of 0.44 standard deviations (95%
CI 0.150.73). The between-study variance was 0.14. Even when trials
were grouped according to their duration, heterogeneity was still substantial.
For the three long-term trials the heterogeneity statistic was 12.9 (P=0.002)
and for the six short-term studies it was 11.1 (P=0.05).
|
Table 3 shows the results of univariate meta-regression analysis. Using the intention-to-treat data-set, it was found that duration, initial BPRS score and financial support from the pharmaceutical industry predicted outcome. Studies that found larger differences in favour of clozapine were of shorter duration and the participants had higher initial BPRS scores. Trials where there was information that some financial support had been provided by the pharmaceutical company manufacturing clozapine also showed a greater benefit of clozapine compared with the conventional neuroleptic. Study duration had the strongest effect on outcome and reduced the between-study variance by the greatest amount.
|
Using the data-set with non-intention-to-treat data for Essock et al (1996) and Kane et al (2001) showed that the same variables predicted outcome, but also showed a significant and strong effect for year of publication with between-study variance reduced to zero. Source of financial support had a stronger effect in this analysis and also reduced between-study variance to zero.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The effect of treatment on symptom levels was chosen for the purposes of the meta-analysis and exploration of heterogeneity. This was felt to be the best reflection of the objectives of clozapine treatment, and as such was present in all trials. Other main published outcomes are summarised, but were not used in the meta-analysis. Continuous data have been used, since these were available for all studies, whereas categorical data were not. Moreover, categorical data may be more subject to bias, if the basis of categorisation has not been specified a priori.
Meta-regression analysis has severe limitations in a small set of nine trials. The small number of studies limits its power, but despite this some consistently significant effects were found. The results for year of publication were only apparent within the data with withdrawals excluded, which demonstrates the sensitivity of the analysis to decisions about which data to use. It is also important to note that meta-regression is an uncontrolled analysis, which lacks the protection of randomisation. It must therefore be viewed cautiously as an exploratory procedure. In addition, since multiple regression was not performed, the possibility of correlation or confounding between explanatory variables could not be explored.
Overall findings
Despite clozapine's reputation, there is substantial heterogeneity among
trials comparing clozapine with conventional antipsychotics for
treatment-resistant schizophrenia. Most recent trials have not replicated the
dramatic superiority shown by clozapine in early trials. In particular, they
fail to demonstrate that the differences between clozapine and conventional
anti-psychotics are clinically relevant in terms of the degree of difference
in reduction of BPRS or other symptom scores. It is interesting to note that
so far no differences have been found between the effects of clozapine and
other atypical antipsychotics in patients with treatment-resistant
schizophrenia (Tuunainen et al,
2002).
This analysis demonstrates the potential danger inherent in combining results from different studies while overlooking important variations between them. The influential Cochrane review (Wahlbeck et al, 2000a) might also have overestimated the effects of clozapine by using non-intention-to-treat data in the largest study, by Rosenheck et al (1997), and by excluding the large study by Essock et al (1996) from the analysis of effects on mental state. The reasons for excluding the findings of Essock et al (1996) from this analysis were not given but it may be because not all the data required were available in the published paper. Other results from the study were included in other analyses.
Sources of heterogeneity
In the Cochrane review including all clozapine trials, no association was
found between various measures of trial quality and outcome, but whether a
study had received commercial sponsorship did predict some outcomes (Wahlbeck
et al, 1999,
2000b). In the current
analysis it appeared that shorter duration of trial, higher levels of baseline
symptoms, commercial support and possibly earlier year of publication
predicted greater superiority of clozapine over conventional anti-psychotics.
The influence of the pharmaceutical industry has probably been understated in
some studies, since published reports declared only whether the study had
received direct funding from a pharmaceutical company. Financial interests of
individual authors were not declared. The duration of the study also emerged
as a strong predictor of outcome, suggesting that initial beneficial effects
may not be maintained over the long term. Only one small long-term study
suggested that clozapine might have substantial benefits
(Kane et al, 2001).
However, the high withdrawal rate from the haloperidol group in this trial
requires explanation and may have some bearing on the results obtained.
All trials in the current analysis used relatively high daily doses of conventional antipsychotics, including the most recent study (Kane et al, 2001). There was probably not enough variation in comparative dosage levels therefore to examine the impact on outcome adequately. This may explain why the results do not confirm those of Geddes et al (2000), who found that the advantages of atypical anti-psychotics were apparent only when doses greater than 12 mg of haloperidol or equivalent were used.
Initial severity and effects of clozapine
Although all patients in the trials examined were classified as having
treatment-resistant disease, there was considerable variation between trials
in average severity of baseline symptoms. The current analysis suggests that
among patients with treatment-resistant disease, the benefits of clozapine may
be most marked in those with higher levels of initial symptoms. Other data on
the relationship between response to clozapine and severity of illness are
inconsistent. In the large study of US veterans included here, high
hospital users showed a smaller advantage for clozapine over
haloperidol in terms of symptom reduction than low hospital
users (Rosenheck et al,
1999). In addition, differences between clozapine and haloperidol
were not significant in the high user group. In the later
out-patient study, patients who were functioning at a lower level at trial
entry as measured by Clinical Global Impression scores
(Guy, 1976) were less likely to
show an enhanced response rate to clozapine compared with haloperidol
(Umbricht et al,
2002). However, curiously, when this was controlled for, higher
baseline rates of symptoms as measured by the BPRS predicted better relative
response rates to clozapine. The relationship between severity and efficacy
found in this analysis may therefore be an ecological effect that may not
necessarily translate to the individual level. It is also not possible to know
from the current analysis whether the greater benefit of clozapine in patients
with higher baseline symptom levels would be maintained in the long term,
since all long-term studies were conducted with patients who had lower levels
of baseline symptoms.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Essock, S. M., Hargreaves, W. A., Covell, N. H., et al (1996) Clozapine's effectiveness for patients in state hospitals: results from a randomized trial. Psychopharmacological Bulletin, 32, 683697.[Medline]
Essock, S. M., Frisman, L. K., Covell, N. H., et al
(2000) Cost-effectiveness of clozapine compared with
conventional antipsychotic medication for patients in state hospitals.
Archives of General Psychiatry,
57,
987994.
Geddes, J., Freemantle, N., Harrison, P., et al
(2000) Atypical antipsychotics in the treatment of
schizophrenia: systematic overview and meta-regression analysis.
BMJ, 321,
13711376.
Guy, W. (1976) ECDEU Assessment Manual for Psychopharmacology. Revised DHEW Pub. (ADM). Rockville, MD: National Institute for Mental Health.
Hedges, L.V. & Olkin, I. (1985) Statistical Methods for Meta-analysis. San Diego, CA: Academic Press.
Hong, C. J., Chen, J.Y., Chiu, H. J., et al (1997) A double-blind comparative study of clozapine versus chlorpromazine on Chinese patients with treatment-refractory schizophrenia. International Clinical Psychopharmacology, 12, 123130.[Medline]
Howanitz, E., Pardo, M., Smelson, D. A., et al (1999) The efficacy and safety of clozapine versus chlorpromazine in geriatric schizophrenia. Journal of Clinical Psychiatry, 60, 4144.[Medline]
Kane, J., Honigfeld, G., Singer, J., et al (1988) Clozapine for the treatment-resistant schizophrenic. A double-blind comparison with chlorpromazine. Archives of General Psychiatry, 45, 789796.[Abstract]
Kane, J., Marder, S. R., Schooler, N. R., et al
(2001) Clozapine and haloperidol in moderately refractory
schizophrenia: a 6-month randomized and double-blind comparison.
Archives of General Psychiatry,
58,
965972.
Kay, S. R., Fiszbein, A. & Opler, L. A. (1987) The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13, 261276.[Medline]
Klieser, E. & Schönell, H. (1990) Klinischpharmakologische studien zur behandlung schizophrener minussymptomatik. In Neuere Ansätze zur Diagnostik und Therapie schizophrener Minussymptomatik (eds H. J. Moller & E. Pelzer). Berlin: Springer Verlag.
Kumra, S., Frazier, J. A., Jacobsen, L. K., et al (1996) Childhood-onset schizophrenia. A double-blind clozapinehaloperidol comparison. Archives of General Psychiatry, 53, 10901097.[Abstract]
Overall, J. E. & Gorham, D. R. (1962) The Brief Psychiatric Rating Scale. Psychological Reports, 10, 799812.
Rosenheck, R., Cramer, J., Xu, W., et al
(1997) A comparison of clozapine and haloperidol in
hospitalized patients with refractory schizophrenia. Department of Veterans
Affairs Cooperative Study Group on Clozapine in Refractory Schizophrenia.
New England Journal of Medicine,
337,
809815.
Rosenheck, R., Cramer, J., Allan, E.,, Allan, E., et al
(1999) Cost-effectiveness of clozapine in patients with high
and low levels of hospital use. Department of Veterans Affairs Cooperative
Study Group on Clozapine in Refractory Schizophrenia. Archives of
General Psychiatry, 56,
565572.
Stata (2001) STATA Statistical Software, version 7. College Station, TX: Stata Corporation.
Sterne, J. A. C., Egger, M. & Smith, G. D. (2001) Investigating and dealing with publication and other biases. In Systematic Reviews in Health Care (eds M. Egger, G. D. Smith & D. Altman), pp. 189211. London: BMJ Publishing.
Tuunainen, A., Wahlbeck, K. & Gilbody, S. M. (2002) Newer atypical antipsychotic medication in comparison to clozapine: a systematic review of randomised trials. Schizophrenia Research, 56, 110.[CrossRef][Medline]
Umbricht, D. S., Wirshing, W. C., Wirshing, D. A., et al (2002) Clinical predictors of response to clozapine treatment in ambulatory patients with schizophrenia. Journal of Clinical Psychiatry, 63, 420424.[Medline]
Wahlbeck, K., Cheine, M., Essali, A., et al
(1999) Evidence of clozapine's effectiveness in
schizophrenia: a systematic review and meta-analysis of randomized trials.
American Journal of Psychiatry,
156,
990999.
Wahlbeck, K., Cheine, M., & Essali, A., et al (2000a) Clozapine versus typical neuroleptic medication for schizophrenia. Cochrane Database Systematic Reviews, 2001.
Wahlbeck, K., Tuunainen, A., Gilbody, S., et al (2000b) Influence of methodology on outcomes of randomised clozapine trials. Pharmacopsychiatry, 33, 5459.[CrossRef][Medline]
Received for publication October 10, 2002. Revision received April 10, 2003. Accepted for publication April 23, 2003.
Related articles in BJP: