Department of Psychiatry, University of Oxford
Unit for Social and Community Psychiatry, Newham Centre for Mental Health, London, UK
Correspondence: Professor Tom Burns, University Department of Psychiatry, Warneford Hospital, Oxford OX3 7JX, UK. E-mail: tom.burns{at}psych.ox.ac.uk
Mental health provision in most of the industrialised world seems to be in a state of continual revolution. In the UK the National Health Service (NHS) Plan (Department of Health, 2000) has mandated a fundamental reorganisation of care, with the establishment of 260 assertive outreach teams, 360 crisis resolution/home treatment teams, and 50 early intervention services (each comprising two to four teams) for first-episode psychosis. This enormous upheaval is an evidence-based policy drawing predominantly on mental health services research. How secure are its foundations?
The past 20 years have witnessed an explosion in the volume of health services research and in community mental health services research in particular (Mueser et al, 1998; Catty et al, 2002). Most of this research has looked at innovations often when a new service is compared against the prevailing standard care. The impression is of a relentless march of progress; outdated and unscientific services are replaced by more effective successors, which persist until something is proved to be even better.
This is not the entire picture. For decades now we have been trying to develop alternatives to in-patient care. Despite this, the acute in-patient ward has survived, essentially unchanged, and many of these alternative services have disappeared. A Health Technology Assessment review of home-based care (Burns et al, 2001) found (by following up the authors of the included studies by questionnaire) that virtually all of the experimental services had changed their practice since the studies were conducted, often departing significantly from the model described. Over half had ceased to operate at all, and even more strikingly 12% had ceased to exist even before the index study was published. This was the case even where significant advantages had been demonstrated for the experimental service.
With no formal evidence to support it,the acute in-patient ward has survived all these attempts to replace it. Its survival cannot be attributed simply to the level of investment in it. These wards have survived despite the wholesale closure of the large mental hospitals which used to house them. New wards have been built with radical improvements in the bricks and mortar, but with little change in clinical practice. In the UK the generic sector community mental health team also seems to have survived repeated attempts to replace it. In the USA and in most European countries, office-based practice stubbornly endures.
PROBLEMS WITH THE CURRENT RESEARCH BASE
Researchers often bemoan the failure of their findings to translate into practice and imply murky political forces. However, there are problems with our current approach to researching services. The National Institute for Clinical Excellence has voiced reservations about the generalisability of current studies, encouraging more effectiveness studies in mental health where services are assessed in real world conditions rather than in highly atypical research settings (Thornicroft et al, 1998). The effect size in effectiveness studies is almost invariably smaller, for predictable reasons (Coid, 1994).
Evaluations of innovation
An exclusive focus on evaluating newly initiated services gives a one-sided
picture. Most service developments inevitably require the demise of other
services and yet we rarely attempt to measure the effects of this. A notable
exception is a New Hampshire study evaluating the impact of closing day
hospitals to liberate staff for a new vocational rehabilitation service
(Drake et al, 1996). A
similar carefully conducted casecontrol study of a day hospital closure
in the UK (Goddard et al,
2004) was rejected without methodological criticism by three
journals simply deemed by the editors to be insufficiently
interesting. Finding out whether patients are significantly disadvantaged by
the loss of a closed service or whether their needs were equally met by other
parts of the system (and at what cost) should be of importance.
The case of day hospitals
Psychiatric day hospitals provide a particularly intriguing case study.
Acute psychiatric day hospitals have been proposed as an evidence-based
approach that has failed to translate into routine practice
(Marshall, 2003;
Briscoe et al, 2004).
There is some evidence that acute day hospitals can work as an alternative to
in-patient care for a substantial proportion of patients, yet it is very
difficult to maintain this focus on acute care over time.
Creeds group in Manchester demonstrated the ability to manage successfully a significant proportion of patients with acute illness without any in-patient care and to reduce duration of care for others (Creed et al, 1990). However, a comparative study of the same approach using a second day hospital less than 30 km distant found much less success, particularly in managing the patients with more severe illness (Creed et al, 1991). The authors concluded that the findings of their study needed to be interpreted in the light of local service issues, such as staffing and investment. This highlights the problems with studies where the service is essentially the subject of research but the unit of outcome analysed is the individual patient. It is not possible with any confidence to conclude which differences in local service issues contributed to the differences found. A randomised controlled trial with several day hospitals themselves as the unit of analysis would address this problem, but would be prohibitively costly and impractical.
An alternative approach would be to focus research specifically on the aspects of services that make them sustainable. These might not necessarily be the same aspects that make a service effective. For instance, clinicians from many day hospitals in Europe emphasise the importance of a fairly rigid programme, with clear rules about who can be admitted and what parts of the programme they must attend. Such factors may have little to do with effectiveness but are supposed to affect the sustainability of a given model in the real world and, thus, the longevity of the service. Yet these assumptions are based on personal experience and anecdotal evidence. No systematic study has ever investigated the factors that, for at least 25 years, have made acute day hospitals survive, change or disappear.
Researchers might be more useful partners in the dialogue with policy-makers if they provided evidence that addressed not only which services are effective, but also what components of the service model and the context influence sustainability over time.
RESEARCH IMPLICATIONS
Careful attention to and structured research into the survival of services and their changes over time could be one way of obtaining a better understanding of the determinants of clinical effectiveness and of the reasons why some innovations become and remain established and others do not. Such an approach might help distinguish the core features of durable services, in the same way that the Health Technology Assessment review mentioned above (Burns et al, 2001) tested for common components of successful services.
Methodological quality in community psychiatry studies has improved enormously since Coids criticisms, but there is still some way to go (Priebe & Slade, 2002). Studies have become larger and are often multicentred, with more attention to selection of outcome measures and statistical rigour. Despite this, naturalistic long-term service observations and studies deserve renewed consideration. The current raft of new community services in the UK should provide both enough features in common and enough differences to identify by careful study those that are associated with sustainability. Such studies would need to assess service components and context which, without a widely accepted taxonomy of services, would have to be comprehensive and detailed. To gain credibility and to develop research skills and capacity, more rigour must be brought to this exercise than is currently the case with many service-level audits. In particular, it requires the formulating of hypotheses to be tested prospectively.
Such an approach to naturalistic long-term studies will be challenging. As relevant factors for the sustainability and survival of services may be both internal to the service and contextual (Burns & Priebe, 1996), studies would need sociological research expertise. They require at least the same careful attention to detail as experimental studies to warrant a place in respectable scientific health service research. Without such a place, findings would have no influence and researchers of sufficient calibre would not be attracted to the field, with the result that, despite a patina of evidence-base, service development and practice would continue to be largely driven by anecdote, individual prejudices and political expedience.
REFERENCES
Briscoe, J., McCabe R., Priebe, S., et al
(2004) A national survey of psychiatric day hospitals.
Psychiatric Bulletin,
28, 160
-163.
Burns, T. & Priebe, S. (1996) Mental health care systems and their characteristics: a proposal. Acta Psychiatrica Scandinavica, 94, 381 -385.[Medline]
Burns, T., Knapp, M., Catty, J., et al (2001) Home treatment for mental health problems: a systematic review. Health Technology Assessment, 5, 1-139.[CrossRef]
Catty, J., Burns, T., Knapp, M., et al (2002) Home treatment for mental health problems: a systematic review. Psychological Medicine, 32, 383 -401.[CrossRef][Medline]
Coid, J. (1994) Failure in community care:
psychiatrys dilemma. BMJ,
308, 805
-806.
Creed, F., Black D., Anthony, P., et al (1990) Randomised controlled trial of day patient versus inpatient psychiatric treatment. BMJ, 300, 1033 -1037.[Medline]
Creed, F., Black, D., Anthony, P., et al (1991) Randomised controlled trial of day and in-patient psychiatric treatment. 2: Comparison of two hospitals. British Journal of Psychiatry, 158, 183 -189.[Abstract]
Department of Health (2000) The NHS Plan A Plan for Investment, A Plan for Reform. London: Department of Health.
Drake, R. E., Becker, D. R., Biesanz, J. C., et al (1996) Drug treatment versus supported employment for persons with severe mental illness: a replication study. Psychiatric Services, 47, 1125 -1127.[Abstract]
Goddard, K., Burns, T. & Catty, J. (2004) The impact of day-hospital closure on social networks, clinical status and service use: a naturalistic experiment. Community Mental Health Journal, 40, 223 -234.[CrossRef][Medline]
Marshall, M. (2003) Acute psychiatric day
hospitals. BMJ, 327, 116
-117.
Mueser, K. T., Bond, G. R., Drake, R. E., et al (1998) Models of community care for severe mental illness: a review of research on case management. Schizophrenia Bulletin, 24, 37 -74.[Medline]
Priebe, S. & Slade, M. (eds) (2002) Evidence in Mental Health Care. Hove:Brunner-Routledge.
Thornicroft, G., Wykes, T., Holloway, F., et al (1998) From efficacy to effectiveness in community mental health services. PRiSM Psychosis Study 10. British Journal of Psychiatry, 173, 423 -427.[Abstract]
Received for publication February 24, 2004. Accepted for publication May 11, 2004.
Related articles in BJP:
HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Psychiatric Bulletin | Advances in Psychiatric Treatment | All RCPsych Journals |