GenCertHort, Sainsbury Library, Said Business School, Oxford
Airedale General Hospital, Keighley
St Andrews Hospital, Northampton
Academic Unit of Psychiatry and Behavioural Sciences, University of Leeds, Leeds, UK
Correspondence: Professor Clive Adams, Academic Unit of Psychiatry and Behavioural Sciences, University of Leeds, 15 Hyde Terrace, Leeds LS2 9LT, UK.Tel: +44 (0)113 343 2730;fax: +44 (0)113 343 2723; e-mail: ceadams{at}cochrane-sz.org
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To identify and survey randomised trials relevant to forensic mental health services.
Method We searched 29 electronic bibliographic databases and acquired randomised trials involving sex offenders, arsonists or people clearly and actively aggressive, or abusive of children or spouse. Two researchers reliably extracted data.
Results Of 409 studies found, we able to acquire 300 for further inspection. They all involved particularly violent people (total n=28 669), mostly adult men; the mean study size was 197 (median 52, mode 60, range 1-1200).In these randomised trials over 700 interventions were evaluated and short-term outcomes were recorded on 345 different scales.
Conclusions Wider collaboration, rationalising treatments and simplifying outcomes could further strengthen the tradition of trialling in forensic psychiatry. Systematic reviews of these studies are overdue.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Although often imperfect (Chalmers et al, 1983; Thornley & Adams, 1998), randomised controlled trials remain the gold standard for the evaluation of mental health interventions (World Health Organization Scientific Group on Treatment of Psychiatric Disorders, 1991). This applies equally to research into the criminal justice system (Farrington & Petrosino, 2001). There are strong arguments for collecting and disseminating a regularly updated register of all randomised trials relevant to this area of work (Davies & Boruch, 2001). In mainstream healthcare the need of both providers and those receiving interventions to have ready access to all relevant high-quality research has been recognised, and the Cochrane Collaboration provides a structure by which this is undertaken. More recently, those working in education, social welfare and the criminal justice system have formed the Campbell Collaboration to address the needs of among others forensic mental health services (Farrington & Petrosino, 2001). However, forensic mental health straddles many professions and this fragmentation makes it difficult for healthcare professionals, criminal justice system workers, consumers, researchers and policy-makers to access relevant information. Anticipating this, Petrosino compiled a database of social, psychological, educational and criminological randomised and possibly randomised studies (Petrosino et al, 2000). Our work benefits from, supersedes and expands Petrosinos initiative. We created and surveyed a register of randomised controlled trials relevant to the management of violent and aggressive people.
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
A priori, we defined a subgroup of these studies as being of higher priority to forensic mental health services. These involved people who were clearly and actively aggressive, people abusive of children or spouse, sex offenders and arsonists, irrespective of age and whether they had underlying disorders. Studies of people at risk of becoming aggressive, for example juvenile offenders with no record of a specified aggressive act, were not included in this higher-priority group. Full copies of these high-priority studies were obtained and, using a data extraction sheet, S.C. recorded information on participants diagnoses, problematic behaviour, stage in criminal justice system, interventions and outcomes; C.A. checked the reliability of the coding by recoding a 10% random sample again. Methodological quality was scored according to the Jadad scale (Jadad et al, 1996). This rates the quality of reporting of randomisation (02), the quality of reporting of masking (02) and the quality of reporting of withdrawals (01). Low scores indicate poor reporting of methods and are linked with estimates of effect substantially greater than when a study is rated as good on the Jadad scale (Moher et al, 1998). This overestimate of effect from studies in which methodology is poorly reported is in keeping with other studies using different parameters to measure study quality (Juni et al, 2001). Data were stored in ProCite (Adept Scientific, Letchworth, UK) and then exported to Epi Info version 6.04d (Centers for Disease Control, Atlanta, Georgia, USA) for analysis.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Because of time constraints and despite our best efforts, we were only able to acquire and survey 300 of the 409 studies that we had identified as being of higher priority. There was an approximately 30% false-positive rate, so we estimate that about 70 studies remain outstanding. These proved inaccessible even through the British Library and direct approaches to the relevant people or institutions.
The reliability of most coding was good, with 90100% agreement for type of publication, country of origin, year of publication, language, participants gender, age and previous offences, intervention, number finishing trial, duration of trial, description of randomisation, description of masking and description of withdrawal. Agreement was between 50% and 90% for number randomised, problematic behaviour and diagnosis. Outcomes were not rated reliably (10% full agreement), probably because data were difficult to identify and involved many variables. Each rater found additional outcomes. The proportion of papers for which raters agreed on most (70%) outcomes was 95%, but the numbers of scales listed below is likely to be an underestimate.
Detailed survey of high-priority reports
The final column of Table 1
shows the proportion of unique high-priority studies identified in each
database as it was searched in turn. For example, after SPECTR (Social,
Psychological, Educational and Criminological Trials Register) was searched, a
Medline search still found 19% of the 300 studies. After 14 other databases
had been searched the Cochrane Library still found 11% of the total, and
Dissertation Abstracts, despite being 18th to be searched, also found 11% of
the total. Most of the 300 reports we were able to acquire were fully
published papers in academic journals (105 different journals), but no core
set of journals deserves a reputation for having a special interest in this
area, and 20% of reports were found only in dissertations or conference
proceedings.
Three-quarters (76%) of randomised controlled trials relevant to the management of very aggressive people originate from the USA. Of the remaining studies, 7% were from the UK, 4% from Europe and 12% from rest of the world (1% not specified). From 1995 there has been a steady increase in the number of relevant studies (1 per month 19912000).
A total of 28 669 people had been randomised within the 300 trials (mean sample size 197, median 52, mode 60, range 11200), and 280 studies clearly reported both the numbers starting and finishing the trial: the average attrition rate was 19% (95% CI 1527%). The great majority of reports involved men; only 15 trials (5%) solely randomised women. Most studies dealt with aggression in adulthood, although one-third focused on adolescents.
It was often difficult to ascertain diagnoses from reports, and when they were specified, often several were described in a single report. Specified diagnoses were categorised and frequencies tallied: psychotic disorders were the most commonly reported (178; 59%), followed by personality disorder (85; 18%), affective disorder (34; 11%), substance misuse (31; 10%), sexual disorders (30%; 10%), behaviour disorders (30; 10%), neurotic problems (26; 9%), problems of organic origin (21; 7%), learning disability (17; 6%) and dementia (7; 2%). Whether or not a diagnosis was specified, reports often listed the problematic behaviours of participants (Table 3). Almost a quarter of reports (n=68) specified that participants had been previously convicted.
|
Multiple interventions per study were common and in 300 randomised trials over 700 interventions were evaluated, including 315 different drug treatments, 21 different packages of care, 328 named talking therapies and over 90 management techniques. It seems likely that many of these therapies are similar, making these figures an overestimate. This, however, cannot be said with certainty, as so many of the variations were specified to be discrete.
Commonly recorded outcome measures in the 300 reports were violence or aggressive behaviour (195; 65%), mental state (121; 40%) adverse effects (94; 31%), global impression (67; 22%), recidivism, arrest or time to arrest (56; 19%) and social function (58; 19%). Cognitive function, attitude or understanding (33; 11%), selfesteem (22; 7%), satisfaction with treatment by participant (25; 8%) and family function (19; 6%) were also measured. Only 13 papers (4%) reported service outcomes admission, discharge, parole or release and few (11; 4%) specified economic outcomes. We also recorded the specific tools used to measure outcome; in total, 345 different scales were used in the 300 high-priority trials. Most trials measured outcomes at 6 months or less: 38 (13%) up to a week; 68 (23%) between 1 week and 6 weeks; 97 (32%) between 6 weeks and 6 months. The proportion of trials (73; 24%) that were longer than 6 months was significantly larger than that seen in other surveys of evaluative studies in psychiatry (Thornley & Adams, 1998) and 7 (2%) lasted longer than 5 years.
Overall, the quality of reporting was poor (median and mode Jadad score 2). Almost three-quarters of the reports (n=220) had a Jadad score of 2 or less, and only four reports (1%) were excellent (Jadad score of 5). These findings are similar to those of previous surveys of psychiatric trials (Thornley & Adams, 1998).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The 300 studies surveyed in detail are likely to be a biased sample. Reports in English are easier to find than similar work in other languages (Nieminen & Isohanni, 1999). Work with statistically significant results tends to be more accessible than trials with equivocal findings (Egger et al, 1997). It seems unlikely, however, that a significant body of higher-quality, larger studies has gone unnoticed. Reliability of coding of the variables used in this report is high, so results should reflect the subpopulation of studies surveyed.
The overall quality of reporting was mediocre. This is also the case in other branches of psychiatry (Thornley & Adams, 1998) and medicine (Gotzsche, 1989; Vanderkerckhove et al, 1993; Fahey et al, 1995; Schulz et al, 1995a; Cheng et al, 2000). This poor quality of reporting is likely to be associated with exaggerated estimates of effect (Schulz et al, 1995b). It is hoped that with CONSORT (Moher et al, 2001), the quality of trial reporting should improve.
People in the trials prioritised for this study commonly had psychosis or personality disorder and exhibited extremely aggressive behaviour. The range of interventions that have been trialled is bewildering, but few studies focus on similar interventions for similar participants. Pioneers have undertaken these important and often ground-breaking studies, but there is little evidence of collaboration between individuals or institutions to rationalise interventions and increase the power of their evaluative studies. Most studies are grossly underpowered for clinically relevant outcomes. Without widespread collaboration this is likely to remain the case.
One in three schizophrenia trials contain a new outcome rating scale (Thornley & Adams, 1998). More than a third of these scales are not validated and produce biased estimates of effect (Marshall et al, 2000). The 300 high-priority studies in this survey contain 1.2 new scales per report. The proportion not validated is likely to be high. Considering the limited clinical usefulness of much scale-derived data, this seems a remarkable waste of resources in a sub-specialty in which concrete and relevant outcomes may be more plentiful than in general psychiatry.
All trials identified by the project were made available within the Cochrane Controlled Trials Register and also offered to the Campbell Collaboration to build on their SPECTR database of trials. It is hoped that this database will allow people in a range of disciplines to have ready access to trial-based information relevant to offenders and potential offenders, and to learn from past practice in order to inform future work.
This broad overview suggests that wider collaboration, rationalising treatments and simplifying outcomes could further strengthen the tradition of trialling in forensic psychiatry. Systematic reviews of these studies are overdue.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Chalmers, T. C., Celano, P., Sacks, H. S., et al (1983) Bias in treatment assignment in controlled clinical trials. New England Journal of Medicine, 309, 1358 -1361.[Abstract]
Cheng, K., Smyth, R. L., Motley, J., et al (2000) Randomized controlled trials in cystic fibrosis (1966-1997) categorized by time, design, and intervention. Pediatric Pulmonology, 29, 1-7.[CrossRef][Medline]
Davies, P. & Boruch, R. (2001) The Campbell
Collaboration. Does for public policy what Cochrane does for health.
BMJ, 323, 294
-295.
Department of Health (2001) Draft Mental Health Bill. London: Stationery Office.
Dickersin, K. (1988) Report from the Panel on the Case for Registers of Clinical Trials at the Eighth Annual Meeting of the Society for Clinical Trials. Controlled Clinical Trials, 9, 76 -81.[Medline]
Egger, M., Zellweger-Zahner, T., Schneider, M., et al (1997) Language bias in randomised controlled trials published in English and German. Lancet, 350, 326 -329.[CrossRef][Medline]
Fahey, T., Hyde, C., Milne, R., et al (1995) The type and quality of randomized controlled trials (RCTs) published in UK public health journals. Journal of Public Health Medicine, 17, 469 -474.[Abstract]
Farrington, D. P. & Petrosino, A. (2001)
The Campbell Collaboration Crime and Justice Group. Annals of the
American Academy of Political and Social Science,
578, 35-49.
Gotzsche, P. C. (1989) Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal anti inflammatory drugs in rheumatoid arthritis. Controlled Clinical Trials, 10, 31 -56.[CrossRef][Medline]
Hetherington, J., Dickersin, K., Chalmers, I., et al (1989) Retrospective and prospective identification of unpublished controlled trials: lessons from a survey of obstetricians and pediatricians. Pediatrics, 84, 374 -380.[Abstract]
Jadad, A. R., Moore, R. A., Carroll, D., et al (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled Clinical Trials, 17, 1 -12.[Medline]
Juni, P., Altman, D. G. & Egger, M. (2001)
Assessing the quality of controlled clinical trials.
BMJ, 323, 42
-46.
Marshall, M., Lockwood, A., Bradley, C., et al
(2000) Unpublished rating scales: a major source of bias in
randomised controlled trials of treatments for schizophrenia.
British Journal of Psychiatry,
176, 249
-252.
Moher, D., Pham, B., Jones, A., et al (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet, 352, 609 -613.[CrossRef][Medline]
Moher, D., Schulz, K. F. & Altman, D. G. (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet, 357, 1191 -1194.[CrossRef][Medline]
Nieminen, P. & Isohanni, M. (1999) Bias against European journals in medical publication databases (letter). Lancet, 353, 1592 .[CrossRef][Medline]
Petrosino, A., Boruch, R. F., Rounding, C., et al (2000) The Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2-SPECTR) to facilitate the preparation and maintenance of systematic reviews of social and educational interventions. Evaluation and Research in Education, 14, 206 -219.
Schulz, K. F., Chalmers, I., Altman, D. G., et al (1995a) The methodologic quality of randomization as assessed from reports of trials in specialist and general medical journals. Online Journal of Current Clinical Trials, doc. 197.
Schulz, K. F., Chalmers, I., Hayes, R. J., et al (1995b) Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA, 273, 4408 -412.
Stern, J. M. & Simes, R. J. (1997)
Publication bias: evidence of delayed publication in a cohort study of
clinical research projects. BMJ,
315, 640
-645.
Taylor, P. J. & Gunn, J. (1999) Homicides by people with mental illness: myth and reality. British Journal of Psychiatry, 174, 9 -14.[Abstract]
Thornley, B. & Adams, C. (1998) Content and
quality of 2000 controlled trials in schizophrenia over 50 years.
BMJ, 317, 1181
-1184.
Vandekerckhove, P., ODonovan, P. A., Lilford, R. J., et al (1993) Infertility treatment: from cookery to science. The epidemiology of randomised controlled trials. British Journal of Obstetrics and Gynaecology, 100, 1005 -1036.[Medline]
World Health Organization Scientific Group on Treatment of Psychiatric Disorders (1991) Evaluation of Methods for the Treatment of Mental Disorders. Geneva: WHO.
Received for publication November 13, 2003. Accepted for publication October 26, 2004.
HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Psychiatric Bulletin | Advances in Psychiatric Treatment | All RCPsych Journals |