Leeds Addiction Unit, 19 Springfield Mount, Leeds, LS2 9NG, UK, 1 Department of Health Sciences, University of York, UK, 2 Centre for Health Economics, University of York, UK, 3 Substance Misuse Services, Birmingham and Solihull Mental Health NHS Trust, UK, 4 Clinical Trials Research Unit, University of Leeds, UK, 5 Alcohol Education and Research Council, London, UK, 6 Leeds North East Primary Care Trust, UK, 7 University of Birmingham, UK, 8 Institute for Medical and Social Care Research, University of Wales, Bangor, UK and 9 North Wales Section of Psychological Medicine, University of Wales, Bangor, UK
* Author to whom correspondence should be addressed at: Leeds Addiction Unit, 19 Springfield Mount, Leeds, LS2 9NG. Tel.: +0113 295 1335; Fax: +0113 295 2789; Email: gillian.tober{at}nhs.net
(Received 8 February 2005; first review notified 8 March 2005; in revised form 8 June 2005; accepted 9 June 2005)
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The aim of UKATT was to enhance the evidence for the practical treatment of alcohol problems in the UK (UKATT Research Team 2001; UKATT Research Team 2005a
, b
). A multi-site trial funded by the Medical Research Council compared the effectiveness and cost-effectiveness of two psycho-social treatmentsSocial Behaviour and Network Therapy (SBNT), described by Copello et al. (2002)
, and Motivational Enhancement Therapy (MET), adapted from Miller et al. (1995)
. To compare two therapies in this way, one needs to ensure that treatment complies with explicit protocols. Training and supervision were, therefore, delivered and monitored centrally.
Training of therapists and delivery of treatment in the UKATT were based upon the technology model of psychotherapy research (Docherty, 1984; Carroll and Nuro, 1996
), which was used in Project MATCH. This model requires specification of the treatment in the form of a manual and observation of the delivery of the treatment through video recordings of sessions, to ensure its consistent delivery (Carroll et al., 1994
). The practice of recording treatments serves the needs of training and supervision in treatment manual adherence. Trainees are better able to develop their skills by direct observation of their own practice, while teaching is enhanced by feedback from external observation of practice.
Standardisation of training and supervision in the delivery and content of treatment is a particular challenge in trials where clinical sites are geographically dispersed. Video recorded practice facilitates the provision of distance training and supervision where both supervisor and supervisee can observe practice simultaneously and discuss it by telephone. Thus the number of individuals teaching and supervising therapy is reduced, inter-supervisor consistency is enhanced, and standardisation of practice is easier to achieve and maintain. This method should be initiated during the training phase of the trial, in order to establish a similar skill level across the different professional groups and treatment sites (Carroll et al., 1994).
To ensure that UKATT informed routine clinical practice, the design was pragmatic and the context was routine practice. In particular, trial treatments were delivered by existing staff in typical UK alcohol treatment agencies, in contrast to Project MATCH, which hired therapists experienced in a single therapy to deliver that therapy in the trial. Existing staff are heterogeneous, with a range of educational qualifications and experience of the field. To minimise the effects of therapist characteristics, therapists were allocated at random between the two treatments, which created new challenges for training and maintaining high standards of practice, as therapists could not choose the treatment closest to their current practice or their preferred theoretical approach. If successful, this approach would demonstrate that therapists could be trained in evidence-based approaches across varied professional backgrounds, levels of educational achievement, and work settings.
Standards and methods for the recruitment, training, supervision, and assessment of therapists were, therefore, specified at the outset. Pre-training therapist data form the basis for examining the extent to which these therapist variables influenced the amount or outcome of training. Data on the number of training cases, the number of supervision sessions needed to achieve competence, and the duration of training form the basis for calculating the effort required to complete the task, and thus the cost of training and treatment. To inform future planning of training, we hypothesised that:
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Selection of clinical services
Services were selected for the trial if they could provide a suitable environment which included having sufficient clients seeking help for alcohol problems, enough space and time for pre-treatment assessments and treatment sessions which could include others in addition to the client, sufficient therapists to provide the two treatments, willingness of staff to receive supervision from outside the agency, and commitment to participating in a clinical trial. Therapist training needed time for initial training and supervision thereafter, and facilities for recording supervised practice. Five services, on seven distinct sites, fulfilled these requirements.
Selection of therapists for training
Once services had been selected for the trial, clinical staff were invited to express an interest in becoming trial therapists to the Principal Investigator (PI) for that service. Suitable staff should have fulfilled five criteria: two years work experience (relaxed as UKATT progressed and more therapists were needed), not currently training in another addiction therapy, willing to learn one of two manual based treatments allocated at random (UKATT Research Team 2001), willing to attend the trial training centre for three days of training, and willing to be supervised throughout the trial from the training centre.
Candidates were invited to submit a curriculum vitae and video recordings of two practice sessions with clients, one demonstrating motivational interviewing skills and the other demonstrating the ability to work simultaneously with the client and others. Submitting recordings of practice also confirmed the acceptability of doing so during the trial. Once selected using these criteria, therapists were allocated at random to one of the two trial treatments.
Initial training of therapists
Three days of group training, provided separately for each treatment with therapists from the different services, covered the background, purpose, personnel, general procedures of the trial, and the development of team spirit. Thereafter, it focused on the specific treatment. Particular attention was paid to the theoretical and evidential basis for the treatment, demonstration of practice, and role play. The PI for the training centre, whose main role was to maintain quality and consistency across treatments, acted as one of the trainers alongside the designated trial supervisor for each treatment and authors of the treatment manual. This PI oversaw each of the treatment-specific supervisors during training and throughout the trial.
Supervision of training clients
Following initial training, candidates returned to their workplace to practise with newly referred clients (UKATT Research Team 2001). Each practice session was video recorded, with the camera trained exclusively upon the therapist. Written consent for video recording was obtained from all training clients. One copy was retained by the trainee and another sent to the training centre. Supervision comprised simultaneous viewing and discussion of the tape by trainee and designated trial supervisor, expert in the allocated therapy, by telephone (or by face to face meeting in Leeds, where the clinical site was adjacent to the training centre). This supervision provided the main opportunity for practising the skills, structure, content, and delivery of the treatment, introduced during the three initial days of training.
Assessment of competence
We compiled checklists of therapist behaviours, session structure, and session content, adjudged by the authors of the manuals to be the essential components of each treatment. Examples from MET were Asks open questions and Uses reflective listening selectively. Examples from SBNT were "Describes the principles of network support" and "Explores the availability of network members". The competence of trainees was assessed against these checklists by the treatment-specific supervisor and the training PI on the basis of video recordings of practice, with a minimum of two training clients in MET or of one client (with or without a social network) in SBNT. If a trainee was not judged to be competent at this point, continuing supervision with additional training cases was required. Competence was then re-assessed. The training process was designed to continue for as long as it took to achieve competence. Once therapists were deemed competent they were allowed to practise in the trial.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
Differences between therapists who did and did not achieve competence
Of the 72 therapists who attended the initial training in Leeds, 52 (72%) achieved competence by fulfilling the specified criteria (Table 1). There were no significant differences in the sex or age of those who did or did not achieve competence. Table 4 shows differences between services in the proportion of therapists achieving competence, but the sample is small and the differences may reflect chance variation. Also shown are the completion rates for each of the treatments, where no difference was found. Table 5 shows the duration of field experience, the educational level, and the professional grouping for those who did and did not complete training. Though no significant differences were found in duration of field experience between those who did and did not complete training, educational level, and professional group had an effect on training outcome. When educational qualifications were grouped by having a first degree and higher or not, those in the higher educational grouping were more likely successfully to complete training. Differences between professional groups in completion rates were marginally significant: all 12 medical practitioners completed training, compared with 22 of 35 nurses (63%) and 18 of 25 (72%) other professions.
|
|
Analysis of variance revealed a statistically significant difference in duration of training between the five services (F = 4.42; df = 4; P = 0.004). The service adjacent to the training centre had a significantly shorter duration of training compared with the other sites.
To check whether the differences in duration of training may have been influenced by rates of attendance for supervision, the duration of training was replaced by the number of supervision sessions required to achieve competence. No significant differences between services were found, and this was confirmed by analysis of variance (P = 0.3). Differences in duration of training may merely reflect a greater availability of training clients in service 5, or the fact that it was adjacent to the training centre.
Differences by treatment type
In addition to the duration of training in days and the number of supervision sessions attended during training, the number of training case clients seen was calculated to examine whether this provided a different dimension of the training resource required. Table 6 shows that MET took significantly longer in training, when measured by the number of both days and training clients required for the achievement of competence. When measured by the number of supervision sessions attended, however, the difference between treatment types was not significant.
|
|
The cost of therapists' time spent in training is the largest component of the training costs for each therapy (Table 8). The cost per day for each therapist was calculated from the individual salary grade inclusive of estimated employers' costs for pensions and national insurance. The calculated cost per day was multiplied by three for the days engaged in the first training episode. The same method was used to cost follow-up training when undertaken.
|
The total costs of training can be expressed in different ways. Calculating averages per person trained yields estimates of £895 per MET therapist trained (31 in all), and £1020 per SBNT therapist trained (41 in all). As a different proportion of each group gained competence, however, dividing the cost of incompetent therapists across all the competent therapists yields estimates of £1260 per competent MET therapist and £1390 per competent SBNT therapist.
Training is an investment expected to deliver benefits over a period of time. However therapists in UKATT had limited opportunities to deliver the treatments, depending on the successful recruitment of clients into the trial and randomisation to their therapy. The training cost per session delivered in UKATT is £35 and £38 for MET and SBNT respectively. However, we use a more realistic assumption to estimate the cost of training therapists for routine practice. Assuming that a competent therapist can deliver 736 sessions per year (or 46 weeks multiplied by 16 sessions per week) and that the effects of training would endure for a year, training costs would amount to £1.71 for MET and £1.89 for SBNT.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In principle, trial staff responsible for supporting therapists during training followed standard procedures. Two factors may have caused variation in this support, accounting for different rates of acquiring competence across services. First, one clinical site was alongside the trial training centre, yielding easier access to clients and allowing supervision face to face. Second, some trial services included multiple sites. Care was, therefore, taken to standardise training for both treatments.
Nevertheless, we found significant differences between MET and SBNT in the number of clients and duration of training needed to acquire competence, though not in number of supervision sessions. Of these we expected significant differences in the number of clients seen, because SBNT therapists have more opportunity to demonstrate competence across their planned eight sessions with the same client than MET therapists across their planned three sessions per client. It was this difference that led us to propose that SBNT needed one client for competence while MET needed two. That this also led to a difference in the duration of training may be explained by differences in the speed with which UKATT recruited clients for each treatment (UKATT Research Team 2001). Fortunately similar numbers of supervision sessions, and thus similar costs of training in both groups, provide reassurance that UKATT was indeed comparing like with like.
A major challenge for training was to develop confidence in the treatment to which therapists had been randomised, so we highlighted the evidence in favour of that treatment and enabled therapists to practise on their return to their workplaces. In some sites video recording was a novelty, perhaps contributing to differences in the time to acquire competence. The provision of accessible equipment was important in establishing the acceptability of that procedure.
Raw training costs differed between treatments. The finding that training in SBNT was costlier was not attributable to greater duration of training, but to the more expensive initial trainer costs. This was the result of the decision to include as SBNT trainers all those who developed the SBNT treatment, which is unlikely to be a model for future training, though it is not possible to say whether the same quality of training could be provided by fewer trainers. Fortunately our main calculation, based on routine practice and the assumption that the effects of training would endure for a year (even though annual training is unlikely) showed that training cost little for either treatment.
![]() |
CONCLUSIONS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Carroll, K. M., Kadden, R. M., Donovan, D. M. et al. (1994) Implementing treatment and protecting the validity of the independent variable in treatment matching studies. Journal of Studies on Alcohol 12, 149155.
Cartwright, A. (1980) The attitudes of helping agents towards the alcoholic client: the influence of experience, support, training and self esteem. British Journal of Addiction 75, 413431.[ISI]
Copello, A., Orford, J., Hodgson, R. et al. (2002) Social behaviour and network therapy: basic principles and early experiences. Addictive Behaviors 27, 345366.[CrossRef][ISI][Medline]
Docherty, J. P. (1984) Implications of the technological model of psychotherapy. In Psychotherapy Research: Where Are We and Where should We Go? Williams, J. B. W. and Spitzer, R. L. eds, pp. 139149, Guilford Press, New York.
Lightfoot, P. J. C. and Orford, J. (1986) Helping agents' attitudes towards alcohol-related problems: situations vacant? A test and elaboration of a model. British Journal of Addiction 81, 749756.[ISI][Medline]
Miller, W. R., Zweben, A., DiClemente, C. C. and Rychtarik, R. G. (1995) Motivational enhancement therapy manual: a clinical guide for therapists treating individuals with alcohol abuse and dependence. NIAAA Project MATCH monograph series, Volume 2, Rockville, USA.
Netten, A., Rees, T. and Harrison, G. (2001) Unit Costs of Health and Social Care 2001. University of Kent at Canterbury Personal Social Services Research Unit, United Kingdom.
UKATT Research Team (2001) The United Kingdom Alcohol Treatment Trial (UKATT): hypotheses, design and methods. Alcohol and Alcoholism 36, 1121.
UKATT Research Team (2005a) Effectiveness of treatment for alcohol problems: findings of the randomised UK Alcohol Treatment Trial. British Medical Journal 331, in press.
UKATT Research Team (2005b) Cost-effectiveness of treatment for alcohol problems: findings of the randomised UK Alcohol Treatment Trial. British Medical Journal 331, in press.
|