Department of Psychological Medicine, Charing Cross Campus, Imperial College School of Medicine, London, UK
Correspondence: Professor Isaac M. Marks, 43 Dulwich Common, London SE21 7EU,UK. E-mail: i.marks{at}iop.kcl.ac.uk
Declaration of interest I.M.M. shares intellectual property rights in the FearFighter,Cope and BTSteps systems, and S.H. has an interest in FearFighter.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims Open evaluation of a free clinic giving immediate computer-aided cognitivebehavioural therapy (CBT) self-help plus brief advice from a therapist.
Method Test of outcome of self-referrals who used one of four computer-aided CBT systems for depression, phobia/panic, general anxiety or obsessivecompulsive disorder.
Results The equivalent of one full-time clinician managed 355 referrals over a year. Of the 266 who had a screening interview 79% were suitable. Completers and non-completers of computer-aided CBT had similar pre-treatment features, with very chronic, moderately severe problems. Completers of the computer-aided self-help had a mean total of an hours live therapist support over 12 weeks. They improved significantly and clinically meaningfully with three of the four systems and felt fairly satisfied. Improvement resembled that in controlled and other trials of computer-aided CBT.
Conclusions Computer-aided self-help is a clinician extenderthat greatly cuts per-patient therapist time without impairing improvement. It could reduce the per-patient cost of CBT.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The clinic accepted subjects who sent a completed screening questionnaire obtained with a pre-addressed envelope from a pack that the clinic sent to GP surgeries, other clinics and enquirers who phoned. Suitability criteria were: presence of an anxiety or depressive disorder; motivation to use self-help; and no substance misuse, psychosis or active suicidal plans. From the screening questionnaire staff judged referrals likely suitability for computer-aided CBT and offered them a 30-min screening interview by telephone or face to face at the clinic. Staff sent unsuitable patients a letter to this effect and copied it to their GP (unless they asked the clinic not to contact their GP). Diagnoses were obtained using a checklist summarising relevant ICD10 (World Health Organization, 1992) diagnostic criteria.
How clinic patients used computer-aided CBT
Patients who were suitable at the screening interview were given an
identification number allowing access to whichever of the clinics four
computer self-help systems most suited them: FearFighter for
phobia/panic (Kenwright et al,
2001; Marks et al,
2003); Cope for depression/anxiety
(Osgood-Hynes et al,
1998); Balance for general anxiety/depression
(Yates, 1996); and
BTSteps for obsessivecompulsive disorder
(Greist et al, 2002).
Patients knew that the information they gave their system was confidential to
staff and could not be accessed without knowing the patients
identification number and password (many said they told the computer sensitive
things they would not tell a human). Their system stored no personal
identifiers such as names or addresses.
Patients were told that they could use their system as much as they wished. They were advised to use FearFighter (for phobia/panic), Cope (for depression/anxiety) or BTSteps (for obsessivecompulsive disorder) at least six times over 12 weeks. During office hours they had six brief scheduled therapist contacts by telephone or face to face for advice. Users of Balance (for general anxiety/depression) were asked to use it at least three times over 4 weeks (Balance is more basic than the three other systems) and to have three brief therapist contacts by telephone or face to face during that period.
Patients accessed FearFighter on a stand-alone personal computer (PC), mostly at the clinic or, rarely, on a stand-alone PC made available free at one internet café and one medical centre. When FearFighter became available on the internet some accessed it on a PC at home or elsewhere which was linked to the internet.
The clinic gave users of Cope and BTSteps, along with their identification number, self-help booklets to guide their free phone calls made from home to either Copes or BTStepss interactive voice response (IVR) system in a computer in Wisconsin, USA. Users could phone that computer from home around the clock for as long and as often as desired, and carried out their interviews by key presses on their telephone keypad. The computer faxed to the clinic weekly reports of patients telephone calls, their duration, the modules accessed and (for Cope patients) suicide risk had this become high, which never happened, this would have been immediately faxed or phoned to the clinic.
Balance users accessed the system by a PC with a CDROM drive at home or at the clinic, at a GP surgery or, rarely, at a free internet café.
Clinic staff
The clinic was mainly run by two nurse therapists (L.G., M.K.) who, between
them, totalled only one whole-time-equivalent clinician (because they spent
much time on research, teaching and publicising the clinics service),
and an administrator/research assistant (R.C.). The research psychologist
(D.M.-C.) had a mainly research rather than clinical role. The clinic was
co-directed by two consultant psychiatrists (I.M.M., S.H.).
Outcome measures
Self-ratings were collected at pre- and post-treatment unless otherwise
stated, and lower scores denoted normality/improvement:
Data analysis
The samples features are described. Univariate analyses of variance
and chisquare tests compared patients who completed treatment with those who
were unsuitable, refused or dropped out early. Paired sample t-tests
were used for outcome analyses. Significance was set at 0.05 and all tests
were two-tailed.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Of the 355 patients, 28 (8%) were unsuitable on the screening questionnaire (Fig. 1), the main reasons being: suicide risk (6), wanting face-to-face therapy (5), poor motivation (2), other diagnosis (7) and other reasons (8). The rest (327 referrals) were offered a screening interview with a clinician, of whom 61 (19%) did not attend and 266 (81%) did.
|
Of the 266 who had a screening interview (Fig. 1), 210 (79%) were suitable for and offered computer-aided CBT and 56 (21%) were unsuitable. The main reasons for unsuitability were: no primary anxiety or depression (18), unsuited for CBT (13), work/social problems (8), poor motivation (6), suicide risk (5), living outisde the free catchment areas of the West London Mental Health Trust and the Hillingdon Primary Care Trust and without funds from an alternative NHS or private source (3), language problem (2) and wanting face-to-face therapy (1).
Refusal and dropping out
Of the 210 patients suitable at the screening interview, 42 (20%) refused
computer-aided CBT and 60 (29%) dropped out early or gave no post-treatment
data. Of the 39 patients who gave information, the main reasons for refusing
or not completing the therapy included: hard to attend clinic/could not commit
(13), the therapy was unhelpful (10), wanting face-to-face help (8), low
motivation (8), offered help elsewhere (2) and problem improved (2). Those who
were unsuitable, refused, dropped out and completed did not differ on initial
severity.
Features of the referrals
Table 1 shows the
pre-treatment features of all referrals, of those unsuitable at the screening
interview, of suitable patients who refused or dropped out early and of those
who completed the computer-aided CBT for all available variables: gender, age,
socio-economic status, referral source, problem duration, medication,
diagnosis and initial Single-Item Depression Scale score. There were no
significant differences between these groups for any of the variables except
diagnosis (see below). The greater initial depression of those who dropped out
compared with that of those who completed the therapy was not significant on a
Bonferroni correction for multiple tests. Some data are lacking for referrals
who were unsuitable on the screening questionnaire and so either had no
screening interview or did not attend it.
|
Of all 355 referrals, 191 (54%) were women and 121 (34%) were unemployed or students the clinic was in a deprived area. Of the 139 patients who gave this information, 76 (54%) had a current partner and 52 (39%) had one or more children. At least 81% had completed basic education and 50% had a higher or further educational qualification.
Initial severity
The sample was very chronic with moderately severe problems. The mean
problem duration was 8 years (s.d.=10) but it was over 10 years in about
one-third of referrals; the mean score on the 08 Single-Item Depression
Scale (McKenzie & Marks,
1999; 0=hardly troubled by depression at all; 8=very severely
disturbed/disabled) was 4.9 (s.d.=2), denoting moderate to severe depression.
Work and social adjustment was moderate: a mean of 21 (s.d.=10) on the
040 WSA scale. Of 138 patients who gave this information, 54 (39%) had
given up work or were on long-term sick leave because of their problem.
Current/past treatment
Of the 139 patients who gave data, about half (45%) were having current
treatment from their GP or a mental health professional and about half were on
psychotropic medication (Table
1). The vast majority (96%) had had past treatment for their
problem; this had been CBT in only 20%.
Diagnoses
At the screening interview of 266 referrals, an experienced clinician made
a primary and, if needed, secondary ICD10 diagnosis
(Table 1).
Primary diagnoses were (numbers of patients): 71, depression (40, recurrent depressive disorder; 22, depressive episode; 8, dysthymia; 1, other mood disorder); 57, phobia (3, agoraphobia; 16, agoraphobia with panic disorder; 17, social phobia; 21, specific phobia); 35, generalised anxiety disorder; 35, obsessivecompulsive disorder; 26, reaction to severe stress/adjustment disorder; 6, mixed anxiety/depressive disorder; 6, somatoform disorder; 3, panic disorder; 27, other.
Secondary diagnoses were made in 118 cases (44%): 26, depression (14, recurrent depressive disorder; 10, depressive episode; 2, dysthymia); 24, phobia (3, agoraphobia; 4, agoraphobia with panic disorder; 14, social phobia; 3, specific phobia); 37, generalised anxiety disorder; 8, obsessivecompulsive disorder; 7, somatoform disorder; 5, panic disorder; 5, reaction to severe stress/adjustment disorder; 2, mixed anxiety/depression; 4, other.
Compared with the suitable patients (Table 1), the 56 who were unsuitable at the screening interview had, as expected, significantly fewer primary diagnoses of depression, phobia, generalised anxiety disorder or obsessivecompulsive disorder and significantly more of the other primary diagnoses.
Computer literacy
Of the 135 patients for whom this information was available, 47 (35%) used
computers most days at work, 16 (12%) quite often, 19 (14%)
often, 41 (30%) occasionally and 12 (9%) had never
used computers before. Computer literacy did not differ between those in the
unsuitable, refused, dropped out and completed categories or between
FearFighter, Cope, Balance and BTSteps users, and did not
correlate with outcome or satisfaction.
Clinical outcome
For all patients
For the 108 patients who had post-treatment data, clinical outcome on
generic outcome measures appears in Table
2. Having been moderately disabled on the WSA at pre-treatment,
post-treatment treatment the patients had improved significantly on the total
WSA and each of its five items.
|
On the PGI scale at post-treatment (Table 3 and Fig. 2), 80% of patients rated themselves as better to some degree, 10% as unchanged and 9% as worse to some degree.
|
|
For users of each computer-aided CBTsystem
Separately per system, improvement on the WSA total score was significant
for FearFighter, Cope and Balance users. Completers of each
system also improved significantly from pre- to post-treatment on measures
specific to their problem (Table
2). The clinically meaningful effect size of 0.8 or more was
exceeded by FearFighter users on the Fear Questionnaires
global phobia and anxiety/depression scores, by Cope users on the BDI
and the WSA and by BTSteps users on the YBOCS total score and the
obsessions and compulsions sub-scores. Balance users did not attain
this clinically meaningful effect size on any measure.
Patient satisfaction at post-treatment
For the 70 patients for whom full data were available, the mean ratings for
the four questions of the satisfaction scale (each rated 08: 0=very
good; 4=neutral; 8=very poor) were: technical aspects of their system, good to
moderate (mean=3.1, s.d.=1.5); content and structure of their system, good to
moderate (mean=2.7, s.d.=1.4); live support from a clinician, very good to
good (mean=1.6, s.d.=1.5); clinic as a whole, good (mean=2; s.d.=1.5).
Patients were thus fairly satisfied with their computer-aided CBT system and even more satisfied with their live support and the clinic as a whole. In keeping with this, on how much of their treatment they would have preferred to have been guided by a therapist and how much by a self-help computer system (08 scale: 0=100% computer, 0% clinician; 4=50% computer, 50% clinician; 8=0% computer, 100% clinician) the mean rating was 4.9 (s.d.=2.2), suggesting a marginal preference for therapist over computer guidance. There were no significant differences in satisfaction or preference (therapist v. computer) between users of the different systems (data not shown).
Features of use of computer-aided CBT
A mean of 58 days (s.d.=49) elapsed from patients starting to ending
the computer-aided CBT. Over that period they had a mean of 64 (s.d.=48)
minutes of support from a clinician, of which 25 min (s.d.=23) were spent on
extra treatment advice, 17 min (s.d.=15) on progress review, 16 min (s.d.=15)
on general support, and only 6 min (s.d.=7) on technical support.
Table 4 shows patients use of each of the four computer-aided systems. Duration of use of Balance (the shortest and least interactive of the four) tended to be the shortest (P=0.1).
|
FearFighter patients had more general therapist support (P<0.001) than Cope and Balance patients and more extra treatment advice and technical support (both P<0.001) than users of the three other systems. Patients who used FearFighter at the clinic (n=17) needed significantly less technical support time than those who used it on the internet (n=10) in the first pilot test of the internet version (mean difference =6.5; 95% CI 12.3 to 0.7; t=2.3, d.f.=24, P=0.03).
About half of the patients had live support by telephone and half had support face to face at the clinic. The live support by telephone was given to users of the Cope and BTSteps IVR systems, of FearFighter on the internet and of Balance on a CDROM.
Patients known total time spent interacting with the computer were: FearFighter on a stand-alone PC (n=17), 266 min (s.d.=71) over a mean of five sessions (s.d.=1, range 27); Cope (n=39), 122 min (s.d.=83) spent on IVR phone calls over a mean of 11 calls (s.d.=8, range 034); Balance (n=23) for patients using it at the clinic, 100 min (s.d.=72) over two sessions (s.d.=2, range 09); BTSteps (n=9), 235 min (s.d.=151) spent on IVR phone calls over a mean of 32 calls (s.d.=26, range 569).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The clinics greater throughput of patients per therapist with the help of computer-aided CBT and shorter patient time per clinician did not appear to sacrifice effectiveness. Individuals with anxiety and depression improved significantly and clinically meaningfully, and were fairly satisfied with computer-aided CBT despite a marginal preference for face-to-face care. When GPs nearby and a secondary CBT service recommended the clinic to many patients, this reduced the GPs referrals to secondary mental health services and slightly shortened the waiting list for face-to-face CBT in secondary care (Kaltenthaler et al, 2002).
Patients who accessed the computer by telephone spent very similar total times calling the computer as in previous studies (Osgood-Hynes et al, 1998; Kenwright et al, 2001; Greist et al, 2002): 2 h on Cope calls and 4 h on BTSteps calls.
Some of the outcomes in the present study can be compared with those in past studies of computer-aided CBT.
For FearFighter completers, pre- to post-FearFighter improvement and effect size (ES) on the Fear Questionnaire Global Phobia item were 41% (ES=1.4) of our sample compared with 54% (ES=1.3) in Kenwright et al (2001) and 38% (ES=1.7) in Marks et al (2003). Our FearFighter users figures of 41% improved (ES=1.4) compared with figures for face-to-face CBT of 37% (ES=1.2) in Kenwright et al (2001) and 51% (ES=2.8) in Marks et al (2003).
For Cope completers, pre- to post-Cope improvement on the Patient Global Impression of Improvement scale was a mean of 2.4 in our patients compared with 3.1 in Osgood-Hynes et al (1998), after correcting the latters 17 range to our 06 range.
For BTSteps completers, pre- to post-improvement on the YBOCS was 40% (ES=1.2) in our sample compared with 23% (ES=1.3) in Greist et al (2002), and compared with 32% (ES=1.7) in patients who had had face-to-face CBT in Greist et al (2002).
Thus, the patients who completed computer-aided CBT in our study improved broadly comparably to completers in other studies which used the same systems and measures. Only in the RCT of Marks et al (2003) did face-to-face care yield an appreciably better effect size than that of the computer-aided care in the present study.
Rates of non-attendance, unsuitability and non-completion
Of the self-referrals who were offered a screening interview, 19% did not
attend. This is well below the 48% non-attendance rate for brief psychiatric
screening of less-severe mental health problems
(Hamilton et al,
2002) and an out-patient anxiety disorders clinic offering CBT
(Kenwright & Marks, 2003). Although we did not specifically measure this, we had the impression that
rates of non-attendance were higher for patients already on a waiting-list for
face-to-face CBT in secondary care.
The 79% suitability rate at screening interview was satisfactory. The combined rate of 20% who refused plus 29% of patients who began computer-aided CBT but gave no post-treatment data was higher than the equivalent for outpatients having CBT for anxiety disorders (Marks et al, 1995). Non-completers had resembled completers on initial severity and other variables.
There is no ideal way to analyse patients who began computer-aided CBT but gave no post-treatment data (Everitt, 1998). It can be misleading to carry forward their baseline data on the assumption that outcome stayed frozen thereafter (Everitt, 1994). Assuming that none improved would ignore those who gave no data but said that after dropping out they continued the self-help they had learned and benefited from. Many non-completers did not answer repeated phone and postal requests for post-baseline ratings. Patients undergoing self-help at home with access to a live helpline had no incentive to attend the clinic or give ratings.
Cost-effectiveness
A rough cost comparison of computer-aided CBT with purely face-to-face CBT
assumed the same throughput of patients managed per therapist using the
computer-aided therapy as in the clinic, a £61/h cost of a CBT therapist
(Netten & Curtis, 2000)
and the licence costs of computer-aided CBT used by the National Institute for
Clinical Excellence (2002).
Assuming administrative costs similar to those of the clinic and 15%
overheads, the estimated per-patient cost advantage of computer-aided CBT over
face-to-face CBT would rise from about 15% per patient for 350 patients a year
to 41% per patient for 1350 patients per year. This advantage rises with
volume savings as the number of patients rises, and discounts any value from
computer-aided CBT at home giving patients immediate rather than delayed
access to CBT, unrestricted access, easier disclosure of sensitive
information, and removal of the need to travel to a therapist. This rough
estimate of cost-effectiveness needs to be checked in a more formal study.
In contrast to its lower per-patient cost, the total cost of computer-aided CBT nationally might rise if previously untreated users sought the therapy in such numbers as to offset savings from lower per-patient costs. Sufficiently widespread dissemination of computer-aided CBT might eventually reduce demands on GP and secondary services and lessen medication use and chronicity.
Wider implications
Computer-aided CBT is developing rapidly. Patients can now help themselves
entirely at home by accessing two of the four computer-aided CBT systems used
by the clinic on the internet (patients without home internet access can use a
link in public libraries, computer learning centres, internet cafés,
etc.) and the other two by telephone. Those who get stuck during self-help can
get support from a clinician on a live helpline.
As referrals can now be screened for suitability to engage in computer-aided CBT and supported by clinicians entirely by telephone while doing computer-aided CBT at home, self-help clinics could act as call centres for wide catchment areas. How many such centres would be needed to cover an entire country might be inferred from the experience of NHS Direct call centres.
Computer-aided CBT systems are clinician extenders, not clinician replacers. Because patients using these systems need screening and brief advice, training is needed to equip clinicians to work in this area. The training might take only 2 days per system and so be inexpensive. Trainees can work through each system systematically, posing as patients who have relevant problems (McDonough & Marks, 2002). The lack of such trained personnel is a barrier to the dissemination of computer-aided CBT.
The model suggested is stepped care, with computer-aided CBT self-help plus brief live helpline advice as a potential first port of call for most individuals with anxiety/depression. Those who fail to improve sufficiently with computer-aided CBT could go on to have entirely live clinician-guided help by telephone or face to face.
Some might benefit from posted self-help instructions (Burgess et al, 1998) or self-help books, perhaps with access to a live helpline. Books may cost less than computer-aided CBT but they are less interactive and harder to modify and to disseminate on a large scale. It is also hard to track the progress of patients who are using books, whereas computer-aided CBT on the internet or a central IVR computer eases the audit of outcome on a mass scale.
Mental health inequalities might lessen with widespread access to computer-aided CBT self-help, given that almost two-thirds of the clinics referrals came from the three lowest socio-economic groups and that initial computer illiteracy did not affect outcome.
Although the present study found that computer-aided CBT self-help at a single clinic supporting patients from a wide area was effective and acceptable, other models for implementation of the technology may work as well. Supporting patients on self-help systems full time may bore highly trained therapists (the clinic therapists spent only half of their time in patient contact). Computer-aided CBT could be integrated into existing therapist services, with clinicians perhaps dedicating 1 day a week to supporting patients doing computer-aided CBT who might otherwise wait months or years to get CBT in their service. A major obstacle to dissemination is the current reluctance of health service managers to fund computer-aided CBT. The clinics time-limited funding came to an end. It can take a long time for socio-political processes to be worked through to make new technologies a routine aspect of health services.
Limitations
Although an open pragmatic evaluation like the present project may tell
more about implementation issues than does an RCT, it cannot say how much of
the patients self-rated improvement was due to the passage of time,
contact with a service, CBT, computer-aided CBT or the brief help given by a
clinician, nor is it known whether similar benefit might have accrued from
offering an appropriate CBT self-help book plus access to a helpline. Also
unclear is the potential effect of psychotropic medication, which about half
of the patients were on at the start and had often been on for a long time;
this is inevitable in any regular service.
Our study could not tell what proportion of all sufferers in the community might want computer-aided CBT as their first option in a stepped-care service instead of an immediate or a delayed interview with a live therapist by telephone or face to face. Nor could our study say whether costs might drop still further without impairing effectiveness if the brief advice given to computer-aided CBT users was from less-highly trained supporters (e.g. the new primary care health workers coming on stream) and if most screening was done via a questionnaire without a back-up live interview.
The amount of improvement should be regarded with some caution because of the high rate of refusal plus non-completion, even though completers and non-completers had been indistinguishable at the start.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Beck, A. T., Epstein, N., Brown, G., et al (1988) An inventory for measuring clinical anxiety: psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893897.[CrossRef][Medline]
Burgess, M., Gill, M. & Marks, I. M. (1998) Postal self-exposure treatment of recurrent nightmares: randomised controlled trial. British Journal of Psychiatry, 172, 257262.[Abstract]
Everitt, B. S. (1994) Statistical Methods for Medical Investigations (2nd edn). London: Edward Arnold.
Everitt, B. S. (1998) Analysis of longitudinal data. Beyond MANOVA. British Journal of Psychiatry, 172, 710.[Abstract]
Goodman, W. K., Price, L. H., Rasmussen, S. A., et al (1989) The YaleBrown Obsessive Compulsive Scale. I: Development, use and reliability. Archives of General Psychiatry, 46, 10061011.[Abstract]
Greist, J. H., Marks, I. M., Baer, L., et al (2002) Behaviour therapy for obsessive compulsive disorder guided by a computer or by a clinician compared with relaxation as a control. Journal of Clinical Psychiatry, 63, 138145.[Medline]
Guy, W. (1976) ECDEU Assessment Manual for Psychopharmacology, Revised. DHEW Publication No. (ADM) 76338.Washington, DC: US Government Printing Office.
Hamilton, M. (1960) A rating scale for depression. Journal of Neurology, Neurosurgery and Psychiatry, 23, 5661.
Hamilton, R., Gordon, P. & Naji, S. (2002)
Service innovation: the first year of a brief psychiatric screening clinic in
primary care. Psychiatric Bulletin,
26,
218221.
Kaltenthaler, E., Shackley, K., Beverley, C., et al (2002) Computerised Cognitive Behaviour Therapy for Depression and Anxiety: Final Assessment Report. London: NICE. http://www.nice.org.uk/pdf/ccbtassessmentreport
Kenwright, M. & Marks, I. M. (2003) Improving first attendance for screening in a psychiatric outpatient clinic: two randomised controlled trials. Journal of Mental Health, in press.
Kenwright, M., Liness, S. & Marks, I. M.
(2001) Reducing demands on clinicians by offering
computer-aided self-help for phobia/panic. Feasibility study.
British Journal of Psychiatry,
179,
456459.
Marks, I. M. (1985) Psychiatric Nurse Therapists in Primary Care: A Controlled Trial. London: Royal College of Nursing.
Marks, I. M. & Mathews, A. M. (1979) Brief standard self-rating for phobic patients. Behaviour Research and Therapy, 17, 263267.[CrossRef][Medline]
Marks, I. M., Blanes, T. & McKenzie, N. (1995) Computerised clinical benefitcost audit of mental health care. I: Theoretical and practical issues. Journal of Mental Health, 1, 6369.[CrossRef]
Marks, I. M., Kenwright, M., McDonough, M., et al (2003) Computer-guided self-help for panic/phobic disorder cut per-patient time with a clinician: A randomised controlled trial. Psychological medicine, in press.
McDonough, M. & Marks, I. M. (2002) Teaching medical students exposure therapy: randomised comparison of face-to-face versus computer instruction. Medical Education, 6, 16.[CrossRef]
McKenzie, N. & Marks, I. M. (1999) Quick rating of depressed mood in patients with anxiety disorders. British Journal of Psychiatry, 174, 266269.[Abstract]
Mundt, J. C., Marks, I. M., Shear, K., et al
(2002) The Work and Social Adjustment Scale: a simple measure
of impairment in functioning. British Journal of
Psychiatry, 180,
461464.
National Institute for Clinical Excellence (2002) Guidance on the Use of Computerised Cognitive Behavioural Therapy for Anxiety and Depression (Technology and Appraisal Guidance no. 51). London: NICE. http://www.nice.org.uk/pdf/51_CCBT_Full_guidance.pdf
Netten, A. & Curtis, L. (2000) Unit Cost of Health and Social Care 2000. Kent: Personal Social Services Research Unit, University of Kent. http://www.kent.ac.uk/PSSRU/UC2000.html
Osgood-Hynes, D. J., Greist, J. H., Marks, I. M., et al (1998) Self-administered psychotherapy for depression using a telephone-accessed computer system plus booklets: an open USUK study. Journal of Clinical Psychiatry, 58, 358365.
Proudfoot, J., Goldberg, D., Mann, A., et al (2003) Computerised, interactive, multimedia CBT reduced anxiety and depression in general practice: a RCT. Psychological Medicine, 33, 217227.[CrossRef][Medline]
World Health Organization (1992) Tenth Revision of the International Classification of Diseases and Related Health Problems (ICD10).Geneva: WHO.
Yates, F. (1996) Restoring the Balance (CDROM). Available from the Mental Health Foundation, 7th Floor, 83 Victoria Street, London SW1H 0HW (tel. 020 7802 0300).
Received for publication August 22, 2002. Revision received January 22, 2003. Accepted for publication February 17, 2003.