School of Psychiatry and Behavioural Sciences, University of Manchester
School of Psychiatry and Behavioural Sciences, University of Manchester
Health Care Trials Unit, School of Epidemiology & Health Sciences, University of Manchester
Lancashire Care Trust, Royal Preston Hospital, Preston, UK
Correspondence: Professor M. Marshall, Academic Unit, Lancashire Care Trust, Royal Preston Hospital, Fulwood, Preston PR2 9HT,UK
Declaration of interest None. Support from the Wellcome Trust.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To determine whether feedback from a standardised assessment of need enhances the effectiveness of care planning and whether exposing care coordinators to feedback on some patients improves their care of other patients.
Method A single-blind, cluster randomised trial involving a within-cluster individually randomised arm: patients' needs were evaluated using the Cardinal Needs Schedule and the findings were fed back to their care coordinators under the care programme approach. A total of 304 patients were recruited from 72 care coordinators and 242 patients (79.6%) were reassessed at 12 months.
Results The only significant effect of the intervention was on patient satisfaction. Patients cluster-randomised to receive feedback were more satisfied than controls, but patients individually randomised to receive feedback were not.
Conclusions Standardised needs assessment did not substantially enhance care planning in this trial. However, giving care coordinators some experience of feedback from a standardised assessment of need could improve satisfaction.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Study design
A potential confounding factor in this study was that the experience of
receiving feedback from a needs assessment might, in itself, alter a care
coordinator's practice, irrespective of the content of the feedback. Such an
effect could plausibly result from improved assessment practices, better
communication with patients or greater awareness of available interventions.
Contamination of this kind would not be undesirable because it
would imply that feedback on a few patients could improve care for a much
larger number of patients. It was therefore necessary for the study to
incorporate both group and individual randomisation to clarify the
contribution of the experience of feedback from the contents
of feedback (see Fig. 1).
|
In the group randomisation, care co-ordinators were allocated to one of three arms: in arm 1 they received feedback from the standardised needs assessment on all of their participating patients; in arm 2 they received feedback on half of their participating patients, who were individually randomised (within care coordinators) to feedback or no feedback, and in arm 3 they received no feedback.
Randomisation
Randomisation of care coordinators and individual patients was by numbered
sealed envelopes, using a computer-generated random allocation sequence. C.R.
provided the allocation sequence and A.L. opened the envelopes and assigned
care coordinators on the basis of a trial number supplied by the research
nurses. Care coordinators were randomised by blocks (to ensure an equal
balance across sites) and randomisation was stratified by professional status
(nurse or non-nurse).
Assessment of need
The information required for the assessment of need was collected by a
research psychiatric nurse who interviewed the patients, their main carer and
the care coordinator. Patients and carers were interviewed at home where
possible. Data from these interviews were entered on a computer program that
identified problems across 16 areas of psychiatric and social functioning. The
program then determined which of these problems required action (known as
cardinal problems) using three criteria: the cooperation
criterion, which takes account of the patient's view of the problem;
the carer stress criterion, which takes account of the carer's
view of the problem; and the severity criterion, which takes
account of the nature and severity of the problem. When the program identified
a cardinal problem, the research nurse and a consultant psychiatrist
considered a comprehensive list of defined interventions for that problem and
decided whether the patient had received, or was likely to benefit from, any
of these interventions. A need was recorded when a patient had a cardinal
problem for which there was at least one suitable intervention that had not
been offered in the past year.
The intervention: needs feedback
Within 2 weeks of the baseline needs assessment the research nurse met with
care coordinators in arms 1 and 2. The nurse gave a detailed explanation of
the process of systematic needs assessment and then provided a report
generated by the needs assessment program for each of the care coordinator's
patients who had been randomised to receive feedback. For each patient this
report explained: exactly why a problem had been identified in an area of
functioning; why the problem was considered worth acting on; what type of
intervention was required; and how the intervention might be obtained, by
reference to a database of local services. This extensive database contained
details of interventions provided by local voluntary and statutory
organisations and was indexed by type of intervention. When a need for an
intervention was identified (e.g. help with literacy), the database would
provide information on how to obtain that intervention from a suitable local
agency. The research nurse and care coordinator then discussed the report.
Subsequently, the care coordinator contacted the patient to discuss the report
and to formulate a care plan for obtaining any interventions that they and the
patient agreed were required. The research nurse telephoned the care
coordinator at 3 and 6 months to discuss progress in implementing the care
plan and the care coordinator was invited to contact the nurse for
clarification as required. All feedback sessions were recorded and a sample
was rated for quality of feedback.
Control: informal needs assessment under the care programme approach
The care programme approach specifies that every patient has a care plan
based on a regular assessment of need from a care coordinator, usually a
community psychiatric nurse or social worker. At the time of the study,
standard care programme approach practice in the participating trusts was for
a care coordinator to carry out an unstructured needs assessment based on
informal discussions with the patient, their carers and the clinical team.
This assessment did not systematically cover particular areas of functioning
or specify any particular questions that should be asked. It was left to the
discretion of the care coordinator to decide what constituted a problem or a
need. Care coordinators were not obliged to consider comprehensive lists of
defined interventions when drawing up a care plan, nor did they have access to
any up-to-date directory of agencies providing interventions.
Measurement of outcome
Follow-up at 12 months was conducted by a different research nurse who was
masked to group allocation. The primary outcome measures were mental state
(Brief Psychiatric Rating Scale, BPRS;
Overall & Gorham, 1962),
social functioning (World Health Organization Psychiatric Disability
Assessment Schedule, WHODAS; World Health
Organization, 1988), quality of life (Manchester Short Assessment
of Quality of Life, MANSA; Priebe et
al, 1999) and patient satisfaction (Client Satisfaction
Index, CSI; Larsen et al,
1979). Secondary outcomes were admission to hospital, needs at
follow-up and needs met during the study (calculated as needs present at
baseline that were no longer present at follow-up). The research nurses
attended a Present State Examination training course and were trained to
achieve reliable ratings on the BPRS. Interviews were recorded throughout the
study and a random sample was scrutinised for reliability.
Statistical analysis
The main hypothesis was that patients would have a better outcome and fewer
needs than controls if their care coordinators had received feedback from a
standardised assessment. The secondary hypothesis was that exposing care
coordinators to feedback on some of their patients would improve their care of
other patients on whom they had not received feedback.
Sample size was determined by calculating the number of care coordinators necessary to have a 90% chance of detecting a clinically significant difference (defined as half of the standard deviation for each main outcome) at a 5% level of significance. The power calculation assumed that each care coordinator provided six patients and that the within-cluster correlation was 0.013.
Statistical analysis treated the care coordinator as a random effect, to adjust for the effect of clustering due to group randomisation and variability between care coordinators. Linear and non-linear random effect models were fitted using STATA (release 6.0 for Windows). Where appropriate, baseline values of the outcome and other known predictors of outcome were included as covariates. To investigate for contamination of the treatment effect among individually randomised patients, a preliminary analysis was carried out by adding an interaction term for method of randomisation (group or individual) and intervention group. For missing data, a logistic regression model was used to investigate possible predictors of non-response. Where missing data were predictable by covariates, these variables were added to the model to reduce any bias due to missing data.
Analysis of BPRS and WHODAS gave non-linear normal probability plots because these variables were skewed. Analyses using the clustered bootstrap (Davison & Hinkley, 1997) were used to check robustness. If a patient had no needs at baseline, then no needs could be met. Hence the analysis of met needs was based on clients with at least one need at baseline, using a logistic random effects model with needs nested within clients within the care coordinator (otherwise all analyses were by intention to treat). The number of medical and social needs at follow-up each had a maximum of eight, so the proportion of medical and social needs present at follow-up was modelled using a binomial model with a denominator of eight and the overall proportion of needs was modelled using a denominator of sixteen. A random effect term was added to account for the nesting of clients within care coordinators. This analysis used the GLAMM algorithm in STATA (Rabe-Hesketh et al, 2001).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
A preliminary analysis was carried out to determine whether there was evidence of a time x randomisation method interaction for each outcome variable. For variables other than satisfaction there was no evidence of an interaction, so a main effects analysis is presented for these variables (Tables 3, 4, 5) without disaggregating the data by method of randomisation. For satisfaction there was evidence of a differential treatment effect as the interaction term approached conventional levels of significance (interaction mean effect=0.21, P=0.07, 95% CI -0.02 to -0.44). Hence the interaction term was retained in the model for satisfaction, resulting in separate estimates of the intervention effect for this variable by method of randomisation (Table 5).
|
|
|
Table 3 gives the number of needs recorded at baseline. At follow-up 88% of baseline needs were met, either because there was no longer a cardinal problem in the area or because the patient had been offered all suitable interventions (Table 4). The adjusted odds ratio (adjusted for care coordinator type) for a need being met in the intervention group relative to the control group was 1.28 (95% CI 0.57-2.88, P=0.52). A sensitivity analysis that included only met needs where there is no longer a cardinal problem did not substantially alter the original finding. In terms of the number of needs remaining at follow-up (which includes baseline needs still not met and any new needs that had arisen), the estimated odds ratio (OR) was not significant (OR=0.98, 95% CI 0.64-1.51, P=0.94) after adjustment for baseline needs and care coordinator type. Thus, there was no evidence that the intervention reduced the number of needs at follow-up. Two sensitivity analyses were performed to examine the effects of missing needs data. The first analysis assumed that all baseline needs on which there were missing data at follow-up remained unmet, whereas the second analysis assumed that all were met. Neither analysis made a substantial difference to the original findings. A further sensitivity analysis compared met needs and needs at follow-up separately for each method of randomisation, but again this did not substantially alter the original finding.
The intervention had no effect on mental state, social functioning or quality of life (Table 5) or likelihood of admission (adjusted OR=1.34, 95% CI 0.58-3.09; 9.7% admitted control, 12.1% feedback). However, there was an effect on satisfaction that depended on the method of randomisation (Table 5). When patients were randomised individually (i.e. within arm 2) there was no difference in satisfaction between the control and intervention patients (adjusted mean difference=-0.05, 95% CI -0.22 to 0.12, P=0.55), but when the randomisation was by care coordinator (i.e. arm 1 v. arm 3) there was a significant difference between control and intervention (adjusted mean difference=0.16, 95% CI 0.01-0.31, P=0.03). This difference according to method of randomisation suggests contamination in the individually randomised arm of the trial.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Why did the intervention not improve clinical outcome?
There are two obvious reasons for the failure of standardised needs
assessment to enhance the informal care planning process: the overall level of
need at baseline was at the lower end of what would have been expected from
previous surveys (see Table 3);
and the control groups, which received an informal needs assessment under the
care programme approach, showed unexpectedly high reductions in their levels
of need (87% of needs identified at baseline in the control groups were met at
follow-up; see Table 4). Both
of these findings suggest that the informal needs assessment that takes place
under the care programme approach was already effective at meeting needs, so
there was little scope for additional improvement. To have any chance of being
clinically effective the Cardinal Needs Schedule would need to be applied in
populations of patients with a higher level of baseline need, such as those
who are homeless or difficult to engage. The Schedule might also be more
effective if the list of suitable interventions were more tightly restricted
to those for which there is a strong evidence base, such as those currently
recommended in the National Institute for Clinical Excellence (NICE) treatment
guidelines for schizophrenia (NICE,
2002). It is possible that other standardised needs assessments
will prove more effective than the Cardinal Needs Assessment at enhancing
routine care planning, but this remains to be demonstrated in randomised
controlled trials.
Should standardised needs assessment be used in routine clinical practice?
With the introduction of the minimum dataset, all mental health services in
the UK will be required to undertake routine outcome assessment
(Glover, 2000) and it has been
proposed that a standardised needs assessment should be used for this purpose
(Slade, 2002). Our trial
suggests that although routine use of standardised needs assessments might
make patients more satisfied, this advantage has to be weighed against the
lack of clinical benefit and the loss of the therapist time spent completing
the assessment. There is some evidence to suggest that simple self-report
needs assessments would produce benefits in patient satisfaction that are
comparable with those produced by more complex schedules
(van Os et al,
2004).
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Davison, A. C. & Hinkley, D. V. (1997) Bootstrap Methods and their Application. Cambridge: Cambridge University Press.
Gilbody, S. M., House, A. O. & Sheldon, T. A. (2003) Outcome measures and needs assessment tools for schizophrenia and related disorders (Cochrane Review). In The Cochrane Library, Issue 1. Oxford: Update Software.
Glover, G. (2000) The minimum data set. At last
- information! Psychiatric Bulletin,
24, 163
-164.
Goldman, H. H., Gattozzi, A. A. & Taube, C. A. (1981) Defining and counting the chronically mentally ill. Hospital and Community Psychiatry, 32, 21-27.[Medline]
Larsen, D. H., Attkison, C. C. & Hargreaves, W. A. (1979) Assessment of client/patient satisfaction: development of a general scale. Evaluation and Programme Planning, 2, 197-207.[CrossRef][Medline]
Lasalvia, A., Stefani, B. & Ruggeri, M. (2000) Needs for care in psychiatric patients: a systematic review. II. Needs for care on individual (in Italian). Epidemiologia e Psichiatria Sociale, 9, 282-307.[Medline]
Lockwood, A. & Marshall, M. (1999) Can a standardised needs assessment be used to improve the care of people with severe mental disorders? A pilot study of needs feedback. Journal of Advanced Nursing, 30, 1408 -1415.[CrossRef][Medline]
Marshall, M., Hogg, L. I., Gath, D. H., et al (1995) The Cardinal Needs Schedule - a modified version of the MRC Needs for Care Assessment Schedule. Psychological Medicine, 25, 605 -617.[Medline]
Middelboe, T., Mackeprang, T., Hansson, L., et al (2001) The Nordic study on schizophrenic patients living in the community. Subjective needs and perceived help. European Psychiatry, 16, 207 -214.[CrossRef][Medline]
Murray, V., Walker, H. W., Mitchell, C., et al
(1996) Needs for care from a demand led community psychiatric
service: a study of patients with major mental illness.
BMJ, 312, 1582
-1586.
NICE (2002) Schizophrenia - Core Interventions in the Treatment and Management of Schizophrenia in Primary and Secondary Care. Clinical Guideline 1. London: National Institute for Clinical Excellence.
O'Leary, D. & Webb, M. (1996) The needs for care assessment - a longitudinal approach. Psychiatric Bulletin, 20, 134 -136.
Overall, J. E. & Gorham, D. R. (1962) The Brief Psychiatric Rating Scale. Psychological Reports, 10, 799 -812.
Phelan, M., Slade, M., Thornicroft, G., et al (1995) The Camberwell Assessment of Need: the validity and reliability of an instrument to assess the needs of people with severe mental illness. British Journal of Psychiatry, 167, 589 -595.[Abstract]
Priebe, S., Huxley, P., Knight, S., et al (1999) Application and results of the Manchester Short Assessment of Quality of Life (MANSA). International Journal of Social Psychiatry, 45, 7 -12.[Medline]
Rabe-Hesketh, S., Pickles, A. & Skrondal, A. (2001) GLAMM Manual Technical Report 2001/01. London: University of London.
Salize, H. J., Kuestner, B. M., Torres-Gonzalez, F., et al (1999) Needs for care and effectiveness of mental health care provision for schizophrenic patients in two European regions: a comparison between Granada (Spain) and Mannheim (Germany). Acta Psychiatrica Scandinavica, 100, 328 -334.[Medline]
Slade, M., Phelan, M. & Thornicroft, G. (1998) A comparison of needs assessed by staff and by an epidemiologically representative sample of patients with psychosis. Psychological Medicine, 28, 543 -550.[CrossRef][Medline]
Slade, M., Beck, A., Bindman, J., et al (1999) Routine clinical outcome measures for patients with severe mental illness: CANSAS and HoNOS. British Journal of Psychiatry, 174, 404 -408.[Abstract]
Slade, M. (2002) Routine outcome assessment in mental health services. Psychological Medicine, 32, 1845 -1856.
Van Os, J., Altamura,, A., Bobes, J., et al (2002) 2-COM: an instrument to facilitate patient-professional communication in routine clinical practice. Acta Psychiatrica Scandinavica, 106, 446 -452.[CrossRef][Medline]
Van Os, J., Altamura, A. C., Bobes, J., et al
(2004) Evaluation of the Two-Way Communication Checklist as a
clinical intervention. Results of a multinational, randomised controlled
trial. British Journal of Psychiatry,
184, 79-83.
World Health Organization (1988) WHO Psychiatric Disability Assessment Schedule (WHO/DAS). Geneva: WHO.
World Health Organization (1992) International Classification of Diseases and Related Health Problems (10th revision) (ICD-10).Geneva: WHO.
Received for publication August 18, 2003. Revision received February 29, 2004. Accepted for publication March 18, 2004.
Related articles in BJP: