Use of a routine, self-report outcome measure (HoNOSCA—SR) in two adolescent mental health services

SIMON GOWERS, FRCPsych

Section of Adolescent Psychiatry, University of Liverpool

WARREN LEVINE, MRCPsych

Child and Adolescent Mental Health Service, Douglas, Isle of Man

SARAH BAILEY-ROGERS, BSc, ALISON SHORE, BSc and EMMA BURHOUSE, BSc

Section of Adolescent Psychiatry, University of Liverpool, UK

Correspondence: Professor Simon Gowers, Mersey Regional Young People's Centre, Pine Lodge, 79 Liverpool Road, Chester CH2 1AW, UK

Declaration of interest The HoNOSCA base was supported by a grant from the NHS Information Authority.


   ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
Background The Health of the Nation Outcome Scale for Children and Adolescents (HoNOSCA) is an established outcome measure for child and adolescent mental health. Little is known of adolescent views on outcome.

Aims To develop and test the properties of an adolescent, self-rated version of the scale (HoNOSCA—SR) against the established clinician-rated version.

Method A comparison was made of 6-weekly clinician-rated and self-rated assessments of adolescents attending two services, using HoNOSCA and other mental health measures.

Results Adolescents found HoNOSCA—SR acceptable and easy to rate. They rated fewer difficulties than the clinicians and these difficulties were felt to improve less during treatment, although this varied with diagnosis and length of treatment. Although HoNOSCA—SR showed satisfactory reliability and validity, agreement between clinicians and users in individual cases was poor.

Conclusions Routine outcome measurement can include adolescent self-rating with modest additional resources. The discrepancy between staff and adolescent views requires further evaluation.


   INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
Outcome measurement in child and adolescent mental health services has recently seen an upsurge in interest. In 1998 the Department of Health launched its approved outcome measure in mental health, the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). Field trials suggested that HoNOSCA had satisfactory validity, interrater reliability and levels of acceptability in multi-disciplinary practice (Gowers et al, 1999a,b). These findings have been replicated in Britain (Yates et al, 1999; Garralda et al, 2000) and in Australia (Brann et al, 2001). The use of HoNOSCA in the Audit Commission's review of child and adolescent mental health services (Audit Commission, 1999) and the establishment of the HoNOSCA training base have led to its routine use in many mental health services. One criticism, however, is that HoNOSCA fails to incorporate a patient perspective. The current project aimed to introduce an adolescent-rated version of HoNOSCA, assess its properties, and explore its performance against other measures of adolescent difficulty.


   METHOD
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
In May 1997, following training in the use of HoNOSCA, the multi-disciplinary team of the Mersey Regional Adolescent Mental Health Service began incorporating this measure into their routine clinical work. All adolescents admitted to the in-patient unit had a clinician-rated HoNOSCA assessment on admission, reviewed at 6-weekly intervals and at discharge. Those attending a regional out-patient eating disorders service were also rated at presentation and at 6-weekly intervals. In parallel with the clinician assessments, a self-rated version of the scale (HoNOSCA—SR; see Appendix) was administered as a guided questionnaire by an assistant psychologist not involved in the patient's care. The self-rated version was based on the items in the clinician-rated version rephrased as questions. The glossary used for HoNOSCA was judged to be inappropriate for adolescents, given the complexity of the language and the time it would take to master. The phrasing of the questions was kept as close as possible to the original HoNOSCA items in order to maximise the possibility of agreement between the two versions. Each question was read out and explained to the adolescent. Any problems reported were explored in terms of severity or frequency in order to help the patient assign a score (from 0=not at all, to 4=severely) before moving on to the next item.

The reliability of HoNOSCA—SR was considered in two ways. Split-half reliability comprised a correlation between scores on odd and even numbered items for a series of 65 patients. Test—retest reliability was rated for 24 in-patients tested on two occasions 1 week apart. Validity was tested by correlation with scores on self-reported and clinician-rated measures of psychopathology and general functioning, namely the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997) and, for the eating disorder group, the Morgan—Russell schedule (Morgan & Hayward, 1988) and the Eating Disorders Inventory (EDI—2; Garner, 1991).

Correlations between clinician- and user-rated versions enabled assessment of agreement between raters. The complete data were analysed using SPSS (SPSS, 1998).


   RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
Recruitment
Over the period of study there were 96 admissions to the in-patient unit, of which 41 were emergencies and 55 were elective. The patients admitted had a mean age of 15.05 (range 13-18). There were 69 girls and 27 boys. The eating disorders service saw 54 new referrals for out-patient treatment for anorexia nervosa. All these patients were female, with a similar mean age (15.6 years). Clinician-rated HoNOSCAs were completed for all in-patients and out-patients at all rating points, comprising 361 scoring episodes.

The self-rated version of HoNOSCA was completed by all adolescents at admission and discharge from the in-patient unit, although two failed to record mid-treatment review ratings. All out-patients completed self-ratings at each time point. There were therefore 359 self-ratings, representing a completion rate of 99.4%.

The completion of the self-rated version took 10-15 minutes for the first rating and about 5 minutes for subsequent ratings.

Reliability of HoNOSCA—SR
Split-half reliability showed a correlation between the odd- and even-numbered scores of 0.73 based on a series of 65. Test—retest reliability of HoNOSCA—SR was explored in a subset of 24 patients to whom the instrument was administered on two occasions, a week apart. As these were in-patients with moderately severe disorder, it was assumed that there was unlikely to have been much change in severity of difficulties between the two rating points. The intraclass correlation coefficients varied from 0.32 for item 1 (antisocial and aggressive behaviour) to 0.88 for item 2 (attention and concentration). The total score (Pearson's) correlation between time 1 and time 2 was 0.806, P < 0.001 (Table 1).


View this table:
[in this window]
[in a new window]
 
Table 1 Test—retest reliability for the self-rated version of HoNOSCA (n=24)
 

Validity of HoNOSCA—SR
The validity of HoNOSCA—SR was explored by correlating it with the SDQ and (for the eating disorder series) the modified Morgan—Russell severity schedule, which provides a global severity measure covering physical, psychological and social adjustment (Morgan & Hayward, 1988). This measure has been widely used as a severity measure in anorexia nervosa (Gowers et al, 2000). The HoNOSCA—SR gave a correlation of 0.66 (P < 0.001) with the adolescent-rated SDQ (n=39 cases); this compares with a correlation of 0.397 between the clinician-rated version of HoNOSCA and the SDQ (Yates et al, 1999). For the patients with eating disorders, HoNOSCA—SR correlated moderately well with the total (self-report) EDI—2 score (r=0.629, n=53, P < 0.001), but only weakly (and inversely, given the direction of scoring) with the clinician-rated Morgan—Russell total score (r=-0.32, n=54, P=0.018).

Patient view of clinical severity and outcome
At admission to the in-patient unit, or at presentation to the out-patient service, the adolescent patients recorded comparable levels of difficulty to the clinician ratings. For the in-patients (n=96) the mean total admission score was 17.8, compared with the mean clinician rating of 19.6. For the out-patients the mean scores were 14.9 and 18.8, respectively. There were, however, notable differences between scale items (Fig. 1), with clinicians tending to give higher ratings to difficulties with peer relationships and school attendance, while the young people themselves drew attention to problems with attention and concentration (question 2) and scholastic and language skills (question 5).



View larger version (21K):
[in this window]
[in a new window]
 
Fig. 1 Comparison of clinician-rated and self-reported scores on the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) at admission and discharge for 96 adolescent in-patients. This scale scores for (1) aggression, (2) concentration, (3) self-injury, (4) substance misuse, (5) language, (6) physical illness, (7) hallucinations, (8) somatic disturbance, (9) emotion, (10) peer relationships, (11) independence, (12) family relationships and (13) school attendance; see Appendix for full details.

 

During the course of treatment both the clinician-rated and self-rated versions of HoNOSCA demonstrated sensitivity to change. The in-patients reported a significant change over time, but slightly less than was reported by the staff (mean admission and discharge scores for 57 patients admitted for longer than 6 weeks were 18.44 and 12.82 self-rated, compared with clinician ratings of 18.35 and 9.75). The out-patients with eating disorders showed changes comparable with the clinician ratings. The changes in ratings over time for a consecutive smaller series from both services who were treated for a minimum of 4 months is shown in Table 2.


View this table:
[in this window]
[in a new window]
 
Table 2 Mean clinician-rated and self-report HoNOSCA scores over time for in-patient and out-patient cases for all patients in treatment for 18 weeks
 

Considering each scale item in turn, both raters reported improvement across the board, but young people were more positive about improvements in family relationships (item 12) and clinicians about behavioural items (items 1-3) (see Fig. 1).

Agreement between clinician and patient ratings
In individual cases agreement between clinician and patient ratings was poor, with a correlation (Pearson's r) between HoNOSCA and HoNOSCA—SR of 0.27 at presentation, rising to 0.58 at discharge. There were notable differences by diagnosis, for the in-patients. Those with psychoses tended to see themselves as having fewer problems than the clinician's assessment, while those with emotional (chiefly affective) disorders rated themselves as having more. A subgroup of patients with incipient borderline personality disorder with poor peer relationships and self-harm were the highest scorers on HoNOSCA—SR (mean admission score 24.6). The out-patient eating-disorder group showed much higher levels of agreement, with a correlation of 0.58 at presentation and quite good agreement (r=0.72) at discharge.


   DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
Strengths and limitations of HoNOSCA
Clinicians, patients and commissioners all require comprehensive outcome data in order to evaluate services and individual treatment response. The HoNOSCA assessment has become well established as a clinician-rated general outcome measure (Garralda et al, 2000), balancing simplicity with a reasonable level of reliability and validity (Brann et al, 2001). It has been shown to correlate highly with the Children's Global Assessment Scale (Shaffer et al, 1983) and to complement the Paddington Complexity Scale (PCS) in terms of its coverage of mental health problems in children and adolescents (Yates et al, 1999).

A survey of 120 child and adolescent mental health services who had received training in the use of HoNOSCA suggested that half were using it 6 months later (further details available from the author upon request). Not all have been persuaded that the enthusiasm for HoNOSCA is fully justified, however (Jaffa, 2000), and other measures have since been developed. Alongside the PCS, the Strengths and Difficulties Questionnaire (Goodman, 1997) is a notable addition to the field. Incorporating both parent and child versions, the SDQ raises the question of who is the most relevant rater of a child's difficulties: clinician or child? One of the major reservations about HoNOSCA is that there is no patient perspective, and that clinicians might inevitably see progress when their interventions are being assessed.

Adolescent views of severity
Despite theoretical difficulties of insight and sometimes poor engagement between adolescents and mental health services, this study shows that adolescents are aware of their difficulties in a range of domains and will freely admit to them. They often do not see difficulties in the same areas as their treating clinicians, however. It is noteworthy that adolescents themselves rate the subjective items (such as difficulty with concentration and attention) as more problematic than staff do. This discrepancy highlights the importance of therapeutic engagement in identifying difficulties and hence common treatment aims.

Adolescent views on outcome
Despite the above reservations, in the two services studied adolescents reported significant improvements with treatment, which spanned the duration of intervention. This gave some support to the average length of treatment of 3-4 months. In the out-patient service, this view matched the progress seen by staff.

Agreement between clinician and patient ratings
The correlation between HoNOSCA and HoNOSCA—SR is weak, particularly for in-patients. These patients have been rated at the point of admission, and their attitude to admission may be reflected in the scores. Approximately one-third was suffering from an acute psychotic disorder, with predictable levels of suspicion and lack of insight. The greater level of agreement between clinicians and patients in the out-patient service requires explanation. This could be related to the diagnostic homogeneity of the out-patient group, but people with eating disorders are not noted for acknowledging difficulties. A more likely explanation is that in an out-patient service clinicians are more reliant on patient and informant report in completing ratings, whereas in-patient staff can observe rather than take the patient's word for performance in such areas as peer relationships. It may therefore be that despite the lower levels of agreement, the clinician ratings for the in-patients are more objective.

Therapeutic use of HoNOSCA—SR
The two services have used the clinician- and self-rated scores to monitor progress through treatment with the patients themselves. In the out-patient eating disorder service particularly, where patients are encouraged to change behaviour, the scores provide regular feedback in addition to physical measures and the scores on specific eating disorder self-report questionnaires. Thus, patients with anorexia nervosa might be encouraged to gain weight and judge for themselves whether this results in an improvement or deterioration in general functioning by their own account. Progress in the first 6 weeks can then be used to boost motivation for the next phase of treatment. The possibilities for the therapeutic use of HoNOSCA to provide feedback to patients look promising.


   Clinical Implications and Limitations
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
CLINICAL IMPLICATIONS

LIMITATIONS


   APPENDIX
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
Self-rated version of the Health of the Nation Outcome Scales for Children and Adolescents
The patient is asked to assess the following items, selecting answers from the categories ‘not at all’, ‘insignificantly’, ‘mild but definitely’, ‘moderately’ or ‘severely’.

In the last two weeks

  1. Have you been troubled by your disruptive behaviour, physical or verbal aggression?
  2. Have you suffered from lack of concentration or restlessness?
  3. Have you done anything to injure or harm yourself on purpose?
  4. Have you had problems as a result of your use of alcohol, drugs or solvents?
  5. Have you experienced difficulties keeping up with your usual educational attainments and abilities?
  6. Has any physical illness or disability restricted your activities?
  7. Have you been troubled by hearing voices, seeing things, suspicious or abnormal thoughts?
  8. Have you suffered from self-induced vomiting, head/stomachaches/bedwetting or soiling, or other symptoms with no physical cause?
  9. Have you been feeling in a low or anxious mood, or troubled by fears, obsessions or rituals?
  10. Have you been troubled by a lack of satisfactory friendships or bullying?
  11. Have you found it difficult to look after yourself or take responsibility for your independence?
  12. Have you been troubled by relationships in your family or substitute home?
  13. Have you stopped attending school/college/education sessions?


   REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 METHOD
 RESULTS
 DISCUSSION
 Clinical Implications and...
 APPENDIX
 REFERENCES
 
Audit Commission (1999) Children in Mind: Child and Adolescent Mental Health Services. Abingdon: Audit Commission Publications.

Brann, P., Coleman, G. & Luk, E. (2001) Routine outcome measurement in a child and adolescent mental health service: an evaluation of HoNOSCA. Australian and New Zealand Journal of Psychiatry, 35, 370-376.[CrossRef][Medline]

Garner, D. M. (1991) The Eating Disorders Inventory — 2. Odessa, FL: PAR.

Garralda, M. E., Yates, P. & Higginson, I. (2000) Child and adolescent mental health service use. HoNOSCA as an outcome measure. British Journal of Psychiatry, 177, 52-58.[Abstract/Free Full Text]

Goodman, R. (1997) The Strength and Difficulties Questionnaire: a research note. Journal of Child Psychology and Psychiatry, 38, 581-586.[Medline]

Gowers, S. G., Harrington, R. C., Whitton, A., et al (1999a) Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). British Journal of Psychiatry, 174, 413-416.[Abstract]

Gowers, S. G., Harrington, R. C., Whitton, A., et al (1999b) Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). Glossary for HoNOSCA score sheet. British Journal of Psychiatry, 174, 428-431.[Medline]

Gowers, S. G., Weetman, J., Shore, A., et al (2000) Impact of hospitalisation on the outcome of adolescent anorexia nervosa. British Journal of Psychiatry, 176, 138-141.[Abstract/Free Full Text]

Jaffa, T. (2000) HoNOSCA: is the enthusiasm justified? Child Psychology and Psychiatry Review, 5, 14.[CrossRef]

Morgan, H. F. & Hayward, A. E. (1988) Clinical assessment of anorexia nervosa. The Morgan—Russell outcome assessment schedule. British Journal of Psychiatry, 152, 367-371.[Abstract]

Shaffer, D., Gould, M. S., Brasic, J., et al (1983) A Children's Global Assessment Scale (CGAS). Archives of General Psychiatry, 40, 1228-1231.[Abstract]

SPSS (1998) SPSS for Windows Base System User's Manuals. Release 9.0. Chicago, IL: SPSS Inc.

Yates, P., Garralda, M. E. & Higginson, I. (1999) Paddington Complexity Scale and Health of the Nation Outcome Scales for Children and Adolescents. British Journal of Psychiatry, 174, 417-423.[Abstract]

Received for publication May 17, 2001. Revision received October 18, 2001. Accepted for publication October 26, 2001.





This Article
Abstract
Full Text (PDF)
Submit a response
Alert me when this article is cited
Alert me when eLetters are posted
Alert me if a correction is posted
Citation Map
Services
Email this article to a friend
Similar articles in this journal
Similar articles in PubMed
Alert me to new issues of the journal
Download to citation manager
Google Scholar
Articles by GOWERS, S.
Articles by BURHOUSE, E.
Articles citing this Article
PubMed
PubMed Citation
Articles by GOWERS, S.
Articles by BURHOUSE, E.


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Psychiatric Bulletin Advances in Psychiatric Treatment All RCPsych Journals