Academic Unit of Child and Adolescent Psychiatry, Imperial College School of Medicine and Dentistry, London
King's College School of Medicine and Dentistry and St Christopher's Hospice, London
Correspondence: P. Yates, Clinical Research Fellow, Academic Unit of Child and Adolescent Psychiatry, Imperial College School of Medicine, St Mary's Campus, Norfolk Place, London W2 IPG, UK
Declaration of interest The study was funded by the North Thames Regional Health Authority.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To examine HoNOSCA's sensitivity to change, convergent validity and clinical usefulness.
Method Prospective study of new CAMHS attenders. Questionnaires completed by clinicians, parents and referrers at initial assessment and after 6 months.
Results Follow-up HoNOSCAs on 203 children indicated statistically significant change. There were significant associations between change in HoNOSCA scores, changes in other clinician- and parent-rated scales (r=0.51 to 0.32) and in global outcome ratings by referrers, parents and clinicians. Intraclass correlation coefficients for the summated HoNOSCA scores were high. HoNOSCA change was positively correlated with initial HoNOSCA score (r-0.46, P<0.001) and it was linked to psychiatric diagnosis.
Conclusions HoNOSCA is a sensitive, valid measure of change among CAMHS attenders.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In this study we explore further the usefulness of HoNOSCA in assessing outcome in routine clinical practice in urban and suburban clinics, its convergent validity with other measures of outcome (including parent and referrer satisfaction) and its ability to measure change in relation to case complexity, frequency of clinic attendance and psychiatric diagnosis.
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Clinician-completed scales used
Health of the Nation Outcome Scales for Children and Adolescents
(HoNOSCA) (version 5) (children aged 3-18 years)
These rate 13 clinical features on a 5-point severity scale. Two additional
questions assess parental understanding of the difficulties, and information
about services. Because of the different nature of these additional two
questions, as well as a total HoNOSCA score we analyse a summate
clinical score and change based on the 13 clinical features
only (not including parental understanding and knowledge of services)
(Gowers et al,
1999).
Children's Global Assessment Scale (CGAS) (children aged 4-16
years)
This reflects the level of functioning of a child or adolescent in the
previous month. It has acceptable discriminant validity and reliability
(Shaffer et al, 1983
; Bird et al,
1987).
Paddington Complexity Scale (PCS) (full age range)
This provides 16 items of information that would be known to professionals
working within multi-disciplinary child and adolescent mental health teams. It
covers three areas : psychiatric, physical/developmental and environmental.
Scores in each problem area are summated into a total case complexity score
and two main sub-scores (clinical and environmental). The items covered are
age, gender, ethnic group, social class, diagnosis (including duration and
severity), comorbidity, presence of chronic physical illness, presence of
learning disability, type of schooling, who the child lives with, parental
attitude to seeking help at clinics, previous contact with mental health
services, and involvement with other agencies (including involvement with the
Children Act) (Yates et al,
1999).
Global outcome (full age range)
At 6-month follow-up, clinicians rated on a 5-point scale the degree of
improvement in the child's condition (much improved, improved, no change,
worse, much worse).
Parent-completed scales used
Strengths and Difficulties Questionnaire (SDQ) (children aged 4-16
years)
This is a brief behavioural screening questionnaire that provides balanced
coverage of children's and young people's behaviour, emotions and
relationships. It has adequate discriminant and predictive validity
(Goodman, 1997 ;
Goodman et al, 1998 ;
Goodman & Scott, 1999).
Behaviour Checklist (BCL) (children aged 2-5 years)
This 12-item checklist relates to common behaviour in young children and
has wellestablished validity and test-retest reliability
(Richman, 1977 ;
Richman et al,
1982).
Global outcome (full age range)
Parents completed global measures of outcome at follow-up to indicate :
changes in the child's condition (5-point scale : much improved, improved, no
change, worse, much worse) ; changes in the parents' concern about this
(4-point scale : much less concerned, less concerned, no different, more
concerned) ; changes in their confidence in dealing with the child's problems
(4-point scale : much more confident, more confident, no different, less
confident) ; and the appropriateness of help offered at clinics (4-point scale
: not at all, to some degree, very much so, completely).
Scales completed by the referrer
Global scales (full age range)
At 6-month follow-up, referres were requested to complete questionnaires to
indicate whether their referral aims had been achieved (3-point scale : yes,
partly, no) and whether the intervention had been helpful to the patient and
family (3-point scale : very helpful, helpful, made no difference).
Training to use the scales and collection of data
Clinic staff received training from a clinical research fellow in the use
of the PCS and HoNOSCA, using case vignettes. The research fellow was
available over the time of the study to help with any problems in completion
or inconsistencies. At the initial assessment, parental and child
questionaires were administered just before the families attended their first
clinic appointment.
For children attending at least twice, the following measures, able to reflect clinical change, were obtained at initial assessment and at 6-month follow-up : clinicians completed HoNOSCAs on all children and CGAS for children aged 4-16 years ; parents completed the BCL for children aged 2-5 years and the SDQ for those aged 4-16. For all children attending we also obtained clinician-rated PCS at the initial assessment, and we sought to obtain measures on global improvement and feedback about the intervention from referrers, parents and clinicians.
Change scores on all measures were derived by subtracting for each case the 6-month follow-up data from the initial scores.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Thirty-one clinicians from seven disciplines working within five psychiatric teams were involved in completing questionnaires as follows : psychiatry (16), nursing (4), psychology (4), social work (3), family therapy (2), speech and language therapy (1), occupational therapy (1).
The sample
Details of the sample of 248 children are given in
Table 1. The mean age of the
total sample was 10 years ; it contained a predominance of boys, and a
comparatively large proportion (one-third) of children from ethnic minorities
and from low socioeconomic groupings and broken families. About half the
children had been referred by general practitioners and two-thirds of cases
had diagnoses within the broad groups of disruptive/externalising or
emotional/internalising disorders. Most were seen as out-patients. Sixty-one
attended for fewer than three sessions, whereas 187 (75%) attended for three
or more. Table 1 shows that
there were demographic differences between children attending for assessment
only (fewer than three sessions) and those attending for treatment (three or
more sessions). There were more children of Asian background and more children
with emotional disorders in the treatment group. The mean clinicl attendance
was eight sessions (s.d. 8.3, range 1-54). At 6-month follow-up 110 cases
(44%) were still open and 138 (56%) were closed.
|
Feasibility of using HoNOSCA in routine clinical practice to assess
outcome
We found the HoNOSCA perfectly feasible to use after training (30-60 min)
at initial assessment and at 6-month follow-up. It was easy and quick to use,
taking between 5 and 10 min to complete, once clinicians were familiar with
it.
Reliability of HoNOSCA
Interrater reliability was established for 15 cases which were rated by
three raters presented with case vignettes. There was good interrater
reliability, with intraclass correlations of over 0.82 for psychiatric
symptoms, and 0.42-0.61 for physical and social impairment, but low for
knowledge about difficulties and services, =0.27 and 0.03
respectively.
We also examined test-retest reliability by selecting cases for which
clinicians had considered there had been no global clinical change over the
6-month period, and by comparing the initial and 6-month followup HoNOSCA
scores. These were closely comparable (mean 11.58, s.d. 4.45 and mean 10.77,
s.d. 3.74) with a correlation of 0.69 (=53, P <0.01,
two-tailed Pearson correlation).
Sensitivity of HoNOSCA and other scales in detecting clinical
change
Table 2 shows changes in
HoNOSCA scores in the group attending more than once. At the 6-month follow-up
there was a significant improvement, with a mean positive change of 3.61 (s.d.
4.7). There were significant changes indicating improvement on all HoNOSCA
sub-scores (behaviour, impairment, symptoms, social functioning and
information). Similarly significant improvements were indicated by changed
scores on the cliniciancompleted impairment measure (CGAS), on parentally
rated BCL and SDQ, and on the parental GHQ reflecting maternal mental
health.
|
The global measures of change completed by clinicians, parents and referrers on cases attending more than once reflected improvement and change in most cases. Clinicians felt that 72% of patients had improved. Parents or carers felt that the child's problem had improved in 66% of cases ; about half felt more confident in dealing with the problem and less concerned about it, and the majority of parents (94%) said that the help offered was appropriate to some degree at least (very appropriate in half). Referrers felt that the contact with the clinic had been helpful for families in 68% of cases and that their referral aims had been achieved to some degree at least in 69% of cases.
Change in HoNOSCA in relation to change in other measures
We found significant, modest Pearson (two-tailed) correlation coefficients
between change in HoNOSCA and change in clinician-rated CGAS (r=0.51,
P<0.001) and to a lesser extent with parentally rated BCL
(r=0.40, P=0.04) and SDQ (r=0.32,
P<0.001). There was however little association with changes in
maternal psychopathology as measured by the GHQ (r=0.16,
P=0.05).
Table 3 indicates associations between HoNOSCA change and change in the global outcome measures. There were significant associations with global impressions of positive change by clinicians, and also with global impressions of improvement by parents and with referrers' ratings on the helpfulness of the intervention for the child and family. Clinicians' and parents' global ratings of marked improvement corresponded with mean HoNOSCA changes of 6.5 and 5.6 respectively (some improvement corresponded with HoNOSCA change scores of 4). Judgements by referrers of marked helpfulness to families reflected a mean HoNOSCA change of 4.6. These were based on only 154 cases, since referrers of the other cases had not seen the family following contact with Child Psychiatric Clinics and could not therefore express an opinion on this.
|
HoNOSCA change in relation to initial HoNOSCA severity and case
complexity
We examined whether HoNOSCA change was associated with initial case
severity on the HoNOSCA. We found a significant positive correlation
(r=0.468, P<0.001, n=215), indicating more
change with more initial severity.
We also explored whether HoNOSCA change was associated with more initial case complexity. We had assessed the latter using the Paddington Complexity Scale (PCS). This provides information on a number of clinical and demographic items, as described above. Individual items are scored by the degree to which they are likely to add complexity to case management. They are summated into total case complexity scores, and into sub-scores (i.e. psychiatric and physical/developmental sub-scores, which are again summated into a clinical complexity sub-score, and an environmental complexity sub-score). We found no association between changes in HoNOSCA score and PCS total complexity score, nor between HoNOSCA change and psychiatric, developmental, clinical or environmental complexity sub-scores. We then examined whether change in HoNOSCA score was linked to any of the individual demographic or complexity items documented by the PCS. The only significant association was with the severity of the clinical picture (mean changes of 0.52 for mild cases, 3.7 for moderate and 4.4 for severe cases, ANOVA P=0.009). At a level short of statistical significance, learning disability was linked to less HoNOSCA improvement (mean HoNOSCA change of 4.0 in those without and 2.8 in those with learning disability, t-test, P=0.07).
Change in HoNOSCA score, and length of treatment
We found little association between the number of sessions attended and
change in HoNOSCA (r=0.18, P=0.01). There was similarly
little association between the number of sessions attended and change on the
CGAS (r=0.20, P=0.004), SDQ (r=0.17,
P=0.02), BCL (r=-0.03, P=0.8) and maternal GHQ
(r=0.17, P=0.03).
There was however a modest correlation between initial HoNOSCA rating and the number of sessions attended at clinics (r=0.31, P<0.001). We examined this issue further by comparing the HoNOSCA change in cases attending for assessment only (defined here as attending twice, as we did not collect repeat measures on those attending once only) and in those attending for treatment (three or more sessions). Table 4 shows that there were no differences between the groups in the degree of change in HoNOSCA, CGAS and SDQ. There were no differences between the groups in global outcome measures (i.e. improvement, concern and confidence regarding the problem). However in the treatment group, significantly more parents thought they had received appropriate help and significantly more referrers felt that the aims of their referrals had been achieved and the intervention had been helpful to the child and family.
|
HoNOSCA change and psychiatric diagnosis
There were significant differences between diagnostic groups in HoNOSCA
initial scores and changes in score (Table
5). The biggest change in HoNOSCA score occurred in the child with
a psychotic disorder. Otherwise, the greatest HoNOSCA changes were seen in
children with mood disorders and adjustment or stress disorders ; and the
smallest changes in those with psychosomatic and eating disorders.
|
To determine whether the lack of associations in the total group between HoNOSCA change and PCS case complexity was masking differential associations in individual diagnostic groups, we analysed PCS total scores and sub-scores by diagnosis ; and we then examined correlations between HoNOSCA change and complexity scores in each individual diagnostic group.
Table 5 shows that there were significant differences between diagnostic groups on both total complexity scores and complexity sub-scores. Physical/developmental complexity was highest for children with conduct, hyperkinetic and other disorders (which included autistic-spectrum disorders). The highest environmental complexity was found in the conduct and psychosomatic disorders. However, we failed to find significant correlations between HoNOSCA change and physical/developmental or environmental complexity for any of the diagnostic groups. The only associations found were between HoNOSCA change and psychiatric complexity (which, as well as diagnosis, includes items related to clinical severity, duration and comorbidity), indicating more change with more complexity or clinical severity in mood disorders (r=0.39, P=0.04, n=26) and in conduct disorders (r=0.33, P=0.01, n=57).
We also examined whether a change in HoNOSCA score might be differentially associated with length of treatment in individual diagnostic groups. The only significant correlation was found in the mood disorder group (r=0.46, P=0.01, n=26).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In line with previous recent reports about the use made of child and adolescent mental health clinics (Hoare et al, 1995 ; Gowers et al, 1999) most parents, referrers and clinicians felt that the children had improved significantly following contact with clinics. Parents also reported benefits to their parenting of the children's difficulties. Marked improvements in parents' and referrers' global outcome measures were associated with HoNOSCA change scores of 4-5. A slightly more demanding change of six HoNOSCA points reflected clinician-rated global judgements of marked improvement. This is similar to the findings by Gowers et al (1999), where a change of 7.7 HoNOSCA points was associated with clinicians' retrospective assessment of a child being much better. This level of change could therefore be used as a guide or anchor-point for child psychiatric clinics wishing to audit clinical change among their clientele. It is important to highlight, however, that these changes were obtained at 6-month follow-up, when only half the cases had been closed. They do not therefore necessarily indicate improvement on discharge from clinics.
The HoNOSCA changes described in this and other papers (Gowers et al, 1998, 1999) are more substantial than those derived from the use of its parent scale (HoNOS or Health of the Nation Outcome Scales) in adult psychiatric patients (Sharma et al, 1999). This indicates that the measure may be particularly helpful for use in child and adolescent psychiatric clinics.
The use of HoNOSCA allowed the study of a variety of clinical and service features. Thus we found associations between HoNOSCA-measured improvement, high initial severity of the clinical condition and individual psychiatric diagnoses. We failed, however, to find associations with initial case complexity or frequency of clinic attendance.
Change in HoNOSCA score and clinical severity
Change in HoNOSCA score was correlated with high initial HoNOSCA score and
with high ratings of the severity of the condition on the Paddington
Complexity Scale. The study was conducted in secondary psychiatric out-patient
services where patients had initial mean HoNOSCA scores of 11.40 (s.d. 4.89).
These services may be expected to attend to the more severe and complex mental
health problems of children and adolescents, and our findings may not be
applicable to other units. HoNOSCA may thus be less sensitive in detecting
change in milder cases seen by single-discipline community units or in primary
care, but may be particularly helpful in in-patient settings.
HoNOSCA change and case complexity
Perhaps surprisingly, HoNOSCA change was not linked to initial case
complexity on the PCS, nor was it associated with the variety of variables
making up the environmental complexity score or with the demographic variables
examined. It had seemed logical to assume that single parenthood, at-risk
status, a non-facilitative attitude by parents, associated medical and
educational problems, ethnicity, age and gender would have an adverse
influence on the implementation of treatments and thus on outcome. A possible
explanation for our negative findings is that clinics were so adept at dealing
with these extraneous factors that they were able to make sure that they did
not work against the treatment : clinicians would successfully adapt the
nature and intensity of their intervention to the complexity of the case.
Alternatively, the treatments put into place would have been so powerful as to
not be adversely influenced by these factors. It is also possible, however,
that change in the HoNOSCA score was a naturally occurring phenomenon,
independent of clinic use, and that the assumed complexity factors simply do
not affect the collective natural outcome of these disorders.
Associations with length of treatment
The lack of associations between the number of sessions attended and
HoNOSCA change could be taken to indicate that the interventions at clinics
had been inconsequential for the improvement noted. However, the positive
associations found between initial HoNOSCA-rated severity and the number of
sessions attended, and the significant correlations between change in HoNOSCA
and length of treatment in the mood disorder group suggest that clinics may
have been skilled at providing the number of sessions required for individual
problems. More parents and referrers felt the children had improved following
attendance and that the help given was appropriate and had met the referrers'
aims, when children had attended for treatment rather than for assessment
only. In parents' and referrers' view there seemed to be a connection between
improvement and clinic intervention.
Change in relation to psychiatric diagnosis
There were differences in the changes in HoNOSCA scores in individual
diagnostic groups, though this did not simply follow diagnostic predictions.
Conduct disorder, anxiety disorders and hyperkinesis improved by similar
amounts, the greatest change being seen in a child with a psychotic disorder.
However, in line with expectations, children with stress and mood disorders
improved most, indicating that the work of the clinics with these disorders
may result in the largest HoNOSCA change. Eating disorders improved little,
which may have been related to the particularly low initial HoNOSCA scores.
Children with psychosomatic disorders were poor responders, and interestingly
had the highest environmental complexity scores. The use of HoNOSCA to audit
work at clinics may therefore helpfully monitor individual conditions
separately.
Need for training
It should be pointed out that the results of this study were based on
HoNOSCA ratings made by clinicians who had been carefully trained in its use.
The importance of scoring HoNOSCAs in a standardised way and of incorporating
regular training into the audit cycle has rightly been emphasised by those
developing the scales (Gowers et al,
1998,
1999).
![]() |
CLINICAL IMPLICATIONS AND LIMITATIONS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Goodman, R. (1997) The Strengths and Difficulties Questionnaire : a research note. Journal of Child Psychology & Psychiatry, 38, 581 -586.[Medline]
, Meltzer, H. & Bailey, V. (1998) The Strengths and Difficulties Questionnaire : a pilot study on the validity of the self-report version. European Child and Adolescent Psychiatry, 7, 125 -130.[CrossRef][Medline]
& Scott, S. (1999) Comparing the Strengths and Difficulties Questionnaire and the Child Behaviour Checklist : is small beautiful ? Journal of Abnormal Child Psychology, 27, 17 -24.[CrossRef][Medline]
Gowers, S. G., Harrington, R. C. & Whitton, A. (1998) Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). London : Royal College of Psychiatrists' Research Unit.
, , , et al (1999) Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). British Journal of Psychiatry, 174, 413 -416.[Abstract]
Hoare, P., Scarth, L. & Forbes, F. (1995) Audit of the Edinburgh Child Psychiatry Out-Patient Service. ACPP Review & Newsletter, 17, 139 -148.
Jensen, P. S., Hoagwood, K. & Petti, T. (1996) Outcomes of mental health care for children and adolescents : II. Literature review and application of a comprehensive model. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1064 -1077.
Richman, N. (1977) Is a behaviour checklist for preschool children useful ? In Epidemiological Approaches in Child Psychiatry (ed. P. J. Graham). London : Academic Press.
, Stevenson, J. & Graham, P.J. (1982) Pre-School to School. A Behavioural Study. London : Academic Press.
Shaffer, D., Gould, M. S., Brasic, J., et al (1983) A Children's Global Assessment Scale (CGAS). Archives of General Psychiatry, 40, 1228 -1231.[Abstract]
Sharma, V. K., Wilkinson, G. & Fear, S. (1999) Health of the Nation Outcome Scales : a case study in general psychiatry. British Journal of Psychiatry, 174, 395 -398.[Abstract]
Yates, P., Garralda, M. E. & Higginson, I. (1999) Paddington Complexity Scale and Health of the Nation Outcome Scales for Children and Adolescents. British Journal of Psychiatry, 174, 417 -423.[Abstract]
Received for publication September 20, 1999. Revision received January 18, 2000. Accepted for publication January 19, 2000.