Brooklands, Marston Green, Birmingham, UK
St David's, Carmarthen, UK
British Psychological Society, UK
Galahad Substance Misuse Solutions Ltd, Oxted, Surrey, UK
Brooklands, Marston Green, Birmingham, UK
Correspondence: Dr A. Roy, Consultant Psychiatrist, Brooklands, Coleshill Road, Marston Green, Birmingham B37 7HL, UK
Declaration of interest Supported by the Royal College of Psychiatrists and the British Psychological Society.
See pp. 6770, this
issue
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims Frequent impaired social functioning, problems with communication and associated physical conditions meant that a bespoke instrument was needed for people with learning disabilities. We describe the development of the Health of the Nation Outcomes Scales for People with Learning Disabilities (HoNOSLD).
Method HoNOSLD was piloted at 26 sites. Two raters, at two points in time, rated 372 subjects. Analysis determined acceptability, ease of use, interrater reliability, sensitivity to change and reliability with the Aberrant Behavior Checklist (ABC).
Results The resulting 18-item instrument demonstrated good reliability and validity characteristics and is generally acceptable to clinicians.
Conclusions HoNOSLD is an appropriate instrument for measuring outcome in people with learning disabilities with additional mental health needs.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The HoNOS provides professionals with a framework to measure risk and vulnerability. There are specific advantages in using HoNOS. It provides a systematic summary of behaviours and functioning, is easy to use, provides consistent measurements and provides basic monitoring and outcome information. In addition it allows all staff to work with the same criteria.
The HoNOS has generated a family of scales that target specific groups. HoNOS-CA (for use with children and adolescents) (Gowers et al, 1999) and HoNOS 65+ (for the population aged 65 years or over) (Burns et al, 1999) have both been designed and tested. There is also a version that has been designed for use with mentally disordered offenders (HoNOS-MDO) (P. Sugarman & H. Everest, personal communication, 1999).
People with learning disabilities are prone to the same range of mental illnesses as the general population, although there is a higher prevalence of mental illness within this group of people (Patel et al, 1993; Corbett, 1979). People with learning disabilities are also sometimes further disadvantaged because of dual diagnosis (Bernal & Hollins, 1995). There are several reliable and valid instruments for measurement of various aspects of learning disabilities, such as behavioural problems, physical disability and sensory disability (Aman, 1991). There are few easy-to-use instruments, however, used by clinicians in day-to-day work, to measure changes in the level of functioning in people with learning disabilities who have added mental health needs.
The generic HoNOS was used with people with learning disabilities in longstay settings and was found to be limited in areas such as communication skills and movement disorders (Ashaye et al, 1997).
Lelliott (1995), in calling for an integrated clinical information system, highlights the need for such an instrument. Problems encountered with poor coordination of services and different agencies means that an individual may not receive the best, or the appropriate care that he or she has a right to in the National Health Service (NHS).
"Agreement within clinical disciplines and between agencies on the use and definitions of common clinical terms concerning symptoms, signs and diagnoses, problems, needs, actions and outcomes could lead to standardisation of the way in which information is gathered and coded. This would open paths for better communication between those involved in mental health care and for comparisons between providers."
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Following extensive consultation, it was agreed that the resulting instrument was worthy of testing in field trials.
The HoNOSLD is designed for use with people with a learning disability and with added mental health needs, irrespective of the degree of their disability. Its primary aim is to measure change in an individual over two or more points in time as a measurement of outcome for therapeutic interventions. It measures change in the level of problems that a person has had and is not intended to be a comprehensive description of the individual. Like the generic HoNOS, HoNOSLD may be used as a simple present state profile for use by clinicians for their own purposes and could also be used, after aggregation and anonymisation, for epidemiological and administrative purposes.
The HoNOSLD measures global outcomes and not inputs. It is not a rating for disability alone. The scale has items designed to measure areas of functioning that are relatively stable, and items measuring more transient phenomena that are more likely to change as a result of treatment.
Although the level of support that an individual receives will usually have a significant impact on performance, HoNOSLD rates performances with the existing support, and not the individual's potential or the cause of the disability. Since it is intended for use in assessing global changes in those undergoing treatment, interventions in any area can lead to changes in several areas of functioning. It may be that a combination of inputs would produce the best outcome for some individuals or groups of people. The HoNOSLD is able to measure no change, or deterioration, as well as improvement.
Field trials
All professional groups involved with the assessment and treatment of
people with a learning disability and mental health needs participated in the
study and the sample was drawn from a wide range of services across the
country.
Acceptability studies
Following planning meetings, questionnaires were sent out to prospective
pilot sites. Questions were asked about the content and construct validity of
HoNOSLD.
Ethical issues
Any studies involving people with learning disabilities or people with
mental health problems should be able to stand up to ethical scrutiny. Work is
sometimes carried out on behalf of people with learning disabilities that has
little or no involvement by service users
(Kiernan, 1999).
Anonymity of the people being rated was preserved through the use of a coding system. Each pilot site and each person could only be identified by their allocated codes.
Research ethics committees were not approached because the information required would have been gathered in any case, anonymity was assured, and the outcome scales themselves would be unlikely to directly affect the clinical treatment a patient or service user might receive from professionals.
Data gathering
Following training within participating NHS trusts, field trial sites were
then asked to rate about 30 subjects. Each subject would have two raters
rating simultaneously and using the same source of information (either an
informant who knew the service user well, or case notes) but without
conferring (T1). They were asked to rate the subject over the previous month.
These same raters would then rate the individual subject using the
same source of information about 3 months later in the same setting
and, again, simultaneously and without conferring (T2). At both times (T1 and
T2), raters were asked to provide a global subjective rating on the same
five-point scale.
It was agreed that ICD-10 codes (World Health Organization, 1992) be used to indicate the degree of disability and any mental illness or developmental condition (such as autism).
Raters were also asked to record any physical conditions (e.g. epilepsy, Down's syndrome, cerebral palsy).
In addition to the ratings, those involved with the field studies were given a questionnaire that asked their views on the instrument itself and the guidelines for rating each instrument. Raters were asked to complete this questionnaire by indicating on a three-point scale (where the choices were good, fairly good or poor).
Data analysis and dissemination of findings
Data analysis began in April 1998 with the completion of the collection and
collation of statistical results in July 1999. The analysis of data took place
at Brooklands, Birmingham and at the Centre for Clinical Outcomes, Research
and Effectiveness (CORE), University College London using Microsoft Access
(Microsoft, 1997a),
Excel (Microsoft,
1997b) and SPSS
(SPSS, 1999) software.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Acceptability studies
There was general agreement that the language used was appropriate and that
the instrument was easy to understand and use. The instruction points were
found to be helpful, and there was consensus that some items might need to be
added but few thought that the instrument should be shortened.
Results from the feedback questionnaire demonstrated that people involved in rating were happy with each instrument's items and the instructions accompanying them.
Demographic data
There was a preponderance of men in the population studied. The degree of
learning disability was spread across the whole spectrum. This is shown in
Table 1.
|
Profession of rater
Three hundred and sixty-four raters rated 372 individuals. The profession
of each rater is indicated in Table
2. Psychiatrists constituted over half the number at T1. At T2
there was a wider spread of professionals involved in rating.
|
The pairs of ratings were analysed to measure reliability between raters taking into account their professional backgrounds. It was found that the Pearson correlation coefficient was high (0.89-0.96) between and within all combinations of professional groups.
Legal status
Information about legal status was available for 317 individuals. Two
hundred and eighty-one (88.6%) were informal. Twenty-seven (8.5%) were
sectioned under section 3 of the Mental Health Act 1983 and a further nine
(2.8%) were under other sections such as 37 and 41.
Patient case status
Of the 372 cases, 360 (96.7%) were current cases and 12 (3.3%) were new
referrals.
Patient accommodation
Information about accommodation was available for 303 individuals. One
hundred and seventy-one (56.4%) lived in staffed group homes, 44 (14.5%) lived
in their family home and a further 13 (4.4%) lived independently. Seventy-five
people lived in hospitals, 46 (15%) in long-stay wards and 29 (9.6%) in acute
assessment and treatment units.
Physical conditions
Information about coexisting physical conditions was available for 303
individuals. The most frequently occurring conditions are listed in
Table 3.
|
Psychiatric/developmental conditions
Information about coexisting psychiatric/developmental conditions (ICD-10
diagnoses) was available for 262 individuals (70.4%). Out of this sample 32
people (12.2%) did not have a diagnosed condition. One hundred and fifty-seven
(59.9%) patients had one condition, 59 (22.5) had two conditions, 10 (3.8%)
had three and 4 (1.5%) had four coexisting psychiatric/developmental
conditions.
Table 4 indicates the frequency of all diagnosed conditions of the subject population (n=262). Ten subjects with a personality disorder also had a sexual disorder. Twenty-seven subjects (10.3%) had a behavioural diagnosis only.
|
Correlation coefficients between raters (total scores at TI)
Eighty-three per cent of the Pearson correlation coefficients
(Pearson, 1902) calculated for
the total scores of Rater 1 by Rater 2 at T1, were above the level of
r=0.7. This showed that there was a high degree of correlation
between the total scores obtained between Rater 1 and Rater 2.
Cohen's was also calculated. The
values and levels of
significance obtained for T1 and T2 between the raters for all individual item
scores show a high level of agreement in scoring between the two raters (see
Table 5). The
values
and levels of significance obtained for T2 between raters for all the
individual item scores again show a high level of agreement in scoring between
the two raters.
|
Subjective score
A Pearson correlation test was carried out to see whether the mean score
for each completed outcome measure correlated with the subjective score for
each rating.
A significant correlation was found between the mean score and the subjective score, at the 0.01 level, at both T1 and T2 for Rater 2. This analysis was completed for both raters at both T1 and T2. Significant results were obtained at the 0.01 level for both raters at both T1 and T2, indicating that the mean score correlated with the subjective score for each questionnaire irrespective of rater and time.
HoNOS-LD as a measure of change
A paired-sample t-test was completed using total scores obtained
by Rater 1 at T1 v. T2 and total scores obtained by Rater 2 at T1
v. T2. This showed that there was a significant difference between
the scores at T1 and T2 for both raters. The level of significance for these
differences was <0.001. This is shown in
Table 6.
|
There was significant correlation between the subject scores and total mean scores at T1 and T2 indicating a reliable measurement of change.
Reliability of data sources
There was a high correlation between Raters 1 and 2 (Pearson's
r=0.88) when clinical notes were used as a source of information
(n=56) and an even higher correlation between Raters 1 and 2
(Pearson's r=0.96) when informants were used as a source of
information (n=316). This indicated that using either notes or
informants provided a reliable method of data collection between two raters,
although informants were found to be more reliable.
Degree of change in mean total score
An attempt was made to determine whether the degree of learning disability
significantly affected the degree of change in mean total score between T1 and
T2.
The mean change score from each service user's total scores at T1 and T2 was calculated as follows: improvement, scores >0.06; no change, scores 0.05 to -0.05; deterioration, scores < -0.06.
There was a high correlation between change in mean total score between T1 and T2 for all degrees of disability at the 0.01 level of significance. The level of disability does not significantly affect the change in mean total score between T1 and T2, although it may be more difficult to interpret the significance of the profound group given the small numbers involved (n=13) (see Table 7).
|
Mean HoNOS-LD scores
Calculation of the mean score of each question at T1 and T2 by Rater 1
indicated that subjects tended to score highly on psycho-social items and on
Scale 1 (behaviour towards others), Scale 4 (attention and concentration) and
Scale 9 (mood disturbance). In contrast, Scale 13 (seizures) was one area
where subjects had scored relatively low, indicating that seizures were being
well controlled.
Convergence study
Two raters completed HoNOS-LD ratings and then completed the Aberrant
Behavior Checklist (ABC; Aman et
al, 1985) on the same service users at T1 and T2. In the
interim one person died (hence n=31 at T1 and n=30 at T2).
At T1 the Pearson correlation coefficient of the two ratings of ABC total
scores was 0.87. The corresponding figure for the HoNOS-LD total scores was
0.93. There was a positive correlation between ABC and HoNOS-LD (Rater 1=0.66;
Rater 2=0.76) at T1.
At T2 the interrater reliability of ABC total scores (Rater 1 v. Rater 2) was 0.81 and the interrater reliability of HoNOS-LD total scores was 0.76. Again there was a positive correlation between ABC and HoNOS-LD (Rater 1=0.71; Rater 2=0.76) at T2.
Measurement of change
The degree of change picked up by HoNOS-LD was evident in 207 service
users. This is shown in Table
8.
|
An analysis of reported improvements suggests that, where there are improvements, these are most likely to be in the areas of mood (Scale 9), behaviour towards others (Scale 1), attention and concentration (Scale 4) and the five psychosocial items (Scales 14-18). The most dramatic improvements were recorded in the areas of mood disturbance (Scale 9), activities in daily living outside the house (Scale 15), attention and concentration (Scale 4), and in relationship problems (Scale 17).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Improving the quality of a service necessitates a move from the measurement of volume to the measurement of outcomes (Lelliot, 1995). The HoNOS-LD provides clinicians with a structured and standardised approach to achieve this. It is simple to use, covers a broad range of clinical and psychosocial problems and is acceptable to clinicians. It has good interrater reliability and can measure change between two points in time. It has good correlation with well-established instruments such as the ABC. Factor analysis of HoNOS-LD shows that the instrument is useful irrespective of the degree of learning disability or gender of the person being rated.
A paired-sample t-test demonstrated that HoNOS-LD is a useful instrument in measuring change. The mean scores between T1 and T2 were demonstrated to have reduced (Rater 1 from 18.26 to 15.91 and Rater 2 18.55 to 16.14). The mean subjective scores for the same subjects had also reduced over the same period and there was a high correlation between the total HoNOS-LD scores and the subjective scores.
Analysis of the interrater reliability of the total scores of various professions demonstrated little variation in the strength of correlation coefficients against the raters' profession. Thus, HoNOS-LD can be used confidently by any service provider who has had appropriate training.
The comparison between the results of the field trials where the information source was either an informant or case notes, demonstrates the greater reliability of the use of informants.
Potential sources of bias
Difficulties with comprehension or expression may increase bias in
reporting. The information that a client gives may need corroboration. Further
bias may arise from the need to use an informant.
Some typical biases in responding include differing perspectives, social desirability, deviation and halo effects. All of these may have affected the field trials. By having a greater understanding, and by cleaning data, it has been possible to minimise the effects of these possible biases in responding.
It is critical that any rating scale asks questions in such a way as to elicit as truthful an answer as possible (i.e. as near to a reflection of reality as possible). A rating scale that does not have this validity may produce skewed data that in turn may lead to further problems, for example, the overdiagnosis of a particular condition or an inaccurate assessment of risk.
The future
Further studies are needed to determine the effectiveness of the instrument
with, for example, different age groups, people with chronic disabilities or
with people from various ethnic backgrounds.
Clinicians can avoid misusing HoNOS-LD if they receive adequate training and appreciate the strengths and limitations of the instrument. Once the gathering of relevant clinical data is performed routinely, then clinicians will find that useful and rational approaches can be found to routine measurement of treatment outcomes in this highly vulnerable group of people.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aman, M. G., Singh, N. N., Stewart, A. W., et al (1985) The Aberrant Behavior Checklist: a behavioral rating scale for the assessment of treatment effects. American Journal of Mental Deficiency, 89, 485-491.[Medline]
Ashaye, O., Mathew, G. & Dhadphale, M. (1997) A comparison of older longstay psychiatric and learning disability inpatients using the Health of the Nation Outcome Scales. International Journal of Geriatric Psychiatry, 12, 548-552.[CrossRef][Medline]
Bebbington, P., Brugha, T., Hill, T., et al (1999) Validation of the Health of the Nation Outcomes Scales. British Journal of Psychiatry, 174, 389-394.[Abstract]
Bernal, J. & Hollins, S. (1995) Psychiatric illness and learning disability: a dual diagnosis. Advances in Psychiatric Treatment, 1, 138-145.
Burns, A., Beevor, A., Lelliot, P., et al (1999) Health of the Nation Outcome Scales for Elderly People (HoNOS 65+). British Journal of Psychiatry, 174, 424-427.[Abstract]
Corbett, J. A. (1979) Psychiatric morbidity and mental retardation. In Psychiatric Illness and Mental Handicap (eds F. E. James & R. P. Snaith), pp. 11-25. London: Gaskell.
Gowers, S. G., Harrington, R. C., Whitton, A., et al (1999) Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). British Journal of Psychiatry, 174, 413-416.[Abstract]
Kiernan, C. (1999) Participation in research by people with learning disability: origins and issues. British Journal of Learning Disabilities, 27, 43-47.
Lelliot, P. (1995) Mental health information systems: problems and opportunities. Advances in Psychiatric Treatment, 1, 216-222.
Microsoft (1997a) Microsoft Access, SR-I. Redmond, WA: Microsoft Corporation.
Microsoft (1997b) Microsoft Excel, SR-I. Redmond, WA: Microsoft Corporation.
Moss, S., Prosser, H. & Goldberg, D. (1996) Validity of the schizophrenia diagnosis of the Psychiatric Assessment Schedule for Adults with Developmental Disability (PAS-ADD). British Journal of Psychiatry, 168, 359-367.[Abstract]
Patel, P., Goldberg, D. & Moss, S. (1993) Psychiatric morbidity in older people with moderate and severe learning disability (mental retardation). II: The prevalence study. British Journal of Psychiatry, 163, 481-491.[Abstract]
Pearson, K. (1902) On mathematical theory of errors of judgement with special reference to the personal equation. Philosophical Transactions of the Royal Society of London, 198, 235-299.
SPSS (1999) SPSS for Windows. Base System User's Guide, Release 9.0. Chicago, IL: SPSS Inc.
Wing, J. K., Beevor, A. S., Curtis, R. H., et al (1998) Health of the Nation Outcome Scales (HoNOS). Research and development. British Journal of Psychiatry, 172, 11-18.[Abstract]
Wing, J. K., Curtis, R. H. & Beevor, A. (1999) Health of the Nation Outcome Scales (HoNOS). Glossary for HoNOS scoresheet. British Journal of Psychiatry, 174, 432-434.[Medline]
World Health Organization (1992) The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. Geneva: WHO.
Received for publication March 29, 2000. Revision received September 19, 2000. Accepted for publication September 19, 2000.
Related articles in BJP: