Healthcare Technology Systems Inc., Madison, Wisconsin, USA
Imperial College and King's College London, UK
Department of Psychiatry, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
Healthcare Technology Systems Inc., Madison, Wisconsin, USA
Correspondence: Dr James C. Mundt, Healthcare Technology Systems, Inc., 7617 Mineral Point Road, Suite 300, Madison, WI 53717, USA
Declaration of interest The copyright in WSAS is owned by I. M. M. Financial support from Pfizer, Inc. (see Acknowledgements).
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To evaluate the reliability and validity of the Work and Social Adjustment Scale (WSAS).
Method Data from two studies were analysed. Reliability analyses included internal scale consistency, testretest and parallel forms. Convergent and criterion validities were examined with respect to disorder severity.
Results Cronbach's measure of internal scale consistency
ranged from 0.70 to 0.94. Testretest correlation was 0.73. Interactive
voice response administrations of the WSAS gave correlations of 0.81 and 0.86
with clinician interviews. Correlations of WSAS with severity of depression
and obsessivecompulsive disorder symptoms were 0.76 and 0.61,
respectively. The scores were sensitive to patient differences in disorder
severity and treatment-related change.
Conclusions The WSAS is a simple, reliable and valid measure of impaired functioning. It is a sensitive and useful outcome measure offering the potential for readily interpretable comparisons across studies and disorders.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Depression study
The first study was a 7-month longitudinal study of patients being treated
for depression (Mundt et al,
2001). The WSAS was completed by 380 patients at the start of
treatment with antidepressant medication and up to three times subsequently,
at intended follow-up intervals of 4, 12 and 30 weeks. Follow-up assessments
were obtained from 217 patients 21-66 days after starting treatment (mean
29.3, s.d. 5.5 days), from 208 of them 70-118 days later (mean 86.2, s.d. 5.9
days) and from 189 patients 174-248 days later (mean 211.8, s.d. 7.7
days).
OCD study
The second study was a randomised, controlled trial of computer-assisted
behaviour therapy for obsessivecompulsive disorder (OCD)
(Greist et al, 2002).
The WSAS was administered at an initial study screening (n=197) and
on four subsequent occasions (at treatment randomisation and at 2, 6 and 10
weeks later). The WSAS was completed by 190 participants 6-26 days after
screening (treatment randomisation; mean 14.8, s.d. 2.5 days); 174 completed
the WSAS 22-41 days after screening (mean 29.9, s.d. 3.5 days), 164 did so
35-70 days after screening (mean 57.4, s.d. 5.4 days) and 150 subjects did so
69-98 days after screening (mean 85.9, s.d. 5.5 days).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Scale reliability
Cronbach's measure of internal consistency, which may be
conceptualised as the mean of all possible split-half correlations
(Cortina, 1993), was used to
assess the internal consistency of the WSAS.
Table 1 provides conservative
estimates of the internal consistency of the items comprising the WSAS,
presenting the distribution of
across different follow-up points in
both studies. Rather than aggregating ratings both between and within
subjects, each participant contributed a single set of ratings to each
estimate. Alpha coefficients of 0.75 or greater are conventionally regarded as
evidence of acceptable internal scale consistency
(Cortina, 1993). A monotonic
rise in internal consistency over time suggests that reliability increased.
All nine principal component analyses (varimax rotation) of the data subsets
listed in Table 1 extracted a
single factor with eigenvalues ranging from 2.73 to 4.05 (54.6% to 81.0% of
the total variance). Individual item factor loadings ranged from 0.66 to
0.93.
|
Testretest
Testretest reliability was examined using data from the OCD study. A
mean of 2 weeks elapsed between the WSAS rating at screening and that at
randomisation; no treatment was given during this interval. The mean change on
the YaleBrown Obsessive Compulsive Scale (YBOCS;
Goodman et al, 1989)
between screening and treatment randomisation was less than 0.1 point. The
testretest correlation for the total WSAS score was 0.73, and the
correlations for each item separately were 0.75, 0.70, 0.72, 0.71 and
0.70.
Alternative forms
In addition, the OCD study obtained WSAS ratings by both IVR and a trained
clinician from 90 participants at treatment randomisation, and from 72 of
these after 10 weeks of treatment. Clinician and IVR scores gave a correlation
of 0.81 (P<0.001) at the start of treatment and 0.86
(P<0.001) after 10 weeks of treatment. The mean WSAS scores
obtained by the IVR system were 20.7 (s.d. 8.0) at the start of treatment and
16.6 (s.d. 9.1) at the end. Mean clinician WSAS scores were 21.0 (s.d. 8.0) at
the start and 16.3 (s.d. 8.6) at the end. Change in scores for both modes of
administration were significantly correlated (r=0.61,
P<0.001), and reflected significant clinical improvement during
study participation: the mean IVR score change was 4.0, s.d. 7.5 (paired
t=4.44, d.f.=69, P<0.001); the mean clinician score
change was 4.6, s.d. 6.8 (paired t=5.7, d.f.=69,
P<0.001).
Scale validity
Convergence with disorder severity
Measurement reliability establishes the reproducibility of stable ratings,
permitting meaningful data collection and analysis, but it is not sufficient
to permit interpretation of the data or valid conclusions to be drawn. The
WSAS is designed to measure functional impairment attributable to an
identified problem or disorder. In the depression study, an abbreviated
eight-item Hamilton Rating Scale for Depression (HRSD;
Hamilton, 1960) administered by
IVR was converted to a 17-item HRSD equivalence score and used to measure
depression severity (Mundt et al,
1998). Across 994 conjoint administrations, the correlation
between the HRSD and WSAS was 0.76 (P<0.001). Correlations across
the four assessment points of the study were 0.63 (P<0.001), 0.73
(P<0.001), 0.77 (P<0.001) and 0.75
(P<0.001) respectively. In the OCD study, an IVR-administered
YBOCS was used to assess severity. Across all 875 conjoint
administrations, the correlation of YBOCS and WSAS scores was 0.61
(P<0.001). Over the five assessment periods of the study the
correlations were 0.45 (P<0.001), 0.48 (P<0.001), 0.56
(P<0.001), 0.69 (P<0.001) and 0.69
(P<0.001) respectively. In both studies, correlation between
symptom severity and WSAS increased over time probably reflecting
truncated distributions and diminished variances of symptom severity scores at
baseline.
Criterion discrimination
The significant association between symptom severity and functional
impairment is evidence of valid measurement properties. To examine further
criterion validity of the WSAS, participants' HRSD scores in the depression
study were stratified by symptom severity: an HRSD score of 18 or over was
classed as moderate to severe (n=422), between 7 and 18 as mild to
moderate (n=382) and 7 or less as subclinical (n=190). The
mean WSAS scores of functional impairment for these categories were 25.0 (s.d.
7.6), 15.5 (s.d. 7.5) and 6.5 (s.d. 6.9) respectively, reflecting significant
differences between groups (F=438, d.f.=2, 991, P<0.001).
Bonferroni-adjusted post hoc comparisons between all three groups
indicated statistically significant differences between all three strata.
Similarly, the YBOC ratings of participants in the OCD study were stratified by severity: YBOC scores of 16 or over were classed as moderate to severe (n=783), between 10 and 16 as mild to moderate (n=68) and 10 or less as subclinical (n=24). The mean WSAS scores for these categories were 20.6 (s.d. 7.5), 10.7 (s.d. 6.4) and 5.1 (s.d. 4.0) respectively, again reflecting significant differences between groups (F=103, d.f.=2, 872, P<0.001). As in the first study, Bonferroni-adjusted post hoc comparisons between each of the groups were statistically significant.
Convergence with perceived improvement
Patients in both studies also provided global impressions of perceived
clinical improvement by IVR at each follow-up. Ratings of perceived global
improvement (PGI) ranged from very much improved (1) to
very much worse (7), with a rating of 4 indicating no change. In
the depression study, 365 PGI ratings of significant clinical improvement (1
or 2) were given. The mean WSAS score associated with these ratings was 10.8
(s.d. 8.8), which is significantly lower than the mean score of 22.4 (s.d.
8.5) for the 247 ratings indicating little or no improvement (t=16.2,
d.f.=610, P<0.001). In the OCD study, the 117 PGI ratings of 1 or
2 had a mean WSAS score of 11.5 (s.d. 7.5), also significantly lower than the
mean WSAS score of 20.2 (s.d. 7.6) for the 561 ratings of little or no
improvement (t=11.2, d.f.=676, P<0.001).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Functional impairment between disorders
Very similar results across two DSM-IV disorders for discriminating between
patients categorised by symptom severity suggest that the WSAS may be a
valuable measure for making comparisons between disorders. A WSAS score above
20 appears to suggest moderately severe or worse psychopathology. Scores
between 10 and 20 are associated with significant functional impairment but
less severe clinical symptomatology. Scores below 10 appear to be associated
with subclinical populations. Whether such a pattern will generalise to other
disorders remains to be tested.
Patients' perspectives
Its simplicity, strong psychometric properties and direct applicability to
a wide range of clinical problems indicate that the WSAS has greater potential
for contributing to epidemiological, service utilisation and clinical trial
research than has been realised to date. While disorder-specific symptoms are
the observable elements defining differential diagnoses, the experiential
impact of a disorder from the patient's point of view is the manner in which
it impairs the ability to function day to day. As interest continues to expand
in measuring and monitoring changes in patient impairment, in addition to
symptoms, a common scale for making comparisons across and between different
disorders and treatment alternatives would be extremely valuable.
Alternative data collection methods
One aspect of the two studies reported above that could limit
generalisability of these WSAS results is the use of IVR technology to
administer the scale and collect data. This fact notwithstanding, the WSAS
scores obtained by clinicians at the beginning and end of treatment for the
subset of patients in the OCD study assessed by both methods were highly
convergent with those obtained using IVR. In addition, paper and
pencil versions of the WSAS have been used as self-report
questionnaires in an ongoing study of alcohol dependency by the first author
(J.C.M.) and in another study of patients with other mood or anxiety disorders
by the third author (M.K.S.); the psychometric properties of the WSAS
administered in this way remain strong (Cronbach's 0.90). In a
study (M.K.S.) that included 108 patients with mood and/or anxiety disorders
and 22 normal control subjects, the contribution of different symptoms to
impaired functioning could be discriminated using the WSAS. Further details on
these three studies are available from the author upon request.
WSAS used with other populations
Use of the WSAS to date has been limited to self-reported impairment in
patients with depression, anxiety or alcohol misuse disorders. No information
regarding the potential of this instrument to assess functional impairment in
patients with psychotic features (such as schizophrenia or bipolar disorder)
is available. With increasing recognition of the importance of functional
impairments and disabilities associated with psychotic symptomatology for
treatment planning and outcome prediction, research with the WSAS in these
patient populations is warranted.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
APPENDIX |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Contact Dr Marks at SSHC, 303 North End Road, London WI4 9NS, UK for permission to use the WSAS in research without charge.
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Cortina, J. M. (1993) What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104.[CrossRef]
Endicott, J., Nee, J., Harrison, W., et al (1993) Quality of life enjoyment and satisfaction questionnaire: a new measure. Psychopharmacology Bulletin, 29, 321-326.[Medline]
Frisch, M. B., Cornell, J., Villanueva, M., et al (1998) Clinical validation of the quality of life inventory: a measure of life satisfaction for use in treatment planning and outcome assessment. Psychological Assessment, 4, 92-101.
Goodman, W. K., Price, L. H., Rasmussen, S. A., et al (1989) The YaleBrown Obsessive Compulsive Scale, II. Archives of General Psychiatry, 46, 1011-1016.
Greist, J. H., Marks, I. M., Baer, L., et al (2002) Behavior therapy for obsessive compulsive disorder guided by a computer or by a clinician compared with relaxation as a control. Journal of Clinical Psychiatry, in press.
Hamilton, M. (1960) A rating scale for depression. Journal of Neurology, Neurosurgery, and Psychiatry, 23, 56-62.
Marks, I. (1986) Behavioural Psychotherapy. Bristol: John Wright [now published by I. Marks, Institute of Psychiatry, London].
Mundt, J. C. (1997) Interactive voice response (IVR) systems in clinical research and treatment. Psychiatric Services, 48, 611-623.[Medline]
Mundt, J. C., Kobak, K. A., Taylor, L. V. H., et al (1998) Administration of the Hamilton Depression Rating Scale using interactive voice response technology. MD Computing, 15, 31-39.[Medline]
Mundt, J. C., Clarke, G. N., Burroughs, D., et al (2001) Effectiveness of antidepressant pharmacotherapy: the impact of medication compliance and patient education. Depression and Anxiety, 13, 1-10.[CrossRef][Medline]
Saintfort, F., Becker, M. & Diamond, R. (1996) Judgments of quality of life of individuals with severe mental disorders: patient self-report versus provider perspectives. American Journal of Psychiatry, 153, 497-502.[Abstract]
Ware, J. E. (1993) SF-36 Health Survey Manual and Interpretation Guide. Boston, MA: Nimrod Press.
Received for publication June 5, 2001. Revision received November 8, 2001. Accepted for publication November 14, 2001.