VA Northeast Program Evaluation Center, West Haven and Yale University, New Haven, Connecticut
University of North Carolina, Chapel Hill, North Carolina
Duke University, Durham, North Carolina
University of North Carolina, Chapel Hill, North Carolina
National Institutes of Mental Health, Bethesda, Maryland
University of California at San Francisco, San Francisco, California
University of North Carolina, Chapel Hill, North Carolina, USA
Correspondence: Dr Robert Rosenheck, Northeast Program Evaluation Center (182), VA Connecticut Health Care System, 950 Campbell Avenue, West Haven, CT 06516, USA. Tel: +1 203 937 3850; fax: +1 203 937 3433; e-mail: Robert.Rosenheck{at}Yale.edu
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To construct and evaluate a multidimensional, preference-weighted mental health index.
Method Each of over 1200 patients identified the relative importance of improvement in six domains: social life, energy, work, symptoms, confusion and side-effects. A mental health index was created in which measures of well-being in these six domains were weighted for their personal importance.
Results The strongest preference was placed on reducing confusion and the least on reducing side-effects. There was no significant difference between the unweighted and preference-weighted mental health status measures and they had similar correlations with global health status measures. Patients with greater preference for functional activities such as work had less preference for medical model goals such as reducing symptoms and had less symptoms.
Conclusions A preference-weighted mental health index demonstrated no advantage over an unweighted index.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Although consumer choice has become an ever larger presence in clinical practice, it has made far less of a mark on research and especially on outcome assessment. Although methods for measuring health state preferences have received considerable attention in other areas of medicine, studies have tended to focus on health state evaluation by the general public rather than the preferences of individual patients (Gold et al, 1996), and with a few exceptions (Rosenheck et al, 1988; Lenert et al, 2000; Sherbourne et al, 2001) such measures have been little used in psychiatric research. Scales used to measure symptoms, side-effects and quality of life in mental health outcome research have been developed by psychometricians with little or no input from service users, and in most cases rely either on clinician ratings based on professional judgement, or on patients responses to structured questions (Guy, 1976; Heinrichs et al, 1984; Kay et al, 1987; Barnes, 1989). One measure that has been used occasionally in studies of psychosocial treatment asks participants to rate diverse features of their lives and their feelings about their life as whole on a 17 (delighted to terrible) scale (Lehman, 1988). Use of this measure has been limited, especially in the evaluation of medications.
Preference assessment is especially important in serious mental illness in which many domains of life may be affected. Whereas some patients might be especially troubled by symptoms or side-effects, others might be more concerned with employment or social relationships. As a result, two people with identical scores on a set of outcome measures might feel very differently about their lives if they had different priorities about various life domains. Although the incorporation of patient preference into outcome assessment has been neglected in clinical research, standardised methods are available that could allow systematic comparisons across participants within particular studies and allow generalisation across studies.
Our study uses baseline data from a large, multisite clinical trial to illustrate a method of quantifying patient preferences; to determine whether specific socio-demographic or clinical characteristics are associated with various preferences; to demonstrate an approach to using measured preferences to construct a preference-weighted, multidimensional mental health status index, and to evaluate the plausibility of this index by determining whether it is more strongly correlated with several measures of current global health status than an unweighted version of the same index. We thus hope to demonstrate a method for incorporating patient preferences into conventional mental health status assessment and to determine if doing so has the potential to make a difference in the ultimate interpretation of study results.
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Measures
Assessment of preferences
Preferences were assessed using a modified version of a method developed
for a previous study (Fisher et
al, 2002). Participants were first presented with a list of
goals in six domains and asked to rank them in order of importance. The six
goals, identified through focus groups with mental health service users,
were:
To assess the magnitude of these relative preferences, participants were further asked how many times more important each item was than the least important item, with a maximum value of 99. To recalibrate these preferences on a uniform scale with possible values ranging from 0.01 to 1, each magnitude assessment was divided by the largest magnitude assessment, i.e. the one associated with the top-ranked goal. The simple 16 ranking and the more nuanced preference scale, which was used in subsequent analyses, were highly correlated with each other (r=0.86, P50.0001).
Client characteristics
Questions concerning socio-demographic status documented age, ethnicity,
gender, marital and educational status, income (including both earned income
and public support payments) and days of paid employment in the past 30
days.
The diagnosis of schizophrenia was confirmed by using the Structured Clinical Interview for DSMIV (SCID; First et al, 1996) for all participants. Symptoms of schizophrenia were assessed with the rater-administered Positive and Negative Syndrome Scale (PANSS; Kay et al, 1987), which yields a total average symptom score based on 31 items rated 17 (with higher scores indicating more severe symptoms), as well as sub-scale scores that reflect positive, negative and general symptoms (Kay et al, 1987).
The HeinrichsCarpenter Quality of Life Scale (QoLS; Heinrichs et al, 1984) is a rater-administered scale that assesses overall quality of life and functioning on 22 items rated 06 (with higher scores reflecting better quality of life) and yields measures on four sub-scales that address social activity, instrumental functioning (e.g. employment, housework), use of objects and participation in activities, and intrapsychic functioning (e.g. motivation, anhedonia and empathy).
Medication side-effects were assessed with the Barnes Akathisia Rating Scale (Barnes, 1989; possible range 014), the Abnormal Involuntary Movement Scale (AIMS; Guy, 1976) for tardive dyskinesia (possible range 040) and the SimpsonAngus SimpsonAngus scale for extrapyramidal side-effects (Simpson & Angus, 1970; possible range 040).
Depression was measured with the Calgary Depression Rating Scale (Addington et al, 1996) and substance use by the Alcohol Use and Drug Use Scales (Drake et al, 1990).
Neurocognitive functioning was measured by separate test scores, described in a previous publication (Keefe et al, 2003), which were converted to z scores and combined to construct five separate scales that were themselves averaged to form an overall neurocognitive functioning scale.
The neurocognitive composite score was the average of these five sub-scale summary scores.
Global status measures
Global self-reported well-being was assessed using the single global
quality-of-life item measured on the terribledelighted
scale from the Lehman Quality of Life Interview (QoLI;
Lehman, 1988), which is also
used in the Lancashire Quality of Life Profile
(Meijer et al, 2002).
The EuroQol feeling thermometer item, in which patients are
asked to rate their health overall on a vertical scale from 0 (worst possible
health) to 100 (perfect health), was also included
(Kind, 1996). The Clinical
Global Impression scale (Guy,
1976) summarises the clinical raters assessment of mental
health status on a scale of 17, where 7 represents poorer health.
Finally, a dichotomous variable identified patients who had entered the study
during a period of exacerbation of illness, in contrast to those whose
clinical status was judged to be stable.
Analysis
Baseline characteristics of participants with complete data
(n=1281; 88%) were compared with those with missing data
(n=179; 12%) using bivariate 2 and t-tests,
followed by multivariable logistic regression to identify factors that
independently differentiated the groups. Second, paired t-tests were
used to determine the statistical significance of differences in average
preference rating for each of the six goals. Next, a series of bivariate
correlations were used to determine whether preference for some domains was
associated with preference for others. A third set of bivariate correlations
was used to identify patient characteristics that were associated with high
preferences for each of the six domains. We predicted that areas of poorer
functioning would be given higher preferences, for example that greater
symptom severity on the PANSS would be associated with greater priority for
reduced symptoms, and that poorer neurocognitive functioning would be
associated with greater preference for reducing confusion.
We then developed two mental health status indexes, one unweighted and one weighted for patient preferences. The unweighted scale was based on the average of six standardised scores representing better health on measures corresponding to each of the six preference domains. Standardised or Z scores are calculated as follows: the individual score for each participant less the mean value for the entire sample is divided by the standard deviation of the mean. The Z scores on various measures can be averaged to create measures such that a change of one unit represents a change of 1 s.d. on the component measures. In constructing these measures, social relationships were represented by the social relationship scale of the QoLS and work by the instrumental activities sub-scale of the QoLS. Energy was represented by the intrapsychic functioning scale of the QoLS, the negative symptom sub-scale of the PANSS and the Calgary depression scale, with the PANSS negative sub-scale and Calgary scores each multiplied by -1 so that higher scores consistently represented better health. Symptoms such as disturbing or unusual experiences were represented by the positive sub-scale of the PANSS, and confusion by the summary neurocognitive scale. Side-effects were represented by the average standardised scores of the Barnes scale for akathisia, the AIMS for tardive dyskinesia and the SimpsonAngus scale for extrapyramidal symptoms.
In the weighted version of the index, each of the six standardised component scores was multiplied by the preference weight on that domain for that particular individual. These individual weighted scores were then averaged and divided by the average of all the weights. Thus if all the weights were the same, the weighted index would have the same value as the unweighted index. If the areas of high current well-being are those given greater priority, the weighted index would be greater than the unweighted. If the areas of lowest current well-being are given greater priority, the weighted index would be lower than the unweighted. Paired t-tests were used to compare the six unweighted and six preference-weighted domain scores and the overall mental health status indices averaging the six scores.
To compare the plausibility of the weighted and unweighted domain measures and the two aggregate indices, we examined the correlation of the unweighted and weighted measures with the two patient-rated global measures of well-being: the CGI and the dichotomous indicator of whether or not the participant was hospitalised and/or experiencing an exacerbation of the illness.
Because we found an intriguing tendency for preferences in the domains of energy, social relations and work to be correlated, a cluster analysis was conducted to identify patients with such recovery-oriented preferences in contrast to those with more medically oriented preferences (i.e. for improvement in symptoms, confusion and side-effects). Stepwise multiple regression with forward selection was then used to identify a parsimonious set of characteristics that differentiated these two groups.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Preferences
Across the sample the strongest priorities were placed on reducing
confusion and increasing energy, and the least on social life and reducing
side-effects (Table 2). Paired
t-tests comparing average priority ratings showed significant
differences on all but one of 15 paired comparisons, indicating a clear
hierarchy of goal priorities for this sample.
|
Examination of the intercorrelation of preference ratings showed that the three goals related to functioning and recovery (social relationships, work and personal energy) were positively and significantly correlated with one another (Table 3). At the same time, concern about confusion was positively correlated with concern about both symptoms and side-effects. In contrast, correlations between the first group of recovery-oriented measures and the second group of illness or medical model measures were, for the most part, significant and negative.
|
The six columns on the right-hand side of Table 1 present bivariate correlation coefficients reflecting the association between preferences and personal characteristics.
Individual correlates of personal preferences
There were few significant correlations with the recovery-oriented
preferences. Those who were eager to improve their social lives were more
likely to be Black, were less educated and had lower neurocognitive
functioning scores. Those who were eager to work had less disability income,
fewer positive symptoms, less depression and akathisia as well as higher
scores on the QoLS, especially the intrapsychic functioning sub-scale. It is
notable that those who put a high preference on work did not work any more
days than others and scored no higher on the instrumental role functioning
sub-scale of the QoLS (see Table
1). A preference for having more energy was associated with less
depression and drug use (see Table
2).
Preference ratings that put greater emphasis on either reducing confusion or symptoms were correlated with several of the same personal characteristics. Black participants were more concerned with symptoms, whereas participants in rehabilitation were concerned with both confusion and symptoms, as were those with more severe psychopathology as measured by both more severe positive symptoms and depression, and lower quality-of-life scores. Alcohol use was also associated with greater concern with symptoms. Unexpectedly, poorer neurocognitive functioning was not associated with greater priority about reducing confusion. Curiously, preference for reduced side-effects was not associated with severity of side-effects on any measure, but was associated with greater age, 12 years of education, less depression and poorer neurocognitive functioning.
Clearer and more consistent patterns emerge between preferences and global assessments of well-being or clinical status. Taken together, greater well-being, especially as measured on the EuroQol 100-point scale, was associated with greater interest in social relations, work and personal energy, and with less interest in symptoms and confusion. Being less well off on all four global health instruments was association with greater concern with symptoms, and (in the case of the Lehman QoLI scale and the EuroQol item) with greater concern with confusion. Higher Lehman QoLI scores were associated with greater concern with side-effects, suggesting that side-effects side-effects may not be seen as a priority until a basic level of well-being has been established. On the other hand, concern with side-effects was also associated with exacerbation of illness.
Weighted health status measure
Comparison of six unweighted and six weighted domain scores revealed
significant difference only in the symptom domain score: unweighted mean 0.0
(s.d.=1), weighted mean -0.044 (s.d.=0.67); t=2.7, P=0.007.
The overall unweighted mental health index, that is the average of the six
z-scored outcome domain measures (mean 0.00, s.d.=0.52) was not
significantly different from the preference-weighted mental health index (mean
0.024, s.d.=1.14; t=1.30, P=0.19). The unweighted and
weighted indices were highly correlated with one another (r=0.94,
P<0.0001). They were also significantly related to the global
measures of well-being and clinical status
(Table 4). Counter to our
expectation, however, the magnitude of correlations between unweighted
measures and measures of global well-being and clinical status were slightly
greater than those of the weighted measures
(Table 4).
|
Recovery orientation
Cluster analysis using the six preference measures revealed a recovery
cluster (n=666; 52%), in which participants had higher preferences
for improving social relationships, work and personal energy, and a medical
model cluster (n=615, 48%), in which participants had higher
preferences for improving symptoms, confusion and side-effects. Stepwise
regression showed that members of the recovery cluster could be parsimoniously
differentiated by three measures: they had higher well-being scores on the
EuroQol, greater total income, and lower positive sub-scale scores on the
PANSS (model r2=0.05).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Recovery-oriented v. medical model preferences
The most consistent pattern of relationships was found between preferences
and global measures of well-being and of clinical status. On these measures
those who were best off were most interested in recovery-oriented goals such
as improved social relationships, employment and personal energy, and those
with the most problems were more concerned with symptoms, confusion or
side-effects. Although there has been great emphasis recently on the
development of recovery attitudes or models of care, we know of only one other
empirical study of correlates of recovery attitudes
(Resnick et al,
2004), which it also found severity of psychopathology
especially depression to be inversely related to the strength of
recovery orientation.
Effect of preference-weighting
Our preference-weighted multidimensional mental health index was not
significantly different from a version of the index that was not weighted for
preferences, and this no doubt reflects the fact that domain preferences were
not, for the most part, associated with status in each domain. If, as we had
predicted, the areas of lowest current well-being had been the areas to which
participants gave the greatest priority, the weighted index would have been
smaller than the unweighted index. In the absence of such correlations, the
preference-weighted index was not much different from the unweighted index and
showed similar (and even slightly weaker) correlations with both
domain-specific and global measures of well-being. Efforts to weight
preferences in other areas have similarly found that weighting did not
increase the validity of the assessment
(Mikes & Hulin, 1968;
Trauer & Mackinnon, 2001). Some have speculated that importance is already embedded in such ratings; for
example, people who are more distressed by their symptoms or side-effects will
discuss them in such a way that they will be given higher scores, or will
report more distress on a self-report measure. However, had this been the case
we would have expected to have seen stronger correlations between preferences
and healthy state measures.
The fact that the expressed preferences of participants in this study were largely unrelated to their health status in specific domains suggests that their understanding of the descriptions of the six preference categories did not correspond precisely to what is measured by psychometric tests, perhaps because the assessments were based on observer ratings rather than self-report data or because preferences concern future health states rather than current ones. For example, priority for improving social relationships was greatest among those with poorer neurocognitive functioning rather than among those with the poorest social relationships, and preference for going to work was greatest among those with less depression and akathisia and superior intrapsychic functioning, not among those who worked least or had worse intrapsychic functioning. Thus, although our analyses did not generate a superior measure of health status, they did highlight illuminating associations with personal preferences, and consistently demonstrated that recovery-oriented preferences were consistently associated with global well-being. This result was confirmed by the results of our cluster analysis and subsequent comparison of the recovery-oriented and medical model-oriented patients. When the CATIE study is completed it will be possible to determine whether longitudinal improvement results in changes in preferences. These cross-sectional data suggest that as individuals with severe symptoms improve, their priorities priorities may shift towards more recovery-oriented goals.
Limitations
Several methodological limitations require comment. First, the range of
preference domains that were addressed was limited to six pre-established
domains. Some respondents may well have had other areas that were of even
greater importance that were not encompassed in our limited framework. In
addition, we do not know how well respondents understood the brief
descriptions of the six domains. Qualitative debriefing on how they
experienced the preference assessment, how they understood the individual
items and why they placed priority on some rather than others would have been
informative. In addition, we do not know how representative the CATIE sample
is or how generalisable our results are.
Although we have shown that it is possible to elicit outcome preferences from patients with schizophrenia, we found these preferences to be only weakly associated with patient characteristics and there was no substantive difference between unweighted and preference-weighted mental health status measures. Patients who put a higher preference on recovery-oriented activities had better functioning and had less symptoms than those who put a higher preference on medical model goals such as reducing symptoms, confusion and side-effects. It thus appears that the recovery and medical models are not in opposition to one another. Rather, effective treatment of symptoms, confusion and side-effects, in themselves, may help foster a recovery orientation, although additional formal and informal services such as supported employment and peer support are likely to be needed.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Anthony, W. A. (1993) Recovery from mental illness: the guiding vision of the mental health service system in the 1990s. Psychosocial Rehabilitation Journal, 16, 11-24.
Barnes, T. R. (1989) A rating scale for drug-induced akathisia. British Journal of Psychiatry, 154, 672 -676.[Abstract]
Drake, R. E., Osher, F. C., Noordsy, D. L., et al (1990) Diagnosis of alcohol use disorders in schizophrenia. Schizophrenia Bulletin, 16, 57-67.[Medline]
First, M. B., Spitzer, R. L., Gibbon, M. B., et al (1996) Structured Clinical Interview for Axes I and II DSM IV Disorders Patient Edition (SCID-I/P). New York: Biometrics Research Institute, New York State Psychiatric Institute.
Fisher, E. P., Shumway, M. & Owen, R. R.
(2002) Priorities of consumers, providers and family members
in the treatment of schizophrenia. Psychiatric
Services, 53, 724
-730.
Frese, F. J. & Davis, W. W. (1997) The consumer-survivor movement, recovery, and consumer professionals. Professional Psychology: Research and Practice, 28, 243 -245.[CrossRef]
Gold, M. R., Siegel, J. E., Russell, L. B., et al (1996) Cost Effectiveness in Health and Medicine. New York: Oxford University Press.
Grob, G. N. (1991) From Asylum to Community. Princeton: Princeton University Press.
Guy, W. (1976) Abnormal involuntary movements. In ECDEU Assessment Manual for Psychopharmacology (ed. W. Guy). DHEW No. ADM 76-338. Rockville, MD: National Institute for Mental Health.
Heinrichs, D. W., Hanlon, E. T. & Carpenter, W. T. (1984) The quality of life scale: an instrument for rating the schizophrenic deficit syndrome. Schizophrenia Bulletin, 10, 388 -398.[Medline]
Kay, S. R., Fiszbein, G. & Opler, D. R. (1987) The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13, 261 -276.[Medline]
Keefe, R. S., Mohs, R. C., Bilder, R. M., et al (2003) Neurocognitive assessment in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) project schizophrenia trial: development, methodology, and rationale. Schizophrenia Bulletin, 29, 45-55.[Medline]
Kind, P. (1996) The EuroQol instrument: an index of health-related quality of life. In Quality of Life and Pharmacoeconomics (ed. B. Spilker), pp. 191 -203. Philadelphia: Lippincott-Raven.
Lehman, A. (1988) A quality of life interview for the chronically mentally ill. Evaluation and Program Planning, 11, 51 -62.[CrossRef]
Lenert, L. A., Ziegler, J., Lee, T., et al (2000) Differences in health values among patients, family members, and providers for outcomes in schizophrenia. Medical Care, 38, 1001 -1021.
Meijer, C. J., Schene, A. H. & Koeter, M. W. (2002) Quality of life in schizophrenia measured by the MOS SF36 and the Lancashire Quality of Life Profile: a comparison. Acta Psychiatrica Scandinavica, 105, 293 -300.[CrossRef][Medline]
Mikes, P. S. & Hulin, C. L. (1968) Use of importance as a weighting component of job satisfaction. Journal of Applied Psychology, 52, 394 -398.[Medline]
Presidents New Freedom Commission on Mental Health (2003) Achieving the Promise: Transforming Mental Health Care in America. Subcommittee on Acute Care Report. DHHS Pub. No. SAM-03-3832. Rockville, MD: National Institute for Mental Health.
Resnick, S., Rosenheck, R. A. & Lehman, A.
(2004) An exploratory analysis of correlates of recovery.
Psychiatric Services,
55, 540
-547.
Rosenheck, R. A., Cramer, J., Xu, W., et al (1998) Multiple outcome assessment in a study of the cost-effectiveness of clozapine in the treatment of refractory schizophrenia. Health Services Research, 33, 1235 -1259.
Sherbourne, C., Unutzer, J., Schoenbaum, M., et al (2001) Can utility-weighted health-related quality-of-life estimates capture health effects of quality improvement for depression? Medical Care, 39, 1246 -1259.[CrossRef][Medline]
Simpson, G. M. & Angus, J. W. S. (1970) A rating scale for extrapyramidal side effects. Acta Psychiatrica Scandinavica Supplementum, 212, 11 -19.[Medline]
Solomon, P. (2004) Peer support/peer provided services: underlying processes, benefits, and critical ingredients. Psychiatric Rehabilitation Journal, 27, 392 -401.[Medline]
Stroup, T. S., McEvoy, J. P., Swartz, M. S., et al (2003) The National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) project: schizophrenia trial design and protocol development. Schizophrenia Bulletin, 29, 15 -31.[Medline]
Trauer, T. & Mackinnon, A. (2001) Why are we weighting? The role of importance ratings in quality of life measurement. Quality of Life Research, 10, 579 -585.[CrossRef][Medline]
Received for publication September 29, 2004. Revision received January 6, 2005. Accepted for publication January 18, 2005.
Related articles in BJP:
HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Psychiatric Bulletin | Advances in Psychiatric Treatment | All RCPsych Journals |