School of Law, University of Virginia, Charlottesville, VA 22903
Policy Research Associates, Delmar, NY 12054
Department of Sociology, The Pennsylvania State University, College Park, PA 16802
Department of Psychiatry, University of Massachusetts Medical Center, Worcester, MA 01655
Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA
![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
See editorial pp.
307311, this issue
Correspondence: John Monahan, PdD, School of Law, University of Virginia, 580 Massie Road, Charlottesville, VA 22903-1789, USA. Tel: (804) 924 3632; Fax: (804) 982 2845; e-mail: jmonahan{at}law5.law.virginia.edu
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To increase the clinical utility of the ICT method by restricting the risk factors used to generate the actuarial tool to those commonly available in hospital records or capable of being routinely assessed in clinical practice.
Method A total of 939 male and female civil psychiatric patients between 18 and 40 years old were assessed on 106 risk factors in the hospital and monitored for violence to others during the first 20 weeks after discharge.
Results The ICT classified 72.6% of the sample as either low risk (less than half of the sample's base rate of violence) or high risk (more than twice the sample's base rate of violence).
Conclusions A clinically useful actuarial method exists to assist in violence risk assessment.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
ITERATIVE CLASSIFICATION TREE METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In addition to its tree-based character, our approach acknowledges the practical impossibility of adequately classifying all persons into a high or a low violence risk group. Therefore, rather than relying on the standard single threshold for distinguishing among cases, our approach employs two thresholds: one for identifying high-risk cases and one for identifying low-risk cases. We assume that inevitably there will be cases that fall between these two thresholds, cases for which any prediction scheme is incapable of making an adequate assessment of high or low risk. Based on current knowledge, the aggregate degree of risk presented by these intermediate cases cannot be distinguished statistically from the base rate of the sample as a whole.
![]() |
CLINICAL UTILITY |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
This article applies the ICT method to the sample of patients assessed in the MacArthur Violence Risk Assessment Study (Steadman et al, 1998). Our goal is to increase the clinical utility of this actuarial method by restricting the risk factors tested to those commonly available in hospital records or capable of being assessed routinely in clinical practice.
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Sample description
We approached a quota sample (to ensure representativeness across sites on
gender, race, and age) of 1695 to participate. The refusal rate was 29%
(n=492). The final sample given a hospital interview was 1136.
Differences between the eligible admissions and the follow-up sample
(n=939) are discussed in detail elsewhere
(Steadman et al,
1998). Males comprised 57.3% of the sample. Ethnically, 68.7% of
the sample was White, 29.1% African American and 2.2% Hispanic. The mean age
was 29.9 (s.d.=6.2) years. Depression was the most frequent primary research
diagnosis on the DSM-III-R Checklist
(Janca & Helzer, 1990;
41.9%), followed by alcohol/drug abuse or dependence (21.8%), schizophrenia
(17.0%), bipolar disorder (14.1%), personality disorder only (2.1%) and other
psychotic disorder (3.1%). The proportion of all cases with a primary research
diagnosis of major mental disorder that had a co-occurring diagnosis of
substance abuse or dependence was as follows: depression, 49.6%;
schizophrenia, 41%; bipolar disorder, 37.7%; and other psychotic disorder,
45%.
Hospital data collection
Hospital data collection was conducted in two parts: an interview by a
research interviewer to obtain data on risk factors and violence; and an
interview by a research clinician (PhD or MA/MSW in psychology or social work)
to confirm the chart diagnosis using the DSM-III-R Checklist and to administer
several clinical instruments.
The hospital data set assembled in the MacArthur Violence Risk Assessment Study consisted of 134 risk factors from four conceptual domains: dispositional or personal factors (e.g. age); historical or developmental factors (e.g. child abuse); contextual or situational factors (e.g. social networks); and clinical or symptom factors (e.g. delusions) (Steadman et al, 1994). For the present analysis, we eliminated 28 risk factors that would be the most difficult to obtain in clinical practice, restricting ourselves to the remaining 106. Two criteria were used to eliminate risk factors. The first was to eliminate information generally unavailable to mental health personnel in the context of brief hospitalisation (e.g. information in official arrest records, in distinction to self-report of prior arrests). The second was to eliminate information that required the administration of a lengthy (>12-item) instrument to obtain (e.g. a social network inventory (Estroff & Zimmer, 1994)). A list of all 134 risk factors, with their bivariate correlations with violence and with an indication of which were eliminated from these analyses, is provided in Table 1.
|
Community data collection
Twenty weeks after hospital discharge was chosen as the time frame for the
analysis here because this was the period during which the prevalence of
violence by patients in the community was at its highest
(Steadman et al,
1998). Research interviewers attempted two follow-up interviews
with enrolled patients in the community during this period, approximately 10
weeks apart. A collateral informant who knew of the patient's behaviour in the
community during the follow-up period - usually, but not always, a family
member - was also interviewed on the same schedule. Arrest and
re-hospitalisation records provided the third source of information about the
patients' behaviour in the community.
Patients and collaterals independently were asked whether the patient had been involved in any of several categories of violent behaviour in the past 10 weeks (Lidz et al, 1993). Only the most serious act for each discrete incident was coded. Violence to others was defined to include the following: acts of battery that resulted in physical injury; sexual assaults; assaultive acts that involved the use of a weapon; or threats made with a weapon in hand. (Battery that did not result in injury was defined as other aggressive act (Steadman et al, 1998) and is not considered in the analyses reported here.) Violence reported by any of the three data sources - subject self-report, collateral report, or official records - was reviewed by a team of trained coders. Ethical and legal issues encountered in conducting this research are discussed elsewhere (Monahan et al, 1994).
Developing the classification tree
To develop the ICT model, we used CHAID (chi-squared automatic interaction
detector) software (SPSS,
1993). Specifically, the CHAID algorithm was used to assess the
statistical significance of the bivariate association between each of the 106
eligible risk factors and the dichotomous outcome measure - violence in the
community - until the most statistically significant value of
2 was identified, with P < 0.05 a necessary
condition for risk factor selection. Once a risk factor was selected, the
sample was partitioned according to the values of that risk factor. This
selection procedure was then repeated for each of the sample partitions, thus
further partitioning the sample. The result of the partitioning process was to
identify groups of cases that shared the same risk factors and that also
shared the same values on the outcome measure of violence.
Iterating the classification tree
We then extended this recursive partitioning approach in an iterative
fashion. That is, all subjects not classified into groups designated as either
high risk or low risk in the first iteration of CHAID were pooled together and
re-analysed in a second iteration of CHAID. This iterative process continued
until it was not possible to classify any additional groups of subjects as
either high or low risk (with no group allowed to contain fewer than 50
cases).
Choosing two cut-offs
The choice of cut-off scores for high-risk and low-risk categories must be
made in the context of legal or policy values external to the methodology
chosen for assessing risk. Here, for illustrative purposes, we defined any
group of patients with a rate of violence that was less than half the
base prevalence rate of the total sample, as in the low-risk category, and any
group of patients whose rate of violence was greater than twice the
base prevalence rate of the total sample, as in the high-risk category.
Because the base prevalence rate of violence during the first 20 weeks after
hospital discharge for the total sample was 18.7% (i.e. 18.7% of the patients
committed at least one violent act during either the first or second 10-week
follow-up period), this meant that the cut-off for the low-risk category was
9% violent and the cut-off for the high-risk category was 37% violent.
The ICT contained three iterations (Fig. 1). In the first iteration, the tree classified 429 of the 939 subjects (45.7%) into either the high- or the low-risk categories. In the second iteration, the tree classified as high- or low-risk 167 (32.7%) of the 510 subjects who were not classified into either high- or low-risk groups at the end of iteration 1. In the third iteration, the tree classified as high- or low-risk 86 of the 343 subjects (25.1%) who were unclassified at the end of iteration 2. At the end of iteration 3, no further groups could be classified as high- or low-risk, given the parameters of the model we had set (e.g. no group with fewer than 50 cases); 257 subjects (27.4% of the total sample) remained unclassified. The final ICT contained 15 contingent risk factors that formed 11 risk groups (four low-risk groups, accounting for 50.9% of the total sample; three high-risk groups, accounting for 21.7% of the total sample; and four unclassified risk groups, accounting for 27.4% of the sample).
|
The risk factors displayed in Fig. 1 are defined as follows. Seriousness of prior arrests was a patient's self-report of the seriousness of arrests since age 15 years. Motor impulsiveness was measured from the motor sub-scale of the Barratt Impulsiveness Scale (Barratt, 1994). Father used drugs was a self-report question on whether the patient's father ever used drugs excessively. Recent violent fantasies was measured by the Schedule of Imagined Violence (Grisso et al, 2000). Major disorder without substance abuse refers to a diagnosis of any major mental disorder without any co-occurring substance abuse diagnosis, as reached by research clinicians using the DSM-III-R Checklist. Legal status was the initial status for the baseline hospitalisation, as recorded in hospital admission records. Schizophrenia was the diagnosis reached by research clinicians using the DSM-III-R Checklist. Anger reaction was measured by a short version of the Behavioural Subscale of the Novaco Anger Scale (Novaco, 1994). Employed was a self-report question regarding the patient's paid full- or part-time employment status in the two months prior to hospital admission. Recent violence was a self-report of violence in the two months prior to hospital admission. Loss of consciousness referred to a self-report of any loss of consciousness due to head injury. Parents fought was a self-report by the patient that his or her parents engaged in physical fights with one another when the patient was growing up. (A complete list of the questions comprising these risk factors is available from the first author upon request.)
Receiver operating characteristic
To assess the predictive accuracy of the actuarial model produced by this
method and to facilitate further comparisons of our results with other
research on violence risk assessment, we used a receiver operating
characteristic (ROC) analysis (Gardner
et al, 1996; Quinsey
et al, 1998). The statistic used to summarise the
analysis is the area under the ROC curve, which corresponds to the probability
that a randomly selected violent patient will have been assessed by the risk
assessment tool as higher risk than a randomly selected non-violent patient
(Swets, 1988). The area under
the ROC curve for the 11 risk groups presented in
Fig. 1 is 0.80
(P<0.001). The distribution of cases that were violent or not
violent during the follow-up as a function of the low- and high-risk cut-offs
used to generate the ICT is presented in
Table 2.
|
Bootstrapping
We did not cross-validate the ICT. Cross-validation of a risk assessment
model requires estimating the model on a subset of the data and validating the
model on the rest. As noted by Gardner et al
(1996), however,
cross-validation "wastes information that ought to be used estimating
the model" (p 43). For this reason, bootstrapping
(Mooney & Duval, 1993) has
become a widely used analytical strategy for estimating the shrinkage to be
expected when a model is generalised to a sample other than the one on which
it was estimated. In conducting such an analysis, 1000 bootstrapped samples
were drawn from the original data set.
Table 3 presents the 95%
confidence intervals for each of the 11 risk groups in the ICT, in order of
decreasing risk. The ranges of these intervals indicate how the ICT is likely
to perform on other similar samples.
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The ICT left 27.4% of the total sample unclassified, meaning that it could find no combination of risk factors that allowed these patients to be classified into either a low- or a high-risk group. The violence rate for the unclassified category was 24.1%.
Clinical illustrations
Illustrating the use of the ICT may be helpful. A clinician evaluating a
patient's risk of violence using the ICT presented in
Fig. 1. would first ask the
patient about the seriousness of his or her prior arrests. If the patient
stated that he or she had previously been arrested for a violent crime, the
clinician would then inquire into whether the patient recently had been
fantasising about being violent. If the patient responded affirmatively to
this second question, he or she at that point would be placed in the high
violence risk category. More specifically, the patient would be placed in risk
group B, a group in which approximately 53% of the patients are expected to
commit a violent act in the next several months.
If, on the other hand, the patient denied having violent fantasies, the clinician would then indicate whether the patient had a diagnosis of schizophrenia. If the patient did have such a diagnosis, he or she at that point would be placed in the low violence risk category. More specifically, the patient would be placed in risk group E, a group in which approximately 7% of the patients are expected to commit a violent act in the next several months. (For other studies finding rates of violence to be lower among patients with schizophrenia than among patients with other, primarily personality disorder, diagnoses, see: Gardner et al, 1996; Quinsey et al, 1998; Wallace et al, 1998.)
Comparative predictive accuracy
We have demonstrated here that the ICT method may be adapted for clinical
use. The method does not require unavailable or costly-to-gather data in order
to characterise the risk of violence. Rather, risk factors usually found in
patient files, or capable of routine assessment, are all that are required for
the ICT to function. The predictive accuracy of the ICT using a reduced set of
106 clinically feasible risk factors from the MacArthur Violence Risk
Assessment Study (an area under the ROC curve of 0.80) is comparable to the
predictive accuracy that we reported
(Steadman et al,
2000) for risk assessment using the expanded set of 134 risk
factors (an area under the ROC curve of 0.82).
Violence risk assessment software
Although the contingent nature of the risk factors identified in
Fig. 1 may appear too intricate
for use in clinical practice, the utility of the ICT model would be enhanced
greatly with the aid of software. Software would facilitate the assessment of
an individual patient by guiding the clinician to ask only those questions
required to assess risk. We are in the process of developing such
software.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Appelbaum, P. S., Robbins, P. C. & Roth, L. H.
(1999) A dimensional approach to delusions: comparison across
delusion types and diagnoses. American Journal of
Psychiatry, 156,
1938-1943.
Barratt, E. (1994) Impulsiveness and aggression. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 61-79. Chicago, IL: University of Chicago Press.
Borum, R. (1996) Improving the clinical practice of violence risk assessment: technology, guidelines, and training. American Psychologist, 51, 945-956.[CrossRef][Medline]
Breiman, L., Freidman, J., Olshen, R., et al (1984) Classification and Regression Trees. Pacific Grove, CA: Wadsworth and Brooks/Cole.
Cohen, S., Kamarck, T. & Mermelstein, R. (1983) A global measure of perceived stress. Journal of Health and Social Behavior, 24, 385-396.[Medline]
Estroff, S. & Zimmer, C. (1994) Social networks, social support, and violence among persons with severe, persistent mental illness. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 259-295. Chicago, IL: University of Chicago Press.
Folstein, M., Folstein, S. & McHugh, P. (1975) Minimental state: a practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189-198.[CrossRef][Medline]
Gardner, W., Lidz, C., Mulvey, E. P., et al (1996) A comparison of actuarial methods for identifying repetitively violent patients with mental illness. Law and Human Behavior, 20, 35-48.
Grisso, T., Davis, J., Vesselinov, R., et al (2000) Violent thoughts and violent behavior following hospitalization for mental disorder. Journal of Consulting and Clinical Psychology, in press.
Hare, R. (1998) The Hare PCL-R: some issues concerning its use and misuse. Legal and Criminological Psychology, 3, 99-119.
Hart, S., Cox, D. & Hare, R. (1995) The Hare Psychopathy Checklist: Screening Version. North Tonawanda, NY: Multi-Health Systems.
Hollingshead, A. & Redlich, F. (1958) Social Class and Mental Illness. New York: Guilford Press.
Janca, A. & Helzer, J. (1990) DSM-III-R Criteria Checklist. DIS Newsletter, 7, 17.
Levy, D. E., Caronna, J. J., Singer, B. H., et al (1985) Predicting outcome from hypoxic-ischemic coma. Journal of the American Medical Association, 253, 1420-1426.[Abstract]
Lidz, C., Mulvey, E. P. & Gardner, W. (1993) The accuracy of predictions of violence to others. Journal of the American Medical Association, 269, 1007-1011.[Abstract]
Monahan, J., Appelbaum, P., Mulvey, E., et al (1994) Ethical and legal duties in conducting research on violence: lessons from the MacArthur Risk Assessment Study. Violence and Victims, 8, 380-390.
Mooney, C. & Duval, R. (1993) Bootstrapping: A Nonparametric Approach to Statistical Inference. Newbury Park, CA: Sage.
Novaco, R. (1994) Anger as a risk factor for violence among the mentally disordered. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 21-59. Chicago, IL: University of Chicago Press.
Overall, J. (1988) The Brief Psychiatric Rating Scale (BPRS): recent developments in ascertainment and scaling. Psychopharmacology Bulletin, 24, 97-99.
Pescosolido, B., Monahan, J., Link, B., et al (1999) The public's view of the competence, dangerousness and need for legal coercion among persons with mental illness. American Journal of Public Health, 89, 1339-1345.[Abstract]
Quinsey, V., Harris, G., Rice, M., et al (1998) Violent Offenders: Appraising and Managing Risk. Washington, DC: American Psychological Association.
Reed, J. (1997) Risk assessment and clinical risk management: the lessons from recent inquiries. British Journal of Psychiatry, 170 (suppl. 32), 4-7.
SPSS (1993) SPSS for Windows CHAID (Release 6.0). Chicago, IL: SPSS Inc.
Steadman, H. J., Monahan, J., Appelbaum, P.S., et al (1994) Designing a new generation of risk assessment research. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 297-318. Chicago, IL: University of Chicago Press.
Steadman, H. J., Mulvey, E. P., Monahan, J., et al (1998) Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Archives of General Psychiatry, 55, 1-9.
Steadman, H. J., Silver, E., Monahan, J., et al (2000) A classification tree approach to the development of actuarial violent risk assessment tools. Law and Human Behavior, 24, 83-100.[CrossRef][Medline]
Swets, J. (1988) Measuring the accuracy of diagnostic systems. Science, 240, 1285-1293.[Medline]
Wallace, C., Mullen, P., Burgess, P., et al (1998) Serious criminal offending and mental disorder. Case linkage study. British Journal of Psychiatry, 172, 477-484.[Abstract]
Received for publication May 19, 1999. Revision received December 3, 1999. Accepted for publication December 7, 1999.