Developing a clinically useful actuarial tool for assessing violence risk{dagger}

JOHN MONAHAN, PhD

School of Law, University of Virginia, Charlottesville, VA 22903

HENRY J. STEADMAN, PhD and PAMELA C. ROBBINS, BA

Policy Research Associates, Delmar, NY 12054

ERIC SILVER, PhD

Department of Sociology, The Pennsylvania State University, College Park, PA 16802

PAUL S. APPELBAUM, MD and THOMAS GRISSO, PhD

Department of Psychiatry, University of Massachusetts Medical Center, Worcester, MA 01655

EDWARD P. MULVEY, PhD and LOREN H. ROTH, MD

Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA


  
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
Declaration of interest Research was funded by the John D. and Catherine T. MacArthur Foundation and by the US National Institute of Mental Health.

{dagger} See editorial pp. 307–311, this issue Back

Correspondence: John Monahan, PdD, School of Law, University of Virginia, 580 Massie Road, Charlottesville, VA 22903-1789, USA. Tel: (804) 924 3632; Fax: (804) 982 2845; e-mail: jmonahan{at}law5.law.virginia.edu


   ABSTRACT
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
Background A new actuarial method for violence risk assessment - the Iterative Classification Tree (ICT) - has become available. It has a high degree of accuracy but can be time and resource intensive to administer.

Aims To increase the clinical utility of the ICT method by restricting the risk factors used to generate the actuarial tool to those commonly available in hospital records or capable of being routinely assessed in clinical practice.

Method A total of 939 male and female civil psychiatric patients between 18 and 40 years old were assessed on 106 risk factors in the hospital and monitored for violence to others during the first 20 weeks after discharge.

Results The ICT classified 72.6% of the sample as either low risk (less than half of the sample's base rate of violence) or high risk (more than twice the sample's base rate of violence).

Conclusions A clinically useful actuarial method exists to assist in violence risk assessment.


   INTRODUCTION
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
A recent survey in the USA found that 95% of the general public believes that when a person with a mental disorder is predicted to be violent to others, legal intervention to avert the anticipated harm is justified (Pescosolido et al, 1999). The pervasiveness of such belief throughout the world helps to explain the wide and growing variety of laws, including, in-patient and out-patient civil commitment, sexual predator commitment, tort liability and employment discrimination, that call upon psychiatrists and psychologists to assess the risk of violence (Reed, 1997). A large body of research conducted in the 1970s and 1980s called into question the ability of clinicians to make accurate risk assessments of violence of the type that the public and the law demand. More recent research (Lidz et al, 1993) has been only slightly more sanguine. One repeatedly suggested way to improve the accuracy of risk assessments of violence has been to use statistical or actuarial methods to inform clinical judgement (Borum, 1996). However, where clinically relevant actuarial tools have been available, their application has been sufficiently cumbersome and time-consuming that actuarial input into the risk assessment of violence has been impractical in most real-world clinical settings (Gardner et al, 1996).


   ITERATIVE CLASSIFICATION TREE METHOD
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
We have recently developed an actuarial tool for assessing the risk of violence by people discharged from psychiatric facilities that we believe has greater potential for clinical application than existing actuarial methods (Steadman et al, 2000). We call this tool the Iterative Classification Tree (ICT). Classification trees (Breiman et al, 1984) have long been used in a number of areas of medicine, including neurology (e.g. Levy et al, 1985). A classification tree approach to the risk assessment of violence is predicated upon an interactive and contingent model of violence: one that allows many different combinations of risk factors to classify a person as high or low risk. Whether a particular question is asked in any clinical assessment grounded in this approach depends on the answers given to each prior question. Based on a sequence established by the classification tree, a first question is asked of all persons being assessed. Contingent on the answer to that question, one or another second question is posed, and so on, until each person is classified into a category on the basis of the risk of violence. This contrasts with the usual approach to actuarial risk assessment in which a common set of questions is asked of everyone being assessed and every answer is weighted and summed to produce a score that can be used for the purposes of categorisation.

In addition to its tree-based character, our approach acknowledges the practical impossibility of adequately classifying all persons into a high or a low violence risk group. Therefore, rather than relying on the standard single threshold for distinguishing among cases, our approach employs two thresholds: one for identifying high-risk cases and one for identifying low-risk cases. We assume that inevitably there will be cases that fall between these two thresholds, cases for which any prediction scheme is incapable of making an adequate assessment of high or low risk. Based on current knowledge, the aggregate degree of risk presented by these intermediate cases cannot be distinguished statistically from the base rate of the sample as a whole.


   CLINICAL UTILITY
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
Our first test of the ICT method (Steadman et al, 2000) focused on how well the method performed in making risk assessments of violence under ideal conditions (i.e. with few constraints on the time or resources necessary to gather risk factors). For example, the risk factor that most clearly differentiated high-risk from low-risk groups was the Hare Psychopathy Checklist: Screening Version (Hare PCL-SV; Hart et al, 1995). Given that the full Hare PCL-R requires several hours to administer - the screening version alone takes over one hour to administer - and that it has to be administered by experienced clinicians whom Hare (1998) recommended should undergo three days of specialised training, resource constraints in many non-forensic clinical settings will preclude its use.

This article applies the ICT method to the sample of patients assessed in the MacArthur Violence Risk Assessment Study (Steadman et al, 1998). Our goal is to increase the clinical utility of this actuarial method by restricting the risk factors tested to those commonly available in hospital records or capable of being assessed routinely in clinical practice.


   METHOD
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
Subject enrolment
Admissions were sampled from acute psychiatric in-patient facilities at three sites: Western Psychiatric Institute and Clinic (Pittsburgh, PA); Western Missouri Mental Health Center (Kansas City, MO); Worcester State Hospital and the University of Massachusetts Medical Center (Worcester, MA). Selection criteria for research subjects were: civil admissions; between the ages of 18 and 40 years; English-speaking; White or African American ethnicity (or Hispanic in Worcester only); and a chart diagnosis of schizophrenia, schizophreniform disorder, schizoaffective disorder, depression, dysthymia, mania, brief reactive psychosis, delusional disorder, alcohol or drug abuse or dependence, or a personality disorder. After complete description of the study to the subjects, written informed consent was obtained.

Sample description
We approached a quota sample (to ensure representativeness across sites on gender, race, and age) of 1695 to participate. The refusal rate was 29% (n=492). The final sample given a hospital interview was 1136. Differences between the eligible admissions and the follow-up sample (n=939) are discussed in detail elsewhere (Steadman et al, 1998). Males comprised 57.3% of the sample. Ethnically, 68.7% of the sample was White, 29.1% African American and 2.2% Hispanic. The mean age was 29.9 (s.d.=6.2) years. Depression was the most frequent primary research diagnosis on the DSM-III-R Checklist (Janca & Helzer, 1990; 41.9%), followed by alcohol/drug abuse or dependence (21.8%), schizophrenia (17.0%), bipolar disorder (14.1%), personality disorder only (2.1%) and other psychotic disorder (3.1%). The proportion of all cases with a primary research diagnosis of major mental disorder that had a co-occurring diagnosis of substance abuse or dependence was as follows: depression, 49.6%; schizophrenia, 41%; bipolar disorder, 37.7%; and other psychotic disorder, 45%.

Hospital data collection
Hospital data collection was conducted in two parts: an interview by a research interviewer to obtain data on risk factors and violence; and an interview by a research clinician (PhD or MA/MSW in psychology or social work) to confirm the chart diagnosis using the DSM-III-R Checklist and to administer several clinical instruments.

The hospital data set assembled in the MacArthur Violence Risk Assessment Study consisted of 134 risk factors from four conceptual domains: dispositional or personal factors (e.g. age); historical or developmental factors (e.g. child abuse); contextual or situational factors (e.g. social networks); and clinical or symptom factors (e.g. delusions) (Steadman et al, 1994). For the present analysis, we eliminated 28 risk factors that would be the most difficult to obtain in clinical practice, restricting ourselves to the remaining 106. Two criteria were used to eliminate risk factors. The first was to eliminate information generally unavailable to mental health personnel in the context of brief hospitalisation (e.g. information in official arrest records, in distinction to self-report of prior arrests). The second was to eliminate information that required the administration of a lengthy (>12-item) instrument to obtain (e.g. a social network inventory (Estroff & Zimmer, 1994)). A list of all 134 risk factors, with their bivariate correlations with violence and with an indication of which were eliminated from these analyses, is provided in Table 1.


View this table:
[in this window]
[in a new window]
 
Table 1 Risk factors and bivariate correlations with violence in the first two follow-ups
 

Community data collection
Twenty weeks after hospital discharge was chosen as the time frame for the analysis here because this was the period during which the prevalence of violence by patients in the community was at its highest (Steadman et al, 1998). Research interviewers attempted two follow-up interviews with enrolled patients in the community during this period, approximately 10 weeks apart. A collateral informant who knew of the patient's behaviour in the community during the follow-up period - usually, but not always, a family member - was also interviewed on the same schedule. Arrest and re-hospitalisation records provided the third source of information about the patients' behaviour in the community.

Patients and collaterals independently were asked whether the patient had been involved in any of several categories of violent behaviour in the past 10 weeks (Lidz et al, 1993). Only the most serious act for each discrete incident was coded. Violence to others was defined to include the following: acts of battery that resulted in physical injury; sexual assaults; assaultive acts that involved the use of a weapon; or threats made with a weapon in hand. (Battery that did not result in injury was defined as ‘other aggressive act’ (Steadman et al, 1998) and is not considered in the analyses reported here.) Violence reported by any of the three data sources - subject self-report, collateral report, or official records - was reviewed by a team of trained coders. Ethical and legal issues encountered in conducting this research are discussed elsewhere (Monahan et al, 1994).

Developing the classification tree
To develop the ICT model, we used CHAID (chi-squared automatic interaction detector) software (SPSS, 1993). Specifically, the CHAID algorithm was used to assess the statistical significance of the bivariate association between each of the 106 eligible risk factors and the dichotomous outcome measure - violence in the community - until the most statistically significant value of {chi}2 was identified, with P < 0.05 a necessary condition for risk factor selection. Once a risk factor was selected, the sample was partitioned according to the values of that risk factor. This selection procedure was then repeated for each of the sample partitions, thus further partitioning the sample. The result of the partitioning process was to identify groups of cases that shared the same risk factors and that also shared the same values on the outcome measure of violence.

Iterating the classification tree
We then extended this recursive partitioning approach in an iterative fashion. That is, all subjects not classified into groups designated as either high risk or low risk in the first iteration of CHAID were pooled together and re-analysed in a second iteration of CHAID. This iterative process continued until it was not possible to classify any additional groups of subjects as either high or low risk (with no group allowed to contain fewer than 50 cases).

Choosing two cut-offs
The choice of cut-off scores for high-risk and low-risk categories must be made in the context of legal or policy values external to the methodology chosen for assessing risk. Here, for illustrative purposes, we defined any group of patients with a rate of violence that was less than half the base prevalence rate of the total sample, as in the low-risk category, and any group of patients whose rate of violence was greater than twice the base prevalence rate of the total sample, as in the high-risk category. Because the base prevalence rate of violence during the first 20 weeks after hospital discharge for the total sample was 18.7% (i.e. 18.7% of the patients committed at least one violent act during either the first or second 10-week follow-up period), this meant that the cut-off for the low-risk category was 9% violent and the cut-off for the high-risk category was 37% violent.

The ICT contained three iterations (Fig. 1). In the first iteration, the tree classified 429 of the 939 subjects (45.7%) into either the high- or the low-risk categories. In the second iteration, the tree classified as high- or low-risk 167 (32.7%) of the 510 subjects who were not classified into either high- or low-risk groups at the end of iteration 1. In the third iteration, the tree classified as high- or low-risk 86 of the 343 subjects (25.1%) who were unclassified at the end of iteration 2. At the end of iteration 3, no further groups could be classified as high- or low-risk, given the parameters of the model we had set (e.g. no group with fewer than 50 cases); 257 subjects (27.4% of the total sample) remained unclassified. The final ICT contained 15 contingent risk factors that formed 11 risk groups (four low-risk groups, accounting for 50.9% of the total sample; three high-risk groups, accounting for 21.7% of the total sample; and four unclassified risk groups, accounting for 27.4% of the sample).



View larger version (22K):
[in this window]
[in a new window]
 
Fig. 1 Clinically useful iterative classification tree. – – –, Low violence risk; ——, high violence risk; · · · ·, unclassified.

 

The risk factors displayed in Fig. 1 are defined as follows. Seriousness of prior arrests was a patient's self-report of the seriousness of arrests since age 15 years. Motor impulsiveness was measured from the motor sub-scale of the Barratt Impulsiveness Scale (Barratt, 1994). Father used drugs was a self-report question on whether the patient's father ever used drugs excessively. Recent violent fantasies was measured by the Schedule of Imagined Violence (Grisso et al, 2000). Major disorder without substance abuse refers to a diagnosis of any major mental disorder without any co-occurring substance abuse diagnosis, as reached by research clinicians using the DSM-III-R Checklist. Legal status was the initial status for the baseline hospitalisation, as recorded in hospital admission records. Schizophrenia was the diagnosis reached by research clinicians using the DSM-III-R Checklist. Anger reaction was measured by a short version of the Behavioural Subscale of the Novaco Anger Scale (Novaco, 1994). Employed was a self-report question regarding the patient's paid full- or part-time employment status in the two months prior to hospital admission. Recent violence was a self-report of violence in the two months prior to hospital admission. Loss of consciousness referred to a self-report of any loss of consciousness due to head injury. Parents fought was a self-report by the patient that his or her parents engaged in physical fights with one another when the patient was growing up. (A complete list of the questions comprising these risk factors is available from the first author upon request.)

Receiver operating characteristic
To assess the predictive accuracy of the actuarial model produced by this method and to facilitate further comparisons of our results with other research on violence risk assessment, we used a receiver operating characteristic (ROC) analysis (Gardner et al, 1996; Quinsey et al, 1998). The statistic used to summarise the analysis is the area under the ROC curve, which corresponds to the probability that a randomly selected violent patient will have been assessed by the risk assessment tool as higher risk than a randomly selected non-violent patient (Swets, 1988). The area under the ROC curve for the 11 risk groups presented in Fig. 1 is 0.80 (P<0.001). The distribution of cases that were violent or not violent during the follow-up as a function of the low- and high-risk cut-offs used to generate the ICT is presented in Table 2.


View this table:
[in this window]
[in a new window]
 
Table 2 Distribution of violent and non-violent cases using two thresholds
 

Bootstrapping
We did not cross-validate the ICT. Cross-validation of a risk assessment model requires estimating the model on a subset of the data and validating the model on the rest. As noted by Gardner et al (1996), however, cross-validation "wastes information that ought to be used estimating the model" (p 43). For this reason, bootstrapping (Mooney & Duval, 1993) has become a widely used analytical strategy for estimating the shrinkage to be expected when a model is generalised to a sample other than the one on which it was estimated. In conducting such an analysis, 1000 bootstrapped samples were drawn from the original data set. Table 3 presents the 95% confidence intervals for each of the 11 risk groups in the ICT, in order of decreasing risk. The ranges of these intervals indicate how the ICT is likely to perform on other similar samples.


View this table:
[in this window]
[in a new window]
 
Table 3 Bootstrapped 95% confidence intervals for the ICT risk groups
 


   DISCUSSION
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
We have sought to increase the utility of an actuarial method for real-world clinical decision-making by applying the newly developed ICT method (Steadman et al, 2000) to a set of violence risk factors commonly available in clinical records or capable of being assessed routinely in clinical practice. We have shown that the ICT partitioned 72.6% of a sample of discharged psychiatric patients into one of two categories with regard to their risk of violence to others during the first 20 weeks after discharge. One category consisted of groups whose rates of violence were no more than half the base rate of the total patient sample (i.e. <=9% violent). The other category consisted of groups whose rates of violence were at least twice the base rate of the total patient sample (i.e. >=37% violent). The actually observed rates of violence in the low- and high-risk categories were 5% and 45%, respectively. The prevalence of violence within individual risk groups within the low- and high-risk categories varied from 2.7% to 52.7% (Table 3).

The ICT left 27.4% of the total sample unclassified, meaning that it could find no combination of risk factors that allowed these patients to be classified into either a low- or a high-risk group. The violence rate for the unclassified category was 24.1%.

Clinical illustrations
Illustrating the use of the ICT may be helpful. A clinician evaluating a patient's risk of violence using the ICT presented in Fig. 1. would first ask the patient about the seriousness of his or her prior arrests. If the patient stated that he or she had previously been arrested for a violent crime, the clinician would then inquire into whether the patient recently had been fantasising about being violent. If the patient responded affirmatively to this second question, he or she at that point would be placed in the high violence risk category. More specifically, the patient would be placed in risk group B, a group in which approximately 53% of the patients are expected to commit a violent act in the next several months.

If, on the other hand, the patient denied having violent fantasies, the clinician would then indicate whether the patient had a diagnosis of schizophrenia. If the patient did have such a diagnosis, he or she at that point would be placed in the low violence risk category. More specifically, the patient would be placed in risk group E, a group in which approximately 7% of the patients are expected to commit a violent act in the next several months. (For other studies finding rates of violence to be lower among patients with schizophrenia than among patients with other, primarily personality disorder, diagnoses, see: Gardner et al, 1996; Quinsey et al, 1998; Wallace et al, 1998.)

Comparative predictive accuracy
We have demonstrated here that the ICT method may be adapted for clinical use. The method does not require unavailable or costly-to-gather data in order to characterise the risk of violence. Rather, risk factors usually found in patient files, or capable of routine assessment, are all that are required for the ICT to function. The predictive accuracy of the ICT using a reduced set of 106 clinically feasible risk factors from the MacArthur Violence Risk Assessment Study (an area under the ROC curve of 0.80) is comparable to the predictive accuracy that we reported (Steadman et al, 2000) for risk assessment using the expanded set of 134 risk factors (an area under the ROC curve of 0.82).

Violence risk assessment software
Although the contingent nature of the risk factors identified in Fig. 1 may appear too intricate for use in clinical practice, the utility of the ICT model would be enhanced greatly with the aid of software. Software would facilitate the assessment of an individual patient by guiding the clinician to ask only those questions required to assess risk. We are in the process of developing such software.


   Clinical Implications and Limitations
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
CLINICAL IMPLICATIONS

LIMITATIONS


   ACKNOWLEDGMENTS
 
This research was supported by the Research Network on Mental Health and the Law of the John D. and Catherine T. MacArthur Foundation and by NIMH grant R01 49696. We are indebted to the members of the Network (Shirley S. Abrahamson, Richard J. Bonnie, Pamela S. Hyde, Stephen J. Morse, Paul Slovic and David B. Wexler) and to Steven Banks and Roumen Vesselinov.


   REFERENCES
 TOP

 ABSTRACT
 INTRODUCTION
 ITERATIVE CLASSIFICATION TREE...
 CLINICAL UTILITY
 METHOD
 DISCUSSION
 Clinical Implications and...
 REFERENCES
 
American Psychiatric Association (1989) Diagnostic and Statistical Manual of Mental Disorders (3rd edn, revised) (DSM-III-R). Washington, DC: American Psychiatric Association.

Appelbaum, P. S., Robbins, P. C. & Roth, L. H. (1999) A dimensional approach to delusions: comparison across delusion types and diagnoses. American Journal of Psychiatry, 156, 1938-1943.[Abstract/Free Full Text]

Barratt, E. (1994) Impulsiveness and aggression. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 61-79. Chicago, IL: University of Chicago Press.

Borum, R. (1996) Improving the clinical practice of violence risk assessment: technology, guidelines, and training. American Psychologist, 51, 945-956.[CrossRef][Medline]

Breiman, L., Freidman, J., Olshen, R., et al (1984) Classification and Regression Trees. Pacific Grove, CA: Wadsworth and Brooks/Cole.

Cohen, S., Kamarck, T. & Mermelstein, R. (1983) A global measure of perceived stress. Journal of Health and Social Behavior, 24, 385-396.[Medline]

Estroff, S. & Zimmer, C. (1994) Social networks, social support, and violence among persons with severe, persistent mental illness. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 259-295. Chicago, IL: University of Chicago Press.

Folstein, M., Folstein, S. & McHugh, P. (1975) Minimental state: a practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189-198.[CrossRef][Medline]

Gardner, W., Lidz, C., Mulvey, E. P., et al (1996) A comparison of actuarial methods for identifying repetitively violent patients with mental illness. Law and Human Behavior, 20, 35-48.

Grisso, T., Davis, J., Vesselinov, R., et al (2000) Violent thoughts and violent behavior following hospitalization for mental disorder. Journal of Consulting and Clinical Psychology, in press.

Hare, R. (1998) The Hare PCL-R: some issues concerning its use and misuse. Legal and Criminological Psychology, 3, 99-119.

Hart, S., Cox, D. & Hare, R. (1995) The Hare Psychopathy Checklist: Screening Version. North Tonawanda, NY: Multi-Health Systems.

Hollingshead, A. & Redlich, F. (1958) Social Class and Mental Illness. New York: Guilford Press.

Janca, A. & Helzer, J. (1990) DSM-III-R Criteria Checklist. DIS Newsletter, 7, 17.

Levy, D. E., Caronna, J. J., Singer, B. H., et al (1985) Predicting outcome from hypoxic-ischemic coma. Journal of the American Medical Association, 253, 1420-1426.[Abstract]

Lidz, C., Mulvey, E. P. & Gardner, W. (1993) The accuracy of predictions of violence to others. Journal of the American Medical Association, 269, 1007-1011.[Abstract]

Monahan, J., Appelbaum, P., Mulvey, E., et al (1994) Ethical and legal duties in conducting research on violence: lessons from the MacArthur Risk Assessment Study. Violence and Victims, 8, 380-390.

Mooney, C. & Duval, R. (1993) Bootstrapping: A Nonparametric Approach to Statistical Inference. Newbury Park, CA: Sage.

Novaco, R. (1994) Anger as a risk factor for violence among the mentally disordered. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 21-59. Chicago, IL: University of Chicago Press.

Overall, J. (1988) The Brief Psychiatric Rating Scale (BPRS): recent developments in ascertainment and scaling. Psychopharmacology Bulletin, 24, 97-99.

Pescosolido, B., Monahan, J., Link, B., et al (1999) The public's view of the competence, dangerousness and need for legal coercion among persons with mental illness. American Journal of Public Health, 89, 1339-1345.[Abstract]

Quinsey, V., Harris, G., Rice, M., et al (1998) Violent Offenders: Appraising and Managing Risk. Washington, DC: American Psychological Association.

Reed, J. (1997) Risk assessment and clinical risk management: the lessons from recent inquiries. British Journal of Psychiatry, 170 (suppl. 32), 4-7.

SPSS (1993) SPSS for Windows CHAID (Release 6.0). Chicago, IL: SPSS Inc.

Steadman, H. J., Monahan, J., Appelbaum, P.S., et al (1994) Designing a new generation of risk assessment research. In Violence and Mental Disorder: Developments in Risk Assessment (eds J. Monahan & H. Steadman), pp. 297-318. Chicago, IL: University of Chicago Press.

Steadman, H. J., Mulvey, E. P., Monahan, J., et al (1998) Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Archives of General Psychiatry, 55, 1-9.

Steadman, H. J., Silver, E., Monahan, J., et al (2000) A classification tree approach to the development of actuarial violent risk assessment tools. Law and Human Behavior, 24, 83-100.[CrossRef][Medline]

Swets, J. (1988) Measuring the accuracy of diagnostic systems. Science, 240, 1285-1293.[Medline]

Wallace, C., Mullen, P., Burgess, P., et al (1998) Serious criminal offending and mental disorder. Case linkage study. British Journal of Psychiatry, 172, 477-484.[Abstract]

Received for publication May 19, 1999. Revision received December 3, 1999. Accepted for publication December 7, 1999.