School of Epidemiology and Health Science, University of Manchester, Manchester, UK
Child Psychiatry Unit, and Social, Genetic, and Developmental Psychiatry Research Centre, Institute of Psychiatry, King's College, London, UK
Department of Child and Adolescent Psychiatry, Guy's, King's and St Thomas' Medical School, London, UK
Royal Children's Hospital Melbourne, Victoria, Australia
Social, Genetic and Developmental Psychiatry Research Centre, Institute of Psychiatry, King's College, London, UK
Virginia Commonwealth University, Department of Human Genetics, Virginia Institute for Psychiatric and Behavioral Genetics, Richmond, Virginia, USA
Correspondence: Professor Andrew Pickles, School of Epidemiology and Health Science, University of Manchester, Stopford Building, Oxford Road, Manchester M13 9PT, UK. E-mail: andrew.pickles{at}man.ac.uk
See editorial, pp.
189190, this issue.
ABSTRACT
Background Relatively little is known about the relationships between psychiatric symptoms, diagnosis and psychosocial impairment.
Aims To examine these contemporaneous relationships and prognostic significance in a large general population sample.
Method Symptoms of major depression, conduct and oppositional defiant disorders were assessed by interview in two waves of the Virginia Twin Study of Adolescent behavioural Development (2800 children aged 8-16 years).
Results Many children below the DSM-III-R diagnostic threshold, especially for depression, had symptom-related impairment, whereas many children reaching the symptom threshold for conduct and oppositional defiant disorders were little impaired. Impairment score was linearly related to symptom count, with no evidence of any additional impairment at the diagnostic threshold. For depression, only symptoms predicted later symptoms and diagnosis. For conduct and oppositional defiant disorders, impairment was additionally predictive of later symptoms and diagnosis.
Conclusions Impairment, in addition to symptoms, is important for both nosology and prognosis.
There is debate in current psychiatric research over the relative importance of diagnosis, extent and intensity of symptomatology, and the presence of functional impairment for psychiatric classification, clinical decision-making and prognosis (e.g. Kendler, 1999; Kessler et al, 1999). Depending upon the diagnostic area, many children with clear symptoms sufficient for a diagnosis appear not to experience significant functional impairment (Simonoff et al, 1997). In addition and of particular concern are the substantial numbers of children apparently impaired by clinical symptoms that are insufficient to assign an operational formal diagnosis (Angold et al, 1999; Costello et al, 1999). Using the Virginia Twin Study of Adolescent Behavioral Development (VTSABD; Eaves et al, 1997; Hewitt et al, 1997; Simonoff et al, 1997), we analyse first the relationship of symptom count and diagnosis to reported impairment, and second the relative importance of these three measures for prognosis over approximately 19 months.
METHOD
Population
The VTSABD is an ongoing cohort-longitudinal study of White twins born
between 1974 and 1983. Twins were recruited through the public and private
school systems in the state of Virginia and through state-wide publicity. From
a target population of 1892 families, 1412 families (75%) participated in the
first wave (Meyer et al,
1996). When interviewed, 62 twins were aged 17-18 years and were
excluded from this study, leaving 2762 children in Wave I. Of these, 2102
children participated in the second wave 18.6 (s.d.=4.7) months later.
Measures
At each wave, clinical symptomatology over the previous 3 months was
elicited by separate interview of each twin about themselves, and each parent
about both twins using the Child and Adolescent Psychiatric Assessment (CAPA;
Angold et al, 1995).
Data from different informants were combined using the standard
or rule (Costello et
al, 1996; Simonoff et
al, 1997) in which a symptom is endorsed if reported by
either the child or parent. Paternal ratings contributed only if maternal
ratings were unavailable. We examine here DSM-III-R symptoms
(American Psychiatric Association,
1987) within the areas of major depression, conduct disorder and
oppositional defiant disorder, each symptom being required to meet
operationalised criteria for intensity, frequency and duration
(Simonoff et al,
1997).
One of the major additions to DSM-III-R encompassed in DSM-IV (American Psychiatric Association, 1994) was a more explicit assessment of clinically significant distress or impairment (Spitzer & Wakefield, 1999). The operational criteria for symptoms in the CAPA explicitly exclude from consideration factors normally thought of as constituting psychosocial impairment. The symptom severity ratings do include consideration of factors such as psychological or physiological intrusiveness while the symptom was present, and in the areas of depression and anxiety this required symptoms to be distressful. More broadly defined psychosocial impairment due to psychiatric symptoms was assessed in an entirely separate section of the interview, usually completed once all symptom areas had been evaluated. As with symptoms, impairment was rated over the preceding 3 months. The 15 areas in which impairment was rated were parent relationships, sibling relationships, self-care, homework and chores, leaving house, school performance, school suspension, teacher relationships, school peer relationships, spare-time activities, non-school adult relationships, non-school peer relationships, employment, treatment and placement. Each area of impairment was rated on a three-point scale: 0 if no impairment was present, 1 if impairment was partial, indicated by decreased functioning in that area, and 2 if impairment was severe, that is functioning in that area had almost ceased. Ratings of impairment were tied to symptom areas. Where symptoms occurred in more than one area, respondents were asked systematically to consider which symptom areas were responsible for the impairment, with a reminder of the different symptom areas that had already been endorsed. When respondents were uncertain whether one or more area was responsible for impairment, impairment was rated as being related to each of the symptom areas that the respondent felt might be involved. Ratings of impairment were made independently by each informant.
To code impairment as present, impairment was required to have started either at the same time or following the onset of symptoms. For symptoms of long duration, impairment could be rated only if there was clear evidence that the symptoms or behaviour were interfering with functioning in that area. For the current analyses, these ratings were combined using the or rule described above for symptom counts, to give a score that could range from 0 to 30 in each symptom area. Angold & Costello (1995) report high levels of reliability for the child-reported symptom, diagnosis and impairment measures.
The VTSABD pairs were ascertained from the general population, and many subjects had few, if any, psychiatric symptoms of clinical severity. It made no sense to ask about symptom-related psychosocial impairment in the absence of symptoms, and therefore a rule was developed to determine whether the impairment section of the interview should be entered with respect to each symptom area. In the areas that we examine the rule required impairment to be assessed only if two or more symptoms in an area were coded with symptom intensity of 2 or greater. For symptoms to contribute to the symptom count, additional criteria relating to frequency and duration were usually required.
In this study we focused on two areas of symptomatology that seemed to possess quite different impairmentsymptom characteristics. These were (a) major depressive disorder (MDD), and (b) conduct disorder (CD) and oppositional defiant disorder (ODD). In what follows, DSM-III-R diagnosis is used to refer to those who meet the operationalised criteria in respect of symptoms. For CD and ODD those criteria relate to the number of criterion symptoms. For MDD there was the additional necessary requirement for either depressed or irritable mood or loss of pleasure or interest. In fact, only two assessments meeting the symptom number criterion failed to meet this additional criterion. For all diagnoses, those also meeting the criterion score of 2 or more for psychosocial impairment (Simonoff et al, 1997) are referred to as having a DSM-III-R diagnosis with impairment.
Analysis
The analyses faced a number of problems. First, observations from twins in
a pair and from Wave I to Wave II are correlated. Second, CD and ODD shared a
combined impairment rating. This rating had to be related to counts of both CD
and ODD symptoms. These two problems were tackled by using the survey analysis
procedures (Binder, 1983) of
STATA Release 6 (StataCorp,
1999) which provide a robust approach to cluster sampling (here
twin-pair) and multivariate responses (here CD and ODD). Third, the counts of
symptoms and ratings of impairment were non-normal. For examining the
linearity of the relationship between impairment rating (Y) and
symptoms (X), we exploited the robust properties of ordinary linear
regression when used in conjunction with robust forms of test statistics
(Huber, 1967). For predicting
symptom severity, the counts of symptoms were treated as an ordinal scale, and
proportional-odds ordinal logistic regression was used. Such models are
invariant to the collapsing of categories, a property that gives two
advantages. First, when diagnosis is defined by a threshold number of symptoms
the proportional odds ratio (POR) estimated by the ordinal logistic regression
should be equal, apart from sampling error, to the odds ratio from the
logistic regression analysis of diagnosis, simplifying comparison. Second, in
the multivariate analyses of CD and ODD symptoms, the ordinal model implicitly
standardises for the fact that the different diagnoses have different ranges
of possible symptom counts. These multivariate ordinal response models were
set up in the fashion of Clayton
(1976) and Stram et al
(1988), practicalities being
described by Maughan et al
(1998).
A final problem related to the missing impairment data. Symptom-related impairment was set to zero for subjects with no clinical symptoms, and was always rated where two or more clinically significant symptoms in an area were identified. However, when only one symptom was present the VTSABD field-work protocol only sometimes required impairment ratings be obtained (when there were other symptoms that met the intensity criterion but failed the subsequently applied duration or frequency criteria). A structured, and thus tractable, missing-data pattern was obtained by setting to missing the impairment scores of all subjects with a single symptom. The resulting missing-data pattern was then not a problem for model-based analyses (ordinary and logistic regressions) that included the contemporaneous symptom score as a predictor, the missing data then being covariate-dependent missing data in the terminology of Little (1993). However, the missing data could still bias some of the simple descriptive statistics, and to those we attach cautionary notes.
For examining contemporaneous associations we made use of both waves of data, with most children being assessed twice (correcting inference for repeated measurement). For simplicity the term child is used when in fact the units are child assessments.
RESULTS
Depression: cross-sectional results
Of the 73 children with MDD, only 15 (21%) had an impairment score of less
than 2. Of the 476 sub-threshold children with two to four symptoms of
depression, 245 (51%) had impairment scores of 2 or above. Regression did not
suggest that being above or below the diagnostic symptom threshold was
associated with any change in impairment beyond that expected from a linear
increase with the number of symptoms (P=1.0).
Depression: longitudinal results
Rates among those with MDD at Wave I
(Table 1) are similar
regardless of impairment at Wave I. Among those without MDD at Wave I, rates
were substantially higher among those with impairment than those without, but
the results above suggest that this is a consequence of symptom differences
rather than impairment differences, since impairment could only be rated as
positive if individuals had at least two MDD symptoms.
|
Comparing children who at Wave I had a diagnosis without impairment with children who had sub-threshold symptoms with impairment suggested that the symptoms were more important for prediction than the impairment. In logistic regression predicting Wave II (n=1723 subjects), Wave I symptom count was highly predictive of later symptom count (POR=1.95, 95% CI 1.78-2.14, P<0.001), diagnosis (OR=2.00, 95% CI 1.71-2.34, P<0.001) and diagnosis with impairment (OR=2.16, 95% CI 1.58-2.95, P<0.001). No significant independent contribution of impairment score was found for predicting either symptom count (POR=1.05, 95% CI 0.94-1.17, P=0.4) or diagnosis (OR=0.97, 95% CI 0.79-1.19, P=0.8), or even diagnosis with impairment (OR=0.95, 95% CI 0.75-1.19, P=0.7). Wave I diagnostic status also made no significant independent contributions (giving, respectively, POR=0.60, 95% CI 0.22-1.61, P=0.3; OR=0.45, 95% CI 0.10-2.10, P=0.3; OR=0.35, 95% CI 0.06-1.94, P=0.2).
Since the prediction of Wave II impairment was hampered by the pattern of missing data we do not present any formal results. None the less, it was noteworthy that regressions indicated that MDD symptoms were a stronger predictor of subsequent impairment than even prior impairment itself (which was not independently significant).
Conduct and oppositional defiance: cross-sectional results
Of the 235 children with a CD diagnosis (requiring three symptoms), 82
(35%) had impairment scores less than 2. Of the 197 with a diagnosis of ODD
(requiring five symptoms) only 44 (22%) had similarly low impairment. There
were an additional 161 children with combined CD and ODD symptom totals of
five or six who did not receive a DSM-III-R diagnosis of either CD or ODD, and
of these 107 (66%) had impairment scores of 2 or more. Altogether 457 children
had impairment above threshold without either diagnosis, and 118 children had
a CD or ODD DSM-III-R diagnosis without significant impairment. Symptoms and
impairment appeared to be broadly linearly related. Once CD and ODD symptom
counts had been included, regressions did not identify either diagnostic
threshold as being associated with additional impairment (P=0.6 for
CD and P=0.3 for ODD diagnoses).
Conduct and oppositional defiance: longitudinal results
Table 1 shows that at Wave
II rates of diagnosis among those with a Wave I CD/ODD diagnosis without
impairment were lower than for those with a Wave I diagnosis with impairment.
For those without a Wave I diagnosis, outcome rate differences could again
have reflected differences in symptoms levels. A comparison of those with a
diagnosis but without impairment and those with sub-threshold symptoms with
impairment suggested impairment as being predictive. The multivariate logistic
analyses provided a more structured assessment of this. For Wave II counts of
CD and ODD symptoms (n=1652) the Wave I symptom counts were highly
predictive, and the association was specific, with interaction terms showing
that CD symptoms predicted more strongly to CD than to ODD symptoms and
vice versa (Wald F(2,978)=14.98, P<0.001).
Estimates of POR were 1.82 (95% CI 1.58-2.10) for CD symptoms predicting to CD
symptoms, 1.44 (95% CI 1.28-1.63) for CD to ODD symptoms, 1.19 (95% CI
1.09-1.29) for ODD to CD symptoms, and 1.49 (95% CI 1.39-1.61) for ODD to ODD
symptoms. The Wave I impairment score added significantly to prediction (main
effect POR=1.16, 95% CI 1.08-1.25, P<0.001), with similar effect
for later CD and ODD symptoms (outcome symptom area by Wave I impairment
interaction P=0.3). The block of terms estimating any additional
effects of Wave I CD and/or ODD diagnosis were non-significant: Wald
F(4,976)=1.72, P=0.1.
For predicting Wave II diagnosis, results were essentially similar to those for symptom count, with significant odds ratios for Wave I symptoms (CD for CD 2.01, CI 1.66-2.45, P<0.001; CD for ODD 1.60, CI 1.30-1.98, P<0.001; ODD for CD 1.14, CI 1.0-1.31, P=0.06; and ODD for ODD 1.46, CI 1.28-1.68, P<0.001). The additional effects of Wave I impairment for later diagnosis were also significant (for CD, OR=1.16, CI 1.02-1.32, P=0.03, and for ODD, OR=1.24, CI 1.08-1.41, P=0.002), but those for Wave I diagnosis were not: Wald F(4,976)=1.61, P=0.2.
Not surprisingly, given the foregoing, both Wave I symptoms and impairment score were predictive of Wave II diagnoses that were impairing (CD for CD OR 1.75, CI 1.42-2.15, P<0.001; CD for ODD OR 1.51, CI 1.21-1.90, P<0.001; ODD for CD OR 1.31, CI 1.13-1.52, P<0.001; ODD for ODD OR 1.52, CI 1.31-1.77, P<0.001; impairment for CD OR=1.23, CI 1.07-1.41, P=0.004 and impairment for ODD OR=1.33, CI 1.16-1.51, P<0.001).
While again cautioning for missing data, it was noteworthy that Wave I impairment was strongly predictive of Wave II impairment independently of Wave I CD/ODD symptoms.
Comparison across areas of symptomatology
Regressions estimated the impairment resulting from the
typical symptom to be 0.89 (CI 0.82-0.96) for MDD, 0.53 (CI
0.48-0.58) for ODD, and 0.40 (CI 0.31-0.48) for CD. The implied expected
impairment scores at the three diagnostic symptom thresholds were 4.42 for
MDD, 2.65 for ODD and 1.20 for CD.
Since the scale of the impairment score is arbitrary, Table 2 reports estimates for predictive effects that have been standardised by the observed variances of the Wave I symptom counts and impairment scores for ease of comparison. The high level of continuity shown by depressive symptoms, even higher than that for CD, is perhaps unexpected. The odds ratio for impairment was close to the null value of 1 for MDD, but for CD and particularly ODD the additional prognostic value of impairment was evident.
|
DISCUSSION
Although the relationship between symptoms and impairment is fundamental to notions of psychiatric disorder and treatment needs, it is one that has been subjected to surprisingly little empirical analysis. A prime alteration in DSM-IV from DSM-III-R is the explicit inclusion of an impairment requirement for diagnosis. A mental disorder must "cause clinically significant distress or impairment in social, occupational or other important areas of functioning" (American Psychiatric Association, 1987: 7). The addition of this impairment criterion was aimed at requiring more than a constellation of symptoms to distinguish pathological conditions from variations in normal experience. In part the objective was to reduce false positives (Spitzer & Wakefield, 1999) and thus lower rates of disorder reported by epidemiological studies that some found embarrassingly high. Some ambiguity remained, however, as disorders of mild severity are described as having "few, if any, symptoms in excess of those required to make the diagnosis as present, and symptoms result in no more than minor impairment in social or occupational functioning" (American Psychiatric Association, 1987: 2, Use of this manual). An implication in the above statement is that the association between number of symptoms and degree of impairment is strong, in that having few symptoms is expected to go along with only minor impairment.
Not all children who attain symptom thresholds for disorder show marked impairment. In a group of referred children aged 7-10 years, Costello et al (1988) reported that 82% of those receiving a diagnosis from parental interview and 86% of those diagnosed from child interview were assessed as impaired, compared with 51-52% of children who did not have such a diagnosis. Similar findings were reported by Bird et al (1988) in their epidemiological study of Puerto Rican children aged 6-16 years. Simonoff et al (1997) demonstrated variability in the extent to which disorders defined by operational criteria are associated with symptom-related psychosocial functioning. They found phobias to be least likely to be associated with impairment, followed by anxiety disorder, with major depressive disorder as likely to be associated with impairment as externalising disorders. More recently concern has been raised at the numbers of children who have psychosocial impairment without reaching the threshold for any one operational diagnosis (Angold et al, 1999; Costello et al, 1999).
Main findings: cross-sectional association of symptoms and
impairment
The results of this study suggest the need for a more fundamental
reassessment of the symptomimpairmentdiagnosis triad. The
comparison across disorders of the cross-sectional relationships was
particularly revealing.
First, in the diagnostic areas considered here (MDD, CD and ODD), the expected level of impairment appears to increase uniformly with the number of symptoms. Reaching the diagnostic symptom threshold has no special implications for the expected level of impairment.
Second, there is considerable variation across diagnostic areas in the extent to which psychiatric symptoms are associated with impairment, with obvious implications for needs assessment. Symptoms sufficient to reach the diagnostic threshold for MDD, in particular, and to a lesser extent ODD, appear to be associated with more impairment than the equivalent diagnostic symptom threshold for CD. For MDD, those meeting the current symptom threshold are virtually always impaired, but it is also clear that these are fewer than the number of children experiencing marked impairment to be found among those below the symptom diagnostic threshold. These findings, and the longitudinal results discussed below, can be seen as offering substantial support for the DSM-IV experimental diagnosis of minor depression (two to four MDD symptoms).
Third, the more impairing nature of ODD symptoms compared with CD runs counter to the commonly held view of ODD as a mild and developmentally immature form of CD. In the case of CD, especially for symptoms relating to behaviours that can remain covert, it is plausible that in the short run individuals themselves could experience little, if any, impairment. Indeed, since for these individuals the costs of such symptoms will be borne by others, it is arguable whether impairment should be a component of the diagnostic criterion at all, or whether instead the impact on others should be included in an extended impairment score.
Fourth, the numerous children with four or five mixed CD/ODD symptoms who did not meet either of the CD or ODD diagnostic thresholds were as impaired as those meeting the CD threshold. This lends support for the ICD-10 approach in which these symptom areas are considered jointly (World Health Organization, 1992), in contrast to the DSM-III-R/DSM-IV approach in which they are considered as independent diagnoses. However, the evidence for some degree of specificity for longitudinal prediction suggests that these symptoms should not be merely lumped together.
Main findings: impairment and prognosis
It was clear that symptom areas differed considerably in the extent to
which impairment was prognostic. For MDD, only current symptoms predicted
future symptoms, diagnosis, diagnosis with impairment and in an
admittedly limited analysis even future impairment. Thus, depressive
symptoms could well be primary, and impairment a simple consequence of those
symptoms. Since impairment itself was not predictive, a natural question is
whether impairment should be considered as an element of depressive diagnosis
at all. Should a positive diagnosis, and more practically treatment, be given
to a child who from all reports is experiencing no impairment? Since MDD
symptom criteria do require some form of distress to be present
anyway, this alone may be all that the clinician needs to consider. Our data
suggest that, even in the absence of any apparent current impairment, the
presence of symptoms alone carries risk for later impairment. Treatment might
reduce future impairment not currently reported.
For CD and ODD both symptoms and impairment contributed to the prediction of future symptoms, diagnosis, diagnosis with impairment and the impairment score. There was some specificity within symptom areas (for ODD to predict ODD and CD to predict CD) potentially supportive of the view that these are different disorders or at least subtypes (Lahey et al, 1997). Cross-symptom prediction (CD to ODD and ODD to CD) was about as influential as current impairment. For ODD the importance of impairment could be artefactual, the distinction between symptoms and impairment being often hard to make, but this argument is more difficult to sustain for CD. If accepted as a real finding, then either impairment should be an additional consideration in assigning a CD/ODD diagnosis, or there are impairing and non-impairing subtypes with, for example, impairment identifying the social difficulties of a more antisocial personality disorder variant of CD or a marker for a more psychosocially demanding environment. Treatments targeted at reducing impairment might also reduce symptoms.
Strengths and limitations of the study
The VTSABD provides a large sample, uses multiple informants and
standardised investigator-based assessments covering both psychiatric symptoms
and impairment. However, psychosocial impairment is still an area of
development in child psychiatry, both conceptually and in terms of
measurement. One possible reason for the observation of impairment without
diagnosis could be that the impairment is not a consequence of the symptoms.
Measures such as the Child Global Assessment Scale assess overall level of
functioning, and when impairment is found it is assumed to relate to
psychopathology. The VTSABD study used the CAPA interview, which is unusual in
the extent to which it attempts to ensure that impairment is symptom-related
and in its attempt to measure this impairment separately for different symptom
areas. It is possible that failings in this measurement task, coupled with a
process in which impairment causes symptoms, could explain some of our
findings. Though not specific to the CAPA, differences in symptom prevalence
and DSM severity or intensity requirements (e.g. distress) across symptom
areas also need to be considered when interpreting the potentially different
role of impairment in different symptom areas.
The measurement sophistication of the VTSABD CAPA interview created a complex data structure that inadvertently made analysis difficult. Progress has been made in tackling this but there remain a number of critical analyses for which we have not yet devised satisfactory methods, in particular relating to the prediction of impairing but sub-symptom threshold outcomes.
A proportion of the sub-threshold children with impairment could have received diagnoses of the not otherwise specified (NOS) class (Pincus et al, 1999). Operational definitions of these disorders are currently lacking, so that accounting for such children would be largely exploratory. Angold et al (1999) operationally defined their NOS class as just that group of children with impairment but without any specific diagnosis. Our use of symptom and impairment scores as predictors implicitly recognises the existence of such sub-threshold (NOS) children. Thus, although we have not explicitly used NOS diagnoses, we doubt whether their inclusion would have materially influenced the findings, merely the terms used to describe them.
Clinical Implications and Limitations
CLINICAL IMPLICATIONS
LIMITATIONS
ACKNOWLEDGMENTS
This study was supported by grant MH-45268 from the US NIMH (to Professor Lindon Eaves and Dr Judy Silberg). We thank all the VTSABD study participants for their time and support. This paper has been much improved as a consequence of particularly helpful comments by an anonymous referee.
REFERENCES
American Psychiatric Association (1987) Diagnostic and Statistical Manual of Mental Disorders (3rd edn, revised) (DSM-III-R). Washington: APA.
American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders (4th edn) (DSM-IV). Washington: APA.
Angold, A. & Costello, E. J. (1995) A test-retest reliability study of child-reported psychiatric symptoms and diagnoses using the Child and Adolescent Psychiatric Assessment (CAPA-C). Psychological Medicine, 25, 755-762.[Medline]
Angold, A. & Costello, E. J., Prendergast, M., Cox., A., et al (1995) The Child and Adolescent Psychiatric Assessment (CAPA). Psychological Medicine, 25, 739-753.
Angold, A. & Costello, E. J., Costello, E. J., Farmer, E. M. Z., et al (1999) Impaired but undiagnosed. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 129-137.[Medline]
Binder, D. A. (1983) On the variances of asymptotically normal estimators from complex surveys. International Statistical Review, 51, 279-292.
Bird, H. R., Canino, G., Rubio-Stipec, M., et al (1988) Estimates of the prevalence of childhood maladjustment in a community survey in Puerto Rico. Archives of General Psychiatry, 45, 1120-1126.[Abstract]
Clayton, D. G. (1976) An odds-ratio comparison for ordered categorical data with censored observations. Biometrika, 63, 405-408.
Costello, E. J., Costello, A. J., Edelbrock, C., et al (1988) Psychiatric disorders in pediatric primary care. Archives of General Psychiatry, 45, 1107-1116.[Abstract]
Costello, Angold, A., Burns, B. J., et al (1996) The Great Smoky Mountains Study of Youth: I. Prevalence and correlates of DSM-III-R disorders. Archives of General Psychiatry, 53, 1137-1143.[Abstract]
Costello, Angold, & Keeler, G. (1999) Adolescent outcomes of childhood disorders: the consequences of severity and impairment. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 121-128.[Medline]
Eaves, L. J., Silberg, J. L., Meyer, J. M., et al (1997) Genetics and developmental psychopathology: 2. The main effects of genes and environment on behavioural problems in the Virginia Twin Study of Adolescent Behavioral Development. Journal of Child Psychology and Psychiatry, 38, 965-980.[Medline]
Hewitt, J. K., Silberg, J. L., Rutter, M., et al (1997) Genetics and developmental psychopathology: I. Phenotypic assessment in the Virginia Twin Study of Adolescent Behavioral Development. Journal of Child Psychology and Psychiatry, 38, 843-863.
Huber, P. J. (1967) The behavior of maximum likelihood estimates under non-standard conditions. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. I, pp. 221-233. Berkeley, CA: University of California Press.
Kendler, K. S. (1999) Setting boundaries for
psychiatric disorders. American Journal of Psychiatry,
156,
1845-1848.
Kessler, R. C., DuPont, R. L., Berglund, P., et al
(1999) Impairment in pure and comorbid generalized anxiety
disorder and major depression at 12 months in two national surveys.
American Journal of Psychiatry,
156 ,
1915-1923.
Lahey, B. B., Loeber, R., Quay, H. C., et al (1997) Oppositional defiant disorder and conduct disorder. In DSM-IV Sourcebook (eds T. A. Widiger, A. J. Frances, H. A. Pincus, et al), pp. 189-209. Washington, DC: American Psychiatric Association.
Little, R. J. A. (1993) Pattern mixture models for multivariate incomplete data. Journal of the American Statistical Association, 88, 125-134.
Maughan, B., Collishaw, S. & Pickles, A. (1998) School achievements and adult qualifications among adoptees: a longitudinal study. Journal of Child Psychology and Psychiatry, 39, 669-685.[CrossRef]
Meyer, J. M., Silberg, J. L., Simonoff, E., et al (1996) The Virginia Twin-family Study of Adolescent Behavioral Development: assessing sample biases in demographic correlates of psychopathology. Psychological Medicine, 26, 1113-1133.
Pincus, H. A., Davis, W.W. & McQueen, L. (1999) Subthreshold mental disorders. A review and synthesis of studies on minor depression and other brand names. British Journal of Psychiatry, 174, 288-296.[Abstract]
Simonoff, E., Pickles, A., Meyer, J. M., et al (1997) The Virginia Twin Study of Adolescent Behavioral Development: influences of age, gender and impairment on rates of disorder. Archives of General Psychiatry, 54, 801-808.[Abstract]
Spitzer, R. L. & Wakefield, J. C. (1999)
DSM-IV diagnostic criterion for clinical significance: does it help solve the
false positives problem? American Journal of
Psychiatry, 156,
1856-1864.
StataCorp (1999) STATA Statistical Software. Release 6.0. College Station, TX: Stata Corporation.
Stram, D. O., Wei, L. J. & Ware, J. H. (1988) Analysis of repeated ordered categorical outcomes with possibly missing observations and time dependent covariates. Journal of the American Statistical Association, 83, 631-637.
World Health Organization (1992) International Classification of Diseases and Related Disorders (ICD-10). Geneva: WHO.
Received for publication February 17, 2000. Revision received October 13, 2000. Accepted for publication October 13, 2000.
Related articles in BJP: