Institute of Child Health, University College London
Department of Psychological Medicine, University of Wales College of Medicine, Cardiff, UK
Correspondence: Professor David Skuse, Behavioural and Brain Sciences Unit, Institute of Child Health, 30 Guilford Street, London WC1N 1EH, UK. E-mail: dskuse{at}ich.ucl.ac.uk
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To assess whether the Social and Communication Disorders Checklist (SCDC) fulfils the need for a sensitive measure of autistic traits, which can be completed in a few minutes and which measures heritable characteristics in both males and females.
Method A12-item scale, the SCDC, was completed by three independent samples drawn from a twin register, a group with Turner syndrome and children with a diagnosis of autistic-spectrum disorder attending clinics. The data were used to establish the heritability, reliability and validity of the checklist.
Results Traits measured by the SCDC were highly heritable in both genders (0.74). Internal consistency was excellent (0.93) and test retest reliability high (0.81). Discriminant validity between pervasive developmental disorder and other clinical groups was good, discrimination from non-clinical samples was better; sensitivity (0.90), specificity (0.69).
Conclusions The SCDC is a unique and efficient first-level screening questionnaire for autistic traits.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The testretest reliability study sample was recruited from a database of females with Turner syndrome, based on a national case register. It comprised 254 individuals (mean age 15.7 years, s.d.=4.2, range 3.319.1). Verbal IQ data were available for 72 of these individuals, scored on the Wechsler Intelligence Scale for Children (Wechsler, 1992): the mean verbal IQ was 96.9 (s.d.=17.6, range 58130).
Participants in the validity study comprised patients from three separate clinics: the social and communication disorders clinic at Great Ormond Street Hospital for Children, London (n=230), a child and adolescent mental health service (CAMHS) clinic in Luton (n=30) and another in Hartlepool (n=23). For the Great Ormond Street recruits, ICD10 psychiatric diagnoses (World Health Organization, 1992) were established using a novel computerised autism interview, the Developmental, Dimensional and Diagnostic Interview (3di; Skuse et al, 2004). In the Great Ormond Street clinic the majority of referrals concerned children with neurodevelopmental or language problems. The CAMHS recruits were categorised according to clinician diagnosis. None of the patients recruited into the survey had participated in previous research, and none had previously been assessed with standardised psychiatric interviews for autism. The Great Ormond Street social and communication disorders clinic is a quaternary referral service that specialises in the assessment of children with high-functioning autism and complex presentations. The mean verbal IQ in the Great Ormond Street sample was 94.2 (n=164; range 40153, s.d.=20.1) and the mean performance IQ was 92.7 (n=118; range 49143, s.d.=18.7). Data on IQ were not available for the CAMHS participants, but all of them were in mainstream education. The Great Ormond Street sample for recruitment into this study comprised consecutive referrals to the clinic over a period of 4.5 years, from July 1999 to December 2003. The CAMHS recruits were consecutive referrals during January and February 2004. An additional sample of normal controls (n=118) was recruited to enable the assessment of the SCDCs validity as a screening instrument for autistic traits in the general population (Table 1). All members of the control group had intellectual abilities within the normal range, were English-speaking and were in mainstream schooling.
|
The SCDC was sent out for completion by parents. For participants attending the Great Ormond Street Hospital clinic, this questionnaire formed part of the preappointment assessment, and if initially incomplete the omission was rectified prior to the clinical assessment. Unlike previous evaluations of the Social Responsiveness Scale (Constantino & Todd, 2003) and the Autism Screening Questionnaire (now known as the Social Communication Questionnaire; Berument et al, 1999) in clinical populations, questionnaire completion never followed the standardised interview.
Establishing SCDC psychometric properties
Reliability
In order to assess internal reliability, the SCDCs internal
consistency was calculated. External reliability was evaluated in terms of
testretest reliability; parents of 188 participants completed the SCDC
for a second time, at a mean retest interval of 2.7 years (s.d.=0.5, range
1.515.39).
Validity
We assessed content validity primarily by comparing questions in the SCDC
with items in standardised interviews, such as the Autism Diagnostic Interview
Revised (ADIR; Lord et
al, 1994) and the 3di
(Skuse et al, 2004),
that most strongly discriminate autistic-spectrum disorders from non-autistic
conditions. The domains of content of the questions (see
Appendix) comprise social
reciprocity (questions 1, 2, 3, 6 and 10), non-verbal skills (8) and pragmatic
language usage (7, 11 and 12). Three questions concern functional impairment
(35). There is, however, no explicit question concerning circumscribed
interests or stereotyped patterns of motor behaviour.
|
For further validity analysis, the diagnosis of an autistic-spectrum disorder was defined according to ICD10 criteria, to include autism, atypical autism and Asperger syndrome. In the Great Ormond Street clinic sample this was established from the 3di (see below) according to conventional criteria based on the ADIR algorithm (Lord et al, 1994), combined with findings from the Autism Diagnostic Observation Scale Generic (Lord et al, 2000). Individuals meeting only ICD10 diagnostic criteria for pervasive developmental disorder not otherwise specified were categorised as non-cases for the purpose of this study. The 3di is a parental autism interview that can be administered to unselected clinical and general population samples; it measures both symptom intensity and comorbidity across the full range of the autistic spectrum (Skuse et al, 2004). It is a computerised procedure, for administration by trained interviewers, which generates symptom and diagnostic profiles for both autism and non-autistic conditions. The 3dis testretest and interrater reliability were assessed in unselected clinical (n=50) and non-clinical (n=30) populations (Skuse et al, 2004). Concurrent (n=120), discriminant (n=120) and criterion (n=29) validity were evaluated in autistic-spectrum disorder and non-autistic patient groups. Testretest and interrater reliabilities were excellent (most intraclass correlation coefficients were greater than 0.9). Concurrent validity of the 3di (agreement with independent clinician formulation) was very good (mean k=0.74). Criterion validity of the 3di, in a comparison with the ADIR (Lord et al, 1994), was excellent, and the instruments ability to discriminate between autistic-spectrum v. non-autistic individuals was almost perfect (sensitivity 1.0, specificity 40.97).
Concurrent validity of the SCDC was assessed by a comparison of mean scores on this measure of children with clinically diagnosed autistic-spectrum disorder (n=208) and children with other clinical diagnoses (n=76). Non-autistic conditions in the comparison samples included conduct disorders, attention-deficit hyperactivity disorder (ADHD), pragmatic disorders of language, Tourette syndrome and obsessivecompulsive disorder, diagnosed by experienced clinicians according to ICD10 criteria. We expected the mean SCDC scores of these clinical groups to be higher than those of children in the general population, because of their association with autistic features (e.g. Geurts et al, 2004; Gilmour et al, 2004). Accordingly a second test of concurrent validity was performed, to compare SCDC scores of the clinically identified samples with general population controls (n=118).
Criterion validity of the SCDC was evaluated by determining correlations between the questionnaire total score and the sub-scale scores of algorithms generated by the 3di (Skuse et al, 2004), which are equivalent to the sub-scale scores of the ADIR algorithm (Lord et al, 1994).
Discriminant validity analysis was conducted using receiver operating characteristic (ROC) analysis (Fombonne, 1991) in which the area under a ROC curve (the AOC) serves as an index of a tests accuracy in discriminating between groups. In this analysis clinical cases of autistic-spectrum disorder were compared with other clinical and non-clinical cases. Subsequently, the sensitivity and specificity of the instrument were determined, based on the optimal cut-off that had been derived for discriminating autistic-spectrum disorder from other clinical conditions and from normal-range behaviour.
Statistical analysis was conducted using the Statistical Package for the Social Sciences (SPSS version 11 for Windows). Testretest reliability of the SCDC was assessed using intraclass correlation coefficients (ICCs). One-way ICCs were used, to allow for inter-individual variability. Internal consistency was evaluated by calculating Cronbachs a coefficient. Concurrent validity was assessed using one-way analysis of variance (ANOVA), with clinical group as the factor. Tamhanes T2 (Tamhane, 1979) was used as a post hoc test to see which diagnostic groups differed from each other in terms of mean SCDC score.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Internal and external reliability of the SCDC
Cronbachs coefficient for the SCDC was 0.93, showing that
the content of the instrument has high internal consistency. The ICC for
testretest SCDC scores on a clinical sample of 188 children
(Skuse et al, 1997)
with a mean retest interval of 2.7 years (s.d.=0.5, range 1.515.39) was
0.81 (95% CI 0.760.86).
Validity of the SCDC
Concurrent validity
The mean SCDC score for the autistic-spectrum group was 16.6 (s.d.=5.7),
which was significantly higher than that of the clinical control group (mean
score 11.6, s.d.=6.6) and the community control group (mean score 2.9,
s.d.=4.0); one-way ANOVA (F(2,346)=258.72,
P<0.001). However, it should be noted that Levenes test
showed that the assumption of equality of variances had been violated for this
analysis (Levene statistic 19.6, P<0.001). Tamhanes T2 was
therefore used for post hoc comparisons between groups, as this test
is specially designed for situations in which population variances differ, and
is conservative in relation to type 1 error. This post hoc analysis
showed that significant differences in SCDC scores exist between all three
clinical groups: for each group comparison, P<0.001.
Discriminant validity
Discriminant validity was then assessed by determining the power of the
SCDC to distinguish participants with autistic-spectrum disorder from
non-autistic participants, using ROC analysis. This analysis was done in two
parts. First, we found that the SCDC showed impressive accuracy in
discriminating children with an autistic-spectrum disorder from (clinical plus
non-clinical) controls (AOC=0.86, P<0.001). Maximal discrimination
between all pervasive developmental disorder (PDD) diagnoses and non-PDD
diagnoses/normal comparisons was obtained at a cut-off score of 9 points (a
score of 9 or above implied a case). Sensitivity was 0.90 and specificity was
0.69 with this cut-off; the positive predictive validity was 0.75 and the
negative predictive validity was 0.86. Of the 61 false positives obtained with
this cut-off, 19 (31%) were clinical control cases selected from children
attending the Great Ormond Street clinic. These were cases of social
communication difficulty on referral that had already been assessed locally
and were referred to our national centre for a second (or even a third)
opinion. Their presence is likely to have raised the false-positive rate;
among comparisons from the general population the false-positive rate was only
9%. We repeated the ROC analysis excluding data from the general population
sample. The sensitivity of the instrument (with the identical cut-off) was the
same (0.9) but the specificity was reduced to 0.35.
Criterion validity
Finally, criterion validity was assessed by comparing total scores on the
SCDC with the ADIR equivalent algorithm output generated by the 3di,
for the Great Ormond Street sample (n=230), comprising 73 children
with autism, 131 with other PDD diagnoses and 26 without a PDD diagnosis.
These correlations were modest, which is unsurprising in view of the fact that
the items that make up the SCDC were not derived from the ADIR and are
designed to measure autistic traits rather than for diagnostic purposes.
Correlation with the social interaction sub-scale was 0.41
(P<0.001), correlation with the language/communication sub-scale
was 0.30 (P<0.001) and correlation with the repetitive and
stereotyped behaviour sub-scale was 0.21 (P<0.01). The correlation
between the SCDC total and the 3di total score was 0.38
(P<0.001).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Heritability of the SCDC
Autism is a highly heritable disorder, but of the various screening
instruments available only the Social Responsiveness Scale has been evaluated
in terms of formal heritability by means of a twin study. Constantino &
Todd (2003) report a best
fitting model heritability estimate of 0.48, with a sample size of 788 twin
pairs. The best fitting model for SCDC data showed no significant gender
differences and, using similar analyses, a substantially greater heritability
of 0.76 with unique environmental influences of 0.24. A heritability of 0.76
is close to the heritability estimates (about 0.9) reported in twin studies of
clinical cases of autism (Bailey et
al, 1995). No significant influence of the shared environment
emerged in our study, although the upper 95% confidence limit of these
estimates was 0.26 in females and 0.45 in males; therefore, it is possible
that a larger sample size might have detected a more significant effect.
Comparative reliability and validity of the SCDC
The instrument compares well with existing autism screening tools in terms
of its psychometric properties. Internal consistency of the SCDC is very high
(0.93), indicating it has a simple factorial structure, both in studies of
symptomatic cases and in the general population. The derivation of this
12-item questionnaire from principal components analysis of a longer
instrument was described by Skuse et al
(1997). Testretest
reliability (established with an interval of nearly 3 years) has not, to the
best of our knowledge, been evaluated with comparable instruments. The issue
of validity was established in terms of content, concurrent validity,
discriminant and criterion validity. In common with the Social Responsiveness
Scale (Constantino et al,
2003), total scores on the SCDC were independent of IQ. The
sensitivity and specificity values obtained by Berument et al
(1999) for the questionnaire
now known as the Social Communication Questionnaire (0.85 and 0.75
respectively) were closely similar to our own estimates of 0.90 and 0.69
respectively, which were based on a ROC analysis of a sample that contained a
high proportion of children with no psychiatric diagnosis. Because the
inclusion of the latter sample might have led to inflated estimates of the
SCDC performance, we subsequently conducted a further ROC analysis in which
cases of autistic-spectrum disorder were compared with other social
communication and neurodevelopmental disorders. The high sensitivity of the
instrument was replicated (0.90), which is appropriate for a screening
instrument, but specificity was substantially reduced. This is no doubt
because autistic traits are strongly correlated with common problems such as
ADHD (Geurts et al,
2004) and conduct disorder
(Gilmour et al,
2004). We did not anticipate that the SCDC would be suitable for
making discriminations within the autistic spectrum of disorders: such
discrimination was a problem even for the Autism Screening Questionnaire
(Berument et al,
1999), which was designed specifically for application to clinical
populations (Volkmar et al,
2004).
Autism as a dimensional disorder
Increasing evidence supports the hypothesis that autism is a quantitative
or dimensional spectrum, with no clear qualitative distinction between traits
found among individuals with the disorder and the general population. The
majority of people with autism probably have IQ scores in the normal range,
although autistic behaviours may be proportionately more common among those
with learning disabilities (Medical
Research Council, 2001). Is autism unidimensional, as claimed by
Constantino & Todd (2003)
and Spiker et al
(2002)? Contrary evidence is
provided by Silverman et al
(2002), who found that social
and language deficits in autistic disorders were not closely correlated with
stereotyped and repetitive behaviours. Screening questionnaires are generally
not sensitive to the latter dimension of autistic impairment, which has proved
to be problematic, in terms of weak diagnostic differentiation, in studies of
autistic individuals with IQ scores in the normal range (e.g.
Berument et al, 1999). Items concerning such traits are virtually absent from similar screening
instruments (Constantino & Todd,
2003).
Implications
Recent surveys of the prevalence of autism in the community indicate not
only an increase in the number of cases meeting conventional criteria, but a
disproportionate increase in the number of milder cases that fail to reach
full ICD10 or DSMIV (American
Psychiatric Association, 1994) criteria
(Chakrabarti & Fombonne,
2001; Yeargin-Allsopp et
al, 2003). Subclinical cases of autism may present
indirectly, for example with conduct problems at school
(Gilmour et al,
2004). The burgeoning recognition of autistic disorders is putting
a great strain on local services. Rational planning for the likely number of
as-yet-unrecognised cases requires a better estimate than we currently have of
where the boundaries of the autistic spectrum lie. The SCDC, a brief, reliable
and valid screening questionnaire, should finally allow this question to be
answered in the context of a whole-population survey of school-age
children.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Bailey, A., Le Couteur, A., Gottesman, I., et al (1995) Autism as a strongly genetic disorder: evidence from a British twin study. Psychological Medicine, 25, 63-77.[Medline]
Berument, S. K., Rutter, M., Lord, C., et al (1999) Autism screening questionnaire: diagnostic validity. British Journal of Psychiatry, 175, 444 -451.[Abstract]
Chakrabarti, S. & Fombonne, E. (2001)
Pervasive developmental disorders in preschool children.
JAMA, 285, 3093
-3099.
Charman, T., Baron-Cohen, I., Baird, G., et al (2001) Commentary: the Modified Checklist for Autism in Toddlers. Journal of Autism and Developmental Disorders, 31, 145 -148.[CrossRef][Medline]
Cohen, I. L., Schmidt-Lackner, S., Romanczyk, R., et al (2003) The PDD Behavior Inventory: a rating scale for assessing response to intervention in children with pervasive developmental disorder. Journal of Autism and Developmental Disorders, 33, 31 -45.[CrossRef][Medline]
Constantino, J. N. & Todd, R. D. (2003)
Autistic traits in the general population: a twin study. Archives
of General Psychiatry, 60, 524
-530.
Constantino, J.J.N., N., Davis, S. A., A.,Todd, Todd, R. R.D., D., et al (2003) Validation of a brief quantitative measure of autistic traits: comparison of the social responsiveness scale with the autism diagnostic interview revised. Journal of Autism and Developmental Disorders, 33, 427 -433.[CrossRef][Medline]
Ehlers, S., Gillberg, C. & Wing, L. (1999) A screening questionnaire for Asperger syndrome and other high functioning autism spectrum disorders in school age children. Journal of Autism and Developmental Disorders, 29, 129 -141.[CrossRef][Medline]
Fombonne, E. (1991) The use of questionnaires in child psychiatry research: measuring their performance and choosing an optimal cut-off. Journal of Child Psychology and Psychiatry, 32, 677 -693.
Geurts, H. M.,Verte, S., Oosterlaan, J., et al (2004) Can the Childrens Communication Checklist differentiate between children with autism, children with ADHD, and normal controls? Journal of Child Psychology and Psychiatry, 45, 437 -453.
Gilmour, J., Hill, B., Place, M., et al (2004) Social communication deficits in conduct disorder: a clinical and community survey. Journal of Child Psychology and Psychiatry, 45, 967 -978.
Lord, C., Rutter, M. & Le Couteur, A. (1994) Autism Diagnostic Interview Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659 -685.[Medline]
Lord, C., Risi, S., Lambrecht, L., et al (2000) The Autism Diagnostic Observation Schedule Generic: a standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 205 -223.[CrossRef][Medline]
Medical Research Council (2001) MRC Review of Autism Research: Epidemiology and Causes. London: MRC.
Scott, F. J., Baron-Cohen, S., Bolton, P., et al
(2002) The CAST (Childhood Asperger Test): preliminary
development of a UK screen for mainstream primaryschool age children.
Autism, 6, 9
-31.
Scourfield, J., Martin, N., Lewis, G., et al (1999) Heritability of social cognitive skills in children and adolescents. British Journal of Psychiatry, 175, 559 -564.[Abstract]
Scourfield, J., Martin, N., Eley, T. C., et al (2004) The genetic relationship between social cognition and conduct problems. Behavioral Genetics, 34, 377 -383.[CrossRef][Medline]
Silverman, J. M., Smith, C. J., Schmeidler, J., et al (2002) Symptom domains in autism and related conditions: evidence for familiality. American Journal of Medical Genetics, 114, 64 -73.[CrossRef][Medline]
Skuse, D.H., James, R. S., Bishop, D.V., et al (1997) Evidence from Turners syndrome of an imprinted X-linked locus affecting cognitive function. Nature, 387, 705 -708.[CrossRef][Medline]
Skuse, D.,Warrington, R., Bishop, D., et al (2004) The developmental, dimensional and diagnostic interview (3di): a novel computerized assessment for autism spectrum disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 43, 548 -558.[CrossRef][Medline]
South, M.,Williams, B. J., McMahon,W. M., et al (2002) Utility of the Gilliam Autism Rating Scale in research and clinical populations. Journal of Autism and Developmental Disorders, 32, 593 -599.[CrossRef][Medline]
Spiker, D., Lotspeich, L. J., Dimiceli, S., et al (2002) Behavioral phenotypic variation in autism multiplex families: evidence for a continuous severity gradient. American Journal of Medical Genetics, 114, 129 -136.[CrossRef][Medline]
Tamhane, A. C. (1979) A comparison of procedures for multiple comparisons of means with unequal variances. Journal ofthe American Statistical Association, 74, 471 -480.
Volkmar, F. R., Lord, C., Bailey, A., et al (2004) Autism and pervasive developmental disorders. Journal of Child Psychology and Psychiatry, 45, 135 -170.[Medline]
Wechsler, D. (1992) Wechsler Intelligence Scale for Children (3rd UK edn). London: Psychological Corporation.
World Health Organization (1992) International Statistical Classification of Diseases and Related Health Problems (ICD10). Geneva: Geneva: WHO.
Yeargin-Allsopp, M., Rice, C., Karapurkar, T., et al
(2003) Prevalence of autism in a US metropolitan area.
JAMA, 289, 49
-55.
Received for publication July 29, 2004. Revision received January 24, 2005. Accepted for publication January 28, 2005.
Related articles in BJP:
HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Psychiatric Bulletin | Advances in Psychiatric Treatment | All RCPsych Journals |