Department of Child and Adolescent Psychiatry, Göteborg University, Sweden
Department of Child and Adolescent Psychiatry, Göteborg University, Sweden and St Georges Hospital Medical School London, UK
Department of Child and Adolescent Psychiatry and Institute of Clinical Neuroscience, Department of Forensic Psychiatry, Göteborg University, Sweden
Correspondence: Henrik Anckarsäter, Forensic Psychiatric Clinic, University Hospital of Malmö, Sege Park, 8A, S-205 02 Malmö, Sweden. Tel: +46 40 334031; fax: +46 40 334127; e-mail: henrik.anckarsater{at}skane.se
Funding detailed in Acknowledgements.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To test a parent telephone interview focused on autism tics, attention-deficit hyperactivity disorder (ADHD) and other comorbidities (ATAC).
Method Parents of 84 children in contact with a child neuropsychiatric clinic and 27 control children were interviewed. Validity and interrater and test retest reliability were assessed.
Results Interrater and test retest reliability were very good. Areas under receiver operating characteristics curves between interview scores and clinical diagnoses were around 0.90 for ADHD and autistic spectrum disorders, and above 0.70 for tics, learning disorders and developmental coordination disorder. Using optimal cut-off scores for autistic spectrum disorder and ADHD, good to excellent kappa levels for interviews and clinical diagnoses were noted.
Conclusions The ATAC appears to be a reliable and valid instrument for identifying autistic spectrum disorder, ADHD, tics, learning disorders and developmental coordination disorder.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The telephone interview is highly structured, with four possible ratings for each item: yes; yes, previously (both scored as 1 in this study); yes, to some extent (scored as 0.5 in this study); and no. It is intended for use with parents as informants and lay persons as interviewers. The interview is preceded by a short introduction to inform the parent that the interview concerns problems or difficulties that the child is either experiencing now or has experienced earlier in life. These problems or difficulties must be pronounced compared with other children of the same age. The parent is also asked to write down the four response alternatives, to have them visually available throughout the interview. In this validation study, the parents were also specifically asked to provide no more facts about the child than those that the interviewer enquired about. This was in order to assure masking of the interviewer to the childs diagnostic status. The time for completing the interview varied from 15 min to 35 min.
Participants
The parents of 118 children and adolescents (aged 718 years) were
asked to participate in the study, and parents of 112 accepted. One of these
had to be excluded because of language difficulties. Of the 111 children, 84
(32 girls and 52 boys, mean age 11.5 years) were patients at the Child
Neuropsychiatric Clinic in Göteborg. They were either under investigation
at the time of the study or had recently been investigated. Children with any
diagnosed or suspected chromosomal or genetic medical disorder other
than high-functioning individuals with fragile X or CATCH 22 (cardiac defects,
abnormal facies, thymic hypoplasia, cleft palate, hypocalcaemia and a deletion
on chromosome 22) were excluded.
Twenty-seven children (10 girls, 17 boys, mean age 12.2 years, range 917) constituted a comparison group of healthy children without any known assessment or treatment for child and adolescent mental health problems. The comparison cases were children of staff at the Child Neuropsychiatric Clinic, the Department of Child and Adolescent Psychiatry and the Department of Forensic Psychiatry in Göteborg, and of their acquaintances. After all the interviews had been completed, parents were again contacted and asked for information about earlier psychiatric problems or contacts with child psychiatry or psychology departments.
Interview procedure
Two medical students (one 4th year, one 5th year) completed the 111
telephone interviews. They were masked to diagnosis of the target cases and to
possible psychiatric history of the comparison cases. The two interviewers
conducted ten of the interviews together, during which they took turns,
interviewing five parents each (all target cases) while the other listened and
filled in the questionnaire independently. The results obtained were then
compared in order to analyse interrater reliability. Ten of the interviewees
(eight target cases, two comparison cases) were contacted again 68
weeks after the first interview and asked to participate in a second
interview; they were informed that the purpose of the second interview was to
determine if responses would vary over time. These parents had not been
informed at the first interview that they would be contacted again. The
interviewers were still masked to diagnoses (target group) as well as to prior
psychiatric problems (comparison group). All clinical information was
collected after all the interviews had been completed.
Diagnostic process
Diagnoses assigned during investigations at the clinic were based on
medical history, physical examination (including a neuromotor assessment) by a
physician with expertise in neuropsychiatry, and psychological examination by
a trained neuropsychologist. In all children, an assessment of cognitive level
was made with a test battery appropriate for the childs mental age
(Doll, 1965;
Griffiths, 1970;
Leiter, 1980;
Wechsler, 1992). Children with
significant school achievement problems were also examined by an educational
specialist using tests of reading and writing skills, observation of the child
at school, and interviews with the childs teachers about school
performance and behaviour. Structured instruments, such as the Autism
Diagnostic InterviewRevised (ADIR;
Lord et al, 1994),
the Diagnostic Interview for Social and Communication Disorders (DISCO;
Leekam et al, 2002;
Wing et al, 2002),
the Childhood Autism Rating Scale
(Schopler et al,
1988), the ASDI (Gillberg
et al, 2001) and the ADHD Rating Scale
(DuPaul et al, 1998)
were used as appropriate, although not the sole basis for a diagnosis. For
each case that fulfilled DSMIV criteria for a specific condition, the
physician in charge was asked to complete a diagnostic protocol specifying
other possible comorbid diagnoses.
Attrition analysis
Six of the initially contacted 118 parents declined to participate in the
study: two lacked motivation for further exploration following the clinical
investigation and diagnosis of their children; one declined owing to a
difficult life situation; and three parents did not supply a reason. One
interview could not be completed owing to language difficulties. All seven
cases of non-completion were from the target group.
Statistical analyses
The interview ratings were coded on a three-point scale: 0 indicating
normality (no), 0.5 indicating some abnormality (yes, to
some extent) and 1.0 indicating abnormality or earlier abnormality
(yes or yes, previously). Sum scores were
calculated for each diagnostic category. Interrater and test retest
reliability was assessed through intraclass correlations between dimensional
ratings within each category. The intraclass correlation coefficient (ICC),
defined as (variance between subject)/(variance between subject+variance of
error), includes both random errors and systematic differences, but is also
dependent on the range of the variable measured. The ICC ranges from 0 (no
agreement) to 1 (perfect agreement); values above 0.75 indicate excellent
reliability, 0.40.75 indicate fair to poor reliability, and values
below 0.4 indicate poor reliability
(Fleiss, 1986). Diagnostic
validity for the neuropsychiatric disorders, where the prevalence of disorders
was sufficiently high for these calculations, were assessed first through a
receiver operating characteristics (ROC) curve, where clinical diagnosis was
the dependent variable and the telephone interview sum score the independent
predictor. The area under the curve (AUC) is a measure of the overall
predictive validity of the instrument where AUC=0.50 signals random
prediction, 0.60<AUC0.70 poor, 0.70<AUC
0.80 fair,
0.80<AUC
0.90 good and AUC>0.90 excellent validity
(Tape, 2004). The inflection
point of the curve is the optimal cut-off value of the dimensional independent
variable for a categorical decision in the dependent variable with maximal
sensitivity and specificity. These cut-offs were then used for calculating
four-field tables comparing the diagnostic results for the telephone
interviews and the clinical assessments through Cohens kappa, values
above 0.60 indicating good correspondence
(Altman, 1991). All statistics
were calculated with the Statistical Package for the Social Sciences, version
11.0, using a significance level of P<0.05.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
Validity in screening and establishing cut-off scores
A ROC curve (Fig. 1)
plotting the sum of the DSMIV criteria (independent variable) and a
diagnosis within the autism spectrum (dependent variable) yielded an AUC of
0.88. The addition of the Gillberg & Gillberg
(1989) criteria for Asperger
syndrome did not improve the screening for any diagnosis in the autism
spectrum, yielding a ROC curve plot with an AUC of 0.88. The best match was
achieved with a cut-off score of 4.5, yielding a four-field table with 34
(31%) true positives, 57 (51%) true negatives, 16 (14%) false positives and 4
(4%) false negatives. Cohens for this model was 0.63
(P<0.001). The sensitivity was 0.89, the specificity 0.78, the
positive predictive value 0.68 and the negative predictive value 0.93. A
cross-tabulation of all specific diagnostic categories within the autism
spectrum with their respective DSMIV criteria in the interview (without
any adjustment of cut-off levels) showed much poorer performance; for autism
=0.22 (P=0.011), for Asperger syndrome
=0.27
(P=0.002) and for pervasive developmental disorders not otherwise
specified
=0.07 (P=0.418).
|
For ADHD the AUC was 0.90 for the DSMIV symptoms and increased to
0.91 with the addition of the ATAC questions Does he/she
alternate between exaggerated activity and passivity? and Does
he/she get excited by having a number of persons around?
(Fig. 2). The optimal cut-off
was eight ATAC symptoms, which yielded a distribution of 58 (52%) true
positives, 36 (32%) true negatives, 12 (11%) false positives and 5 (5%) false
negatives; Cohens =0.68 (P<0.001). The sensitivity
was 0.92, the specificity 0.75, the positive predictive value 0.83 and the
negative predictive value 0.88.
|
For tic disorders (Tourette syndrome or chronic tics) the AUC was 0.84
(Fig. 3) and the optimal
cut-off was two symptoms, which yielded a distribution of 7 (6%) true
positives, 86 (77%) true negatives, 13 (12%) false positives and 5 (5%) false
negatives; =0.35 (P<0.001). The sensitivity was 0.58, the
specificity 0.87, the positive predictive value 0.35 and the negative
predictive value 0.95.
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We are now pursuing the further development of this instrument through the incorporation of more questions under each domain, to provide both screening questions and a wider set of more detailed questions with dimensional symptom ratings for those who screen positive. This instrument will be further validated in other neuropsychiatric patient groups, in general child and adolescent psychiatry groups, and in the normal population.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders (4th edn) (DSMIV). Washington, DC: APA.
Baron-Cohen, S., Allen, J. & Gillberg, C. (1992) Can autism be detected at 18 months? The needle, the haystack, and the CHAT. British Journal of Psychiatry, 161, 839 -843.[Abstract]
Baron-Cohen, S., Wheelwright, S., Skinner, R., et al (2001) The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31, 5 -17.[CrossRef][Medline]
Berument, S. K., Rutter, M., Lord, C., et al (1999) Autism screening questionnaire: diagnostic validity. British Journal of Psychiatry, 175, 444 -451.[Abstract]
Doll, E. (1965) Vineland Social Maturity Scale. Circle Pines, MN: American Guidance Service.
DuPaul, G. J., Power, T. J., Anastopoulus, A. D., et al (1998) ADHD Rating ScaleIV. Checklists, Norms and Clinical Interpretation. New York: Guilford.
Ehlers, S. & Gillberg, C. (1993) The epidemiology of Asperger syndrome. A total population study. Journal of Child Psychology and Psychiatry, 34, 1327 -1350.[Medline]
Fleiss, J. L. (1986) Reliability of measurements. In The Design and Analysis of Clinical Experiments (ed. J. L. Fleiss), pp. 2 -31. New York: Wiley.
Gillberg, I. & Gillberg, C. (1989) Asperger syndrome some epidemiological considerations: a research note. Journal of Child Psychology and Psychiatry, 30, 631 -638.[Medline]
Gillberg, C., Gillberg, I. C., Råstam, M., et al
(2001) The Asperger Syndrome (and high-functioning autism)
Diagnostic Interview (ASDI): a preliminary study of a new structured clinical
interview. Autism, 5, 57
-66.
Griffiths, R. (1970) The Abilities of Young Children. A Study in Mental Measurement. London: University of London Press.
Holmes, J., Lawson, D., Langley, K., et al
(2004) The Child Attention-Deficit Hyperactivity Disorder
Teacher Telephone Interview (CHATTI): reliability and validity.
British Journal of Psychiatry,
184, 74-78.
Kadesjo, B., Janols, L. O., Korkman, M., et al (2004) The FTF (Five to Fifteen): the development of a parent questionnaire for the assessment of ADHD and comorbid conditions. European Child and Adolescent Psychiatry, 13 (suppl. 3), 3 -13.[Medline]
Leekam, S. R., Libby, S. J., Wing, L., et al (2002) The Diagnostic Interview for Social and Communication Disorders: algorithms for ICD10 childhood autism and Wing and Gould autistic spectrum disorder. Journal of Child Psychology and Psychiatry, 43, 327 -342.
Leiter, R. G. (1980) Leiter International Performance Scale. Instruction Manual. Chicago: Stoelting.
Lord, C., Rutter, M. & Le Couteur, A. (1994) Autism Diagnostic Interview Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659 -685.[Medline]
Nadder, T. S., Silberg, J. L., Eaves, L. J., et al (1998) Genetic effects on ADHD symptomatology in 7- to 13-year-old twins: results from a telephone survey. Behaviour Genetics, 28, 83 -99.[CrossRef]
Rohde, P., Lewinsohn, P. M. & Seely, J. R. (1998) Comparability of telephone and face-to-face interviews in assessing axis I and II disorders. American Journal of Psychiatry, 154, 1593 -1598.
Schopler, E., Reichler, R. J. & Renner, B. R. (1988) The Childhood Autism Rating Scale (CARS). Los Angeles: Western Psychological Services.
Tape, T. G. (2004) Interpreting Diagnostic Tests. University of Nebraska Medical Center: http://gim.unmc.edu/dxtests
Wechsler, D. (1992) Manual for the Wechsler Intelligence Scale for Children (3rd edn). New York: Psychological Corporation.
Wing, L., Leekam, S. R., Libby, S. J., et al (2002) The Diagnostic Interview for Social and Communication Disorders: background, inter-rater reliability and clinical use. Journal of Child Psychology and Psychiatry, 43, 307 -325.[Medline]
Received for publication March 19, 2004. Revision received September 14, 2004. Accepted for publication September 29, 2004.
Related articles in BJP:
HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Psychiatric Bulletin | Advances in Psychiatric Treatment | All RCPsych Journals |