Section of Child and Adolescent Psychiatry, Department of Psychological Medicine, University of Wales College of Medicine, Cardiff
Section of Child and Adolescent Psychiatry, Department of Psychological Medicine, University of Wales College of Medicine, Cardiff
Section of Child and Adolescent Psychiatry, Department of Psychological Medicine, University of Wales College of Medicine, Cardiff
University of Manchester Department of Child and Adolescent Psychiatry, Royal Manchester Childrens Hospital, Manchester
University of Manchester Department of Child and Adolescent Psychiatry, Royal Manchester Childrens Hospital, Manchester
Section of Child and Adolescent Psychiatry, Department of Psychological Medicine, University of Wales College of Medicine, Cardiff
University of Manchester Department of Child and Adolescent Psychiatry, Royal Manchester Childrens Hospital, Manchester
Section of Child and Adolescent Psychiatry, Department of Psychological Medicine, University of Wales College of Medicine, Cardiff
Correspondence: Anita Thapar, Section of Child and Adolescent Psychiatry, Department of Psychological Medicine, University of Wales College of Medicine, Heath Park, Cardiff CF14 4XN, UK
Declaration of interest Work funded by the Wellcome Trust, Action Research and Sparks.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Aims To examine the stability, testretest reliability and criterion validity of the CHATTI for children referred with a suspected diagnosis of ADHD.
Method Data were obtained from 79 teachers, of whom 36 were interviewed on two occasions.
Results Overall, the CHATTI shows good stability, testretest
reliability and criterion validity for symptom scores. Testretest
reliability for some individual items was low. Reliability for the
operationalised criteria of pervasiveness (i.e. symptoms at
school and home) and school impairment was excellent
(=1).
Conclusions The CHATTI appears to be a promising tool for assessing ADHD symptoms in a school setting and could be useful in clinical as well as research settings.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
METHOD |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Procedure and sample characteristics
The parents of each child were asked for written parental consent to
contact the school. This consent form was then posted to the childs
school, together with a study information sheet, teacher consent form (giving
us permission to contact them by telephone to administer the CHATTI) and a
short questionnaire package that contained the Abbreviated Conners Teacher
Ratings Scale (ACTRS; Conners,
1973), which at the time of the study was one of the rating scales
most commonly used by clinicians in the UK, and the DuPaul ADHD rating scale
(DuPaul, 1981). A reminder
letter was sent to teachers who failed to respond to the first mailshot. The
return rate for the questionnaires was 94% (79/84). Data from the teachers of
this sample of 79 children (73 males; 6 females) aged between 6 and 13 years
(mean=8.76; s.d.=1.75) were used for the purposes of assessing the criterion
validity of the CHATTI.
Once the consent form and questionnaire package had been returned, the researcher telephoned the teacher to carry out the CHATTI. The interview and questionnaires were always completed by the same teacher, typically the childs class teacher because the majority of children who participated in the study were in primary school. However, for those children in secondary school the teacher who had the most extensive knowledge of the childs behaviour was chosen to complete the interview and questionnaires. Phase 1 of the study involved interviews with 79 teachers, of whom 20 were reinterviewed one week later by the same experienced interviewer (a research psychologist (J.H.) and two child psychiatrists (A.Tr., H.F.), who had all been trained previously to use a research diagnostic interview). Phase 2 of the study involved a further 16 interviews (new sample) undertaken with the same teacher 1 week apart by two different interviewers (one of whom had been trained in the CAPA (D.L.) and the other who was an assistant psychologist (H.P.) who had not received training in diagnostic interviews).
All teachers who consented to the study (94%) were interviewed. The interviews took place during regular school hours. The children in these groups were aged between 6 and 13 years (mean=9.13, s.d.=1.7; 32 males, 4 females) and fulfilled either DSMIIIR or DSMIV criteria for ADHD or ICD10 criteria for hyperkinetic disorder.
Measures
The CHATTI is a structured interview that takes 1520 min to complete
and should be conducted with the teacher who has the most extensive knowledge
of the childs behaviour (class teacher for primary school children).
The interview focuses on the occurrence of ADHD symptoms during the preceding
3 months. The CHATTI contains 18 items included in the ICD10 criteria
for hyperkinetic disorder and DSMIV diagnostic criteria for ADHD.
The CHATTI is divided into three overall symptom areas: inattention (e.g. difficulty concentrating); hyperactivity (e.g. fidgets or squirms in seat); impulsiveness (e.g. interrupts or intrudes on others). Each symptom is explicitly defined and teachers are asked whether the index child shows the symptom in different lessons and to a greater extent than other children of his/her developmental age. In addition to the 18 ADHD items, the CHATTI includes a question on whether the ADHD symptoms cause significant impairment in the childs social or academic functioning at school.
Symptom scores were summed to provide continuous measures, a total ADHD score and scores from the three sub-scales, namely inattention, hyperactivity and impulsiveness.
In this study the interview was developed to be used in conjunction with the parent version of CAPA (Angold et al, 1995) but could be used with other similar diagnostic interviews. Again in this study, diagnoses were primarily based on parent-derived interview data with the teacher information used only to define the ICD10 and DSMIV criterion of pervasiveness (i.e. symptoms at school as well as at home), but the instrument could be used differently. At the start of the study, the criterion of symptom pervasiveness for ICD10 hyperkinetic disorder was operationally defined by a consensus of experienced child psychiatrists as the presence of at least one definite symptom from each of the symptom areas (i.e. inattention, hyperactivity, impulsiveness) reported by the teacher, with associated impairment in functioning in school in addition to meeting the diagnostic criteria at home using parental interviews. For DSMIV ADHD, the criterion of some impairment from the symptoms is present in two or more settings was rated using the response to the question on impairment of functioning.
Statistics
Symptom scores (categorical data) and scale scores (continuous data) were
generated from the interview data. For the reliability analysis, Cohens
(Cohen, 1960) was used
to assess agreement on categorical variables, whereas the scale score
agreement was measured by the intraclass correlation coefficient (ICC)
(Everitt, 1996). Criterion
validity was assessed by investigating the association between scores on the
CHATTI and ACTRS (Conners,
1973) and the DuPaul ADHD scale
(DuPaul, 1981). Internal
consistency also was checked using Cronbachs
coefficient.
Within-subject associations of measures were investigated using
Spearmans correlations, because questionnaire scores from the DuPaul
rating scales and ACTRS were negatively skewed.
All statistical tests were considered significant at P<0.05. Two-tailed P values are presented. Statistical analyses were carried out using the Statistical Package for the Social Sciences, Windows version 7.5 (SPSSW; SPSS Inc).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Stability of the CHATTI across time same interviewer
Table 1 shows the agreement
across time for scores on the CHATTI total scale and sub-scales. The ICCs
ranged from 0.94 to 0.98. Kappa coefficients were then calculated to assess
agreement between time 1 and time 2 for each individual item and for the
criterion of pervasiveness.
Table 2 shows the stability of
the criteria of pervasiveness (1.0) and impairment from
symptoms (1.0) necessary to make a diagnosis of ICD10
hyperkinetic disorder and DSMIV ADHD, respectively. According to the
benchmarks provided by Landis & Koch
(1977), the strength of
agreement for CHATTI items ranged from fair (0.35 for avoids
tasks) to perfect (1.0 for cant wait turn).
Cronbachs was 0.91 for internal consistency. Some researchers
operationalise the ICD10 criterion of pervasiveness as requiring that
the full ICD10 criteria for hyperkinetic disorder (i.e. at least six
symptoms of inattention, three symptoms of hyperactivity and one symptom of
impulsivity) are met at school (as well as at home). Therefore we also
examined the stability of this stricter definition of
pervasiveness. The
coefficient was 0.79.
|
|
Testretest reliability across time two different interviewers
Intraclass correlations for total and sub-scores for different raters
across time are shown in Table
1. These range from 0.76 to 0.92.
Table 3 shows that agreement
across time for the criteria of impairment (1.0) and
pervasiveness was perfect (1.0). However, coefficients
for individual items were very variable (ranging from 0.16 for avoids
tasks to 0.87 for constantly on the go). The
testretest reliability for the stricter definition of
pervasiveness was 0.71.
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Although gathering information from teachers by telephone may occur commonly in clinical situations, to our knowledge there has been only one published report describing the use of a structured teacher telephone interview as an adjunct for making the diagnosis of ADHD for research purposes (Tannock et al, 2000). Although psychometric data have not been published, this instrument has been found to be a useful adjunct to parent interviews (R. Tannock, personal communication, 2003).
Although it is usual to consider interview methods as the gold standard for assessing psychopathology it is still essential to assess the reliability, validity and acceptability of a new interview-based instrument. We sought to examine criterion validity using two commonly used questionnaires. The CHATTI was found to be strongly correlated with the ADHD rating scale (DuPaul, 1981) and moderately correlated with the ACTRS (Conners, 1973). The observed strong correlations between the CHATTI and the DuPaul ADHD rating scale are not surprising, given that both measures include the DSMIIIR symptoms of ADHD.
The CHATTI was found to yield highly consistent results across a 1-week
testretest period. Specifically, the CHATTI showed excellent
reliability for our operationalised definition of symptom pervasiveness
(=1.00) and high stability for total and sub-scale scores based on
symptom counts and the stricter definition of pervasiveness.
Testretest reliability for total CHATTI scores (0.98) are similar or
higher than those for the DuPaul ADHD scale (0.96) and ACTRS (0.70.90).
Moreover, the CHATTI showed acceptable levels of stability even at the
individual symptom level. Kappa coefficients for the majority of individual
symptoms ranged between moderate to perfect agreement.
Testretest reliability over time with two different interviewers was also examined. Here, the ICCs for symptom scores were still high and reliability for the categories of symptom pervasiveness and impairment was perfect and for the stricter definition of pervasiveness was good. Nevertheless, reliability for individual items was highly variable and for some items it was low. We conclude that one contributory factor to this may have been the choice of interviewers. One of the two interviewers was a trained interviewer whereas the other was a psychology assistant who had not been trained in research diagnostic interviews. However, this approach was adopted not only for practical reasons (availability of researcher time) but also to consider whether it would be feasible for a clinician untrained in research diagnostic interviews to use this instrument.
Overall initial findings suggest that the CHATTI is cost- and time-efficient and acceptable to teachers. It provides a highly stable measure of symptom pervasiveness and teacher-reported total ADHD symptom scores and impairment at school. Although questionnaires are easy to use and cheap to administer, and many of them show high reliability, they can be inaccurate at identifying individuals as hyperactive and can be subject to rater biases and poor response rates (Taylor, 1994; Conners et al, 1998). Moreover, it is not clear how to integrate questionnaire-derived data with parent interviews to generate the criteria of symptom pervasiveness or impairment in two or more settings reliably. The CHATTI represents an attractive alternative to teacher questionnaires, particularly when a systematic method is required to be used in conjunction with a standard parent diagnostic interview for assigning the diagnosis of ADHD or hyperkinetic disorder. The CHATTI also provides an alternative means of assessing symptoms in studies focusing on teacher-reported ADHD symptoms and in clinical settings. Indeed, it can be argued that in clinical settings, for children with suspected ADHD, early clinician contact with schools by telephone rather than by letter is highly desirable for assessment and treatment purposes.
One limitation of this study is that data were collected from a clinic sample of children with suspected ADHD, nearly all of whom fulfilled the diagnostic criteria for hyperkinetic disorder or ADHD. Diagnostic severity may influence the measurement of reliability, with reliability coefficients being higher in more severely affected groups (Jensen et al, 1995). Thus, it is important to examine the psychometric properties of the CHATTI within a non-clinic sample and in children with other diagnoses before it can be recommended for widespread use in other populations. Further research also will be necessary to investigate the discriminant validity of the CHATTI to differentiate children with ADHD from other clinic groups, such as those with oppositional defiant disorder, anxiety and depression. However, we suggest that it is most useful when used as an adjunct to parent interviews to assess the presence of ADHD symptoms or impairment in more than one setting rather than as a diagnostic tool in itself.
In summary, with the advent of ICD10 and DSMIV, clinicians and researchers are required to assess the presence of hyperactive, impulsive and inattentive symptoms or impairment across settings, in order to determine a diagnosis of hyperkinetic disorder or ADHD. Research findings also suggest the importance of using multiple informants for the diagnosis of hyperactivity in order to reduce rater biases and discrepancies between parent and teacher ratings of hyperactivity (Simonoff et al, 1998; Mitsis et al, 2000). The CHATTI is a new instrument designed for the assessment of ADHD symptomatology within school settings. Preliminary data suggest that the CHATTI shows acceptable reliability in clinical samples with suspected ADHD, it is easy to use and it is acceptable to teachers.
![]() |
Clinical Implications and Limitations |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
LIMITATIONS
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
American Psychiatric Association (1987) Diagnostic and Statistical Manual of Mental Disorders (3rd edn, revised) (DSMIIIR). Washington, DC: APA.
American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders (4th edn) (DSMIV). Washington, DC: APA.
Angold, A., Prendergast, M., Cox, A., et al (1995) The Child and Adolescent Psychiatric Assessment. Psychological Medicine, 25, 739 753.[Medline]
Barkley, R. A. (1998) Attention-Deficit Hyperactivity Disorder (2nd edn). New York: Guilford Press.
Cohen, J. (1960) A coefficient of agreement for nominal scales. Educational Psychological Measures, 20, 37 46.
Conners, C. (1973) Psychological assessment of children with minimal brain dysfunction. Annals of the New York Academy of Sciences, 205, 283 302.[Medline]
Conners, C., Sitarenios, G., Parker, J. D., et al (1998) Revision and restandardization of the Conners Teacher Ratings Scale (CTRSR): factor structure, reliability and criterion validity. Journal of Abnormal Child Psychology, 26, 279 291.[CrossRef][Medline]
DuPaul, G. (1981) Parent and teacher ratings of ADHD symptoms: psychometric properties in a community based sample. Journal of Clinical Child Psychology, 20, 245 253.
Everitt, B. S. (1996) Making Sense of Statistics in Psychology: A Second-level Course. New York: Oxford University Press.
Holmes, J., Payton, A., Barrett, J. H., et al (2000) A family-based and casecontrol association study of the dopamine D4 receptor gene and dopamine transporter gene in attention deficit hyperactivity disorder. Molecular Psychiatry, 5, 523 530.[CrossRef][Medline]
Jensen, P., Roper, M., Fisher, P., et al (1995) Testretest reliability of the Diagnostic Interview Schedule for Children (DISC 2.1). Parent, child, and combined algorithms. Archives of General Psychiatry, 52, 61 71.[Abstract]
Landis, J. R. & Koch, G. G. (1977) The measurement of observer agreement for categorical data. Biometrics, 33, 159 174.[Medline]
Mitsis, J. M., McKay, K. E., Schulz, K. P., et al (2000) Parentteacher concordance for DSMIV attention-deficit/hyperactivity disorder in a clinic-referred sample. Journal of the American Academy of Child & Adolescent Psychiatry, 39, 308 313.
Nadder, N. S., Silberg, J., Eaves, L., et al (1998) Genetic effects on ADHD symptomatology in 7-to-13-year-old twins: results from a telephone survey (1998). Behavior Genetics, 28, 83 99.[CrossRef][Medline]
Rohde, P., Lewinsohn, P. M. & Seeley, J. R.
(1997) Comparability of telephone and face-to-face interviews
in assessing axis I and II disorders. American Journal of
Psychiatry, 154, 1593
1598.
Simonoff, E., Pickles, A., Hervas, A., et al (1998) Genetic influences on childhood hyperactivity: contrast effects imply parental rating bias, not sibling interaction. Psychological Medicine, 28, 825 837.[CrossRef][Medline]
Tannock, R., Hum, M., Maselli, M., et al (2000) Teacher Telephone Interview. Toronto: The Hospital for Sick Children.
Taylor, E. (1994) Syndromes of attention deficit and overactivity. In Child and Adolescent Psychiatry (eds M. Rutter, E. Taylor & L. Hersov), vol. 3 , pp. 285307. Oxford: Blackwell Scientific Publications.
Taylor, E., Sandberg, S., Thorley, G., et al (1991) The Epidemiology of Childhood Hyperactivity. New York: Oxford University Press.
Wechsler, D. (1992) Wechsler Intelligence Scale for Children: UK (3rd edn). London: Psychological Corporation.
World Health Organization (1992) The ICD10 Classification of Mental and Behavioural Disorders. Geneva: WHO.
Received for publication January 14, 2003. Revision received May 28, 2003. Accepted for publication August 12, 2003.
Related articles in BJP: