INTERPRETATION OF ITEMS IN THE AUDIT QUESTIONNAIRE

Kypros Kypri*, Rob McGee1, John B. Saunders2, John D. Langley and Johanna I. Dean3

Injury Prevention Research Unit, Department of Preventive and Social Medicine and
1 Department of Preventive and Social Medicine, University of Otago Medical School, P. O. Box 913, Dunedin, New Zealand,
2 Centre for Drug and Alcohol Studies, Department of Psychiatry, School of Medicine, University of Queensland, Brisbane, Qld 4029, Australia and
3 Early Intervention for Psychosis Service, Otago District Health Board, Dunedin, New Zealand

Received 6 December 2001; in revised form 12 February 2002; accepted 28 February 2002


    ABSTRACT
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
Aims: To test for the possibility that tertiary students misinterpret certain items on the Alcohol Use Disorders Identification Test (AUDIT). Methods: Responses to alternative question wordings were compared with responses to standard items. Results: Alterations to items 5 and 9, so that consequences of drinking epitomized in these items were more specifically defined, resulted in markedly different response distributions to the item, but the total AUDIT score was not changed. Conclusions: Caution is necessary before using individual AUDIT items as measures of consequences in population surveys, and the possibility of false positives in total scores should be borne in mind.


    INTRODUCTION
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
In the late 1970s, the World Health Organization (WHO) set about developing an instrument for the purpose of identifying individuals with hazardous or harmful drinking (WHO Expert Committee on Problems Related to Alcohol Consumption, 1980Go). The resulting Alcohol Use Disorders Identification Test (AUDIT) (Saunders and Aasland, 1987Go; Saunders et al., 1993Go) was found to have high specificity and sensitivity in identifying hazardous drinkers in clinical settings (see Allen et al., 1997Go for a review). However, for certain groups, such as university students, its specificity was found to be lower (Fleming et al., 1991Go), such that the test identified a significant proportion (29%) of individuals as hazardous drinkers when they were not so considered by a clinical assessment (i.e. they were ‘false positives’).

This high false positive rate does not necessarily detract from the AUDIT as a screening instrument, if a clinical judgement can be used to confirm hazardous drinking. The AUDIT, however, has been employed in population surveys as an epidemiological measure of hazardous drinking levels (e.g. Fleming, 1996Go; Ministry of Health, 1999Go), and as an outcome measure in analytical studies with specific populations (e.g. Kozyk et al., 1998Go; Claussen, 1999Go). In these contexts, the specificity of the instrument is as important as its sensitivity. The AUDIT has been exhaustively studied in terms of its psychometric qualities and suitability for various subgroups. What has not been examined in any depth since the original developmental work is how those completing the AUDIT interpret the questions.

As part of an ongoing study of tertiary student alcohol consumption, we were interested in exploring aspects of student life that might be adversely affected by hazardous drinking, including fulfilment of role expectations. Our initial experience using the AUDIT in a large self-completed survey of tertiary students (Kypri et al., 2002Go) led us to speculate that there might be variation in respondents’ interpretations of certain items.

Comments made by students during the survey administration indicated that items 5 and 9 had ambiguous meanings. For item 5, which asks ‘How often in the last year have you failed to do what was normally expected from you because of drinking?’, students indicated uncertainty as to the meaning of the question. For item 9, which asks ‘Have you or someone else been injured as a result of your drinking?’, the absence of an injury threshold (e.g. injury requiring treatment), raises questions as to what some respondents may include. On the AUDIT, a score of >=8 is considered to be indicative of hazardous drinking (Conigrave et al., 1995Go). Given the heavy weighting for item 9 (4 points for an answer of ‘yes, during the last year’), variation in what individuals consider worthy of reporting as injury would have important implications for estimating the prevalence of hazardous drinking in a population. We therefore sought to investigate interpretations of AUDIT items, to generate testable hypotheses concerning item wording, and to measure the effect of modifying items 5 and 9 on item scores and the estimated prevalence of hazardous drinking among tertiary students.


    SUBJECTS AND METHODS
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
Focus group studies
Advertisements were placed on notice boards at the University of Otago in Dunedin, New Zealand and at the student job service, for students aged 17–25 years who drank alcohol ‘most weeks’, to participate in ‘a study of student drinking’. A $20 music voucher was offered as reimbursement for participating in a 2-h focus group on drinking, use of health services, and interpretation of AUDIT items previously identified as ambiguous. Seven focus groups were formed, comprising 31 students (20 females and 11 males). Item 5 was displayed and participants were asked to describe the kinds of things they considered in making their response. Item 9 was presented and participants were asked to say what they counted as injury. Students were asked in turn for their views, and group discussion of possible improvements to wording followed.

For item 5, 14 focus group participants said they considered problems they might experience after a heavy night drinking, e.g. missing class the next day or failing to complete an assignment due to a hangover. The remaining 16 said they based their answers on experiences they had while drinking, e.g. behaving in an unruly manner, or stumbling, because of the acute effects of intoxication.

For item 9, 17 participants said they considered only injuries that required medical attention of some description. Eleven participants said they counted relatively minor injuries, for example, grazes or bruises they woke up with after a night of heavy drinking. The remaining two participants took the word ‘injury’ to include verbal insults that might hurt another person’s feelings.

The focus group research led us to generate hypotheses concerning the wording of AUDIT items 5 and 9. In respect to item 5, concerning the failure to fulfil role expectations, on the basis of focus group results we constructed a modified item to examine specific instances of such failings. The standard wording of item 5 was followed by: ‘For example: (a) been late for class; (b) missed a class; (c) failed to complete an assignment on time; (d) been late for work; (e) missed practice or training for a sport; (f) let down a friend; (g) let down a family member’. These examples were derived from focus group discussion and appear to cover four major domains in which tertiary students might be considered to have role expectations: academic commitments (a, b, c); part-time work (d); sport (e); personal and family relationships (f, g). The response categories were the same as for the original AUDIT item. For responses (a) to (g), the highest frequency was taken as the value for the modified item 5. We hypothesized that, by providing examples of the behaviours of interest and thereby discouraging an interpretation in terms of acute effects (e.g. disinhibited behaviour), the mean score would be lower than that for the standard item.

For item 9, concerning alcohol-related injury, we sought to specify a level of injury that required medical attention, and to distinguish injuries experienced by the respondent versus those sustained by others as a result of the respondent’s drinking. For the purpose of specifying the level of injury severity and determining who sustained the injury referred to in item 9, the following questions were asked: ‘As a result of your drinking, have you suffered an injury that required medical attention (e.g. at a hospital, medical centre, your GP’s office, or Student Health)?’, and ‘As a result of your drinking, has someone else suffered an injury that required medical attention (e.g. at a hospital, medical centre, your GP’s office, or Student Health)?’. The highest value response to these two questions was taken as the value for the modified item 9. We hypothesized that, given the higher threshold for injury, the score on our modified item would be lower than that for the standard item.

Survey of student drinking and related harm
To test these hypotheses, we included the standard AUDIT and the modified AUDIT items in a survey of a large sample of tertiary students in Dunedin. The sample and data collection procedures are described in an accompanying paper reporting on that study (Kypri et al., 2002Go). In summary, tertiary students completed surveys at the start of the first semester of the academic year (Time 1) and in the middle of the second semester (Time 2). The Time 2 data alone are utilized in the present study.

Of 1748 forms received from students (97% of the those present at contact), 1672 (96%) met minimum data requirements, i.e. a complete AUDIT, gender, and indication of recent alcohol use. These 1672 cases were utilized for the analyses presented below.

Modified AUDIT items, embedded in a 12-page health questionnaire, were prefaced as follows: ‘The following three questions may seem like a repetition of some asked earlier, but please answer these also. Please do not change your answers to earlier questions based on your answers here’. Responses to the original AUDIT items were compared with responses to the modified items. A total score was produced for the AUDIT and modified AUDIT, and distributions of scores derived from these scales were compared. Paired t-tests were used to examine mean differences.


    RESULTS
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
Table 1Go shows that in response to the modified version of item 5, students more frequently reported failing to fulfil role expectations (59.9 versus 48.5%) as a result of their drinking. In contrast, reports of recent injury were markedly less common for the modified version of item 9 (11.3 versus 21.7%).


View this table:
[in this window]
[in a new window]
 
Table 1. Frequency distributions for standard and modified Alcohol Use Disorders Identification Test (AUDIT) items and hazardous drinking classifications
 
The mean score for the modified item 5 (1.05) was 0.35 of a point higher than that for the standard item (0.70) [t(1654) = 16.2, P < 0.01]. The modified item 9 mean score (0.57) was 0.38 of a point lower than that for the standard item (0.94) [t(1651) = –10.1, P < 0.01]. The total score derived from the modified AUDIT (10.13) was 0.03 of a point lower than that derived from the standard AUDIT (10.16), but this difference was not statistically significant. Table 1Go also shows that 65.3% of males and 55.3% of females scored >=8 (Total = 59.2%) on the standard AUDIT. On the modified AUDIT, 65.5% of males and 56.5% of females scored >=8 (Total = 60.4%).


    DISCUSSION
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
Our first hypothesis was not supported by the results: reports of failure to live up to role expectations were more frequent for the modified version of item 5. Our second hypothesis was supported by the results: reports of alcohol-related injury were less frequent when it was specified that only injuries requiring medical attention were to be included. The combined effect of these two modified items was neutral in relation to total AUDIT scores and had minimal effect on the estimate of the prevalence of hazardous drinking.

Approximately half of the focus group participants gave unexpected interpretations of item 5. Rather than considering failure to fulfil role expectations, these students focused on behaviour that might have been out of character if they had been sober, e.g. stumbling or vomiting. Although this interpretation is technically a miscomprehension, it was sufficiently common to require a rewording of the question. By providing examples of failure to fulfil role expectations, memory was perhaps cued, thus resulting in a measure with greater sensitivity than the standard question.

How other populations might interpret item 5 is unknown. Given the changes in role expectations from adolescence through early and later adulthood (Bachman et al., 1997Go), it may be that this question is understood differently by older people or those in full-time work. Furthermore, if the AUDIT is to be used with school-aged youth, some investigation of this group’s interpretation of AUDIT items is advisable.

One objective of employing the AUDIT in a population survey (e.g. Ministry of Health, 1999Go), is to contribute to official and public understanding of alcohol-related harm and to inform policy decisions. For the purpose of injury prevention, we want to be able to report the incidence of alcohol-related injury that meets a specified minimum level of severity. An AUDIT scale including the modification to item 9 tested in this study would facilitate the collection of 1-year injury incidence and lifetime incidence data. Also, by splitting item 9 into two parts: injury to self and injury to someone else, one can make clearer statements concerning the incidence of alcohol-related injury.

Our review of the literature identified the AUDIT as the best brief instrument available for the purpose of measuring the prevalence of hazardous drinking in a tertiary student population. Although we anticipated problems related to potential ambiguity of two items, the process of inquiry described in this paper revealed the AUDIT to be remarkably robust to varied item interpretations. It is, of course, possible that other AUDIT items are variably interpreted, and that interpretations vary across segments of the wider population. Our study also suggests that use of individual AUDIT items as indicators of particular alcohol-related problems (e.g. item 9 for the incidence of injury) may be problematic. We concur with the suggestion by Ivis et al. (2000) that there is merit in conducting research on the use of the AUDIT in population surveys.


    ACKNOWLEDGEMENTS
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
This research was funded by the Alcohol Advisory Council of New Zealand, the Harold Richardson Trust, and the Health Research Council of New Zealand. The Injury Prevention Research Unit is jointly funded by the Accident Compensation Corporation and the Health Research Council of New Zealand. The first author is grateful for the support of the Health Research Council and a John Dobson Memorial Scholarship.


    FOOTNOTES
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
* Author to whom correspondence should be addressed. Back


    REFERENCES
 TOP
 FOOTNOTES
 ABSTRACT
 INTRODUCTION
 SUBJECTS AND METHODS
 RESULTS
 DISCUSSION
 ACKNOWLEDGEMENTS
 REFERENCES
 
Allen, J. P., Litten, R. Z., Fertig, J. B. and Babor, T. (1997) A review of research on the Alcohol Use Disorders Identification Test (AUDIT). Alcoholism: Clinical and Experimental Research 21, 613–619.[ISI][Medline]

Bachman, J. G., Wadsworth, K. N., O’Malley, P. M., Johnston, L. D. and Schulenberg, J. (1997) Smoking, Drinking, and Drug Use in Young Adulthood: The Impacts of New Freedoms and New Responsibilities. L. Erlbaum Associates, Mahwah, NJ.

Claussen, B. (1999) Alcohol disorders and re-employment in a 5-year follow-up of long-term unemployed. Addiction 94, 133–138.[ISI][Medline]

Conigrave, K. M., Saunders, J. B. and Reznik, R. B. (1995) Predictive capacity of the AUDIT questionnaire for alcohol-related harm. Addiction 90, 1479–1485.[ISI][Medline]

Fleming, J. (1996) The epidemiology of alcohol use in Australian women: findings from a national survey of women’s drinking. Addiction 91, 1325–1334.[ISI][Medline]

Fleming, M. F., Barry, K. L. and MacDonald, R. (1991) The alcohol use disorders identification test (AUDIT) in a college sample. International Journal of Addictions 26, 1173–1185.[ISI][Medline]

Ivis, F. J., Adlaf, E. M. and Rehm, J. (2000) Incorporating the AUDIT into a general population telephone survey: a methodological experiment. Drug and Alcohol Dependence 60, 97–104.[ISI][Medline]

Kozyk, J. C., Touyz, S. W. and Beumont, P. J. (1998) Is there a relationship between bulimia nervosa and hazardous alcohol use? International Journal of Eating Disorders 24, 95–99.[ISI][Medline]

Kypri, K., Langley, J. D., McGee, R., Saunders, J. B. and Williams, S. (2002) High prevalence, persistent hazardous drinking among New Zealand tertiary students. Alcohol and Alcoholism 37, 457–464.[Abstract/Free Full Text]

Ministry of Health (1999) Taking the Pulse: The 1996/97 New Zealand Health Survey, Report HP3261. Ministry of Health, Wellington.

Saunders, J. B. and Aasland, O. G. (1987) WHO Collaborative Project on Identification and Treatment of Persons with Harmful Alcohol Consumption. Report on Phase I. Development of a Screening Tool. World Health Organization, Geneva.

Saunders, J. B., Aasland, O. G., Babor, T. F., de la Fuente, J. R. and Grant, M. (1993) Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption — II. Addiction 88, 791–804.[ISI][Medline]

WHO Expert Committee on Problems Related to Alcohol Consumption (1980) Problems Related to Alcohol Consumption Report of a WHO Expert Committee (WHO Technical Report Series No. 650). World Health Organization, Geneva.