1 Tampere School of Public Health, FIN-33014 University of Tampere, Finland and 2 STAKES, The National Research and Development Centre for Welfare and Health, PO Box 220, FIN-00531 Helsinki, Finland
* Author to whom correspondence should be addressed at: Tampere School of Public Health, University of Tampere, FIN-33014, Finland. Tel.: +358 3 215 6797; Fax: +358 3 215 6057; Email: tomi.lintonen{at}uta.fi
(Received 11 November 2003; first review notified 3 February 2004; in revised form 13 March 2004; accepted 13 April 2004)
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In a review of methodological issues involved in substance use self-report (Morgan, 1997) the conclusions were: 1) self-reports of substance use are as reliable and valid as most other behaviours, 2) anonymity and confidentiality are sufficient conditions for enhancing validity, 3) other measures (e.g. collateral reports) cannot be assumed to be automatically more valid than self-reports, 4) self-reports have shown highest construct validity. Plant and co-workers (1985)
argued that the overwhelming majority of adolescents were able to provide alcohol drinking data without undue difficulties. Smith et al. (1995)
reported that the questionnaire measures of drinking correlated highly with both collateral and diary measures, but were lower than either of the other two measures. In a clinical sample of adolescents and young adults, more frequent drinking was reported on a diary compared with a retrospective method; the discrepancy was larger among more frequent drinkers (Leigh et al., 1998
). In some studies, biological tests have been used to study the validity of substance use in school surveys (Campanelli et al., 1987
; Kokkevi and Stefanis, 1991
; Wagenaar et al., 1993
; Bailey, 1999
). Harrison (1997)
concluded that self-administrated questionnaires tend to produce more valid data than interviews where the respondents must speak their responses aloud. Using qualitative methodology in Finland, Jaatinen (2000)
noted that the early adolescents were both willing and honest when telling about their drinking.
Regardless of considerable research support for the validity and reliability of adolescents' alcohol self-report measures (e.g. Ahlström et al., 1979; O'Callaghan and Callan 1992
; O'Malley et al., 1983
; Barnea et al., 1987
; Brener et al., 1995
; Torsheim et al., 1997
; Lintonen, 2001
), issues in drinking pattern measurement are constantly being questioned. In a recent paper on the use of qualitative interviews in assessing drinking patterns, Strunin (2001)
argued that the self-report questionnaire method possessed notable shortcomings, especially in the case of adolescents. In his review of the literature, Strunin (2001)
cites several studies reporting inconsistent findings on the measurement of adolescent drinking related behaviours. The neglect of specific drinking settings, the failure to address different beverage types, the temporal nature of adolescent drinking behaviour, and more generally, the use of closed-ended questions were identified as likely causes of unreliability in adolescent drinking measurement. Strunin (2001)
used a method called ethnographic interviewing among Black adolescents in the USA to illustrate the adolescents' interpretation and contextualization of questions about frequency, type and amount of alcohol consumed. His analyses suggest that the use of pre-coded survey methods can distort our understanding of drinking behaviours among adolescents.
Although adolescent drinking self-report reliability has been studied widely, the studies have mostly concerned themselves with testretest and construct reliabilities. The general aim of this present work is to study the multiple form reliability (Bowling, 1991, p. 16) of adolescent self-reports on alcohol drinking in the context of survey research. We will compare drinking estimates concerning the latest drinking occasion based on traditional closed-category questions with estimates encoded from a verbal description of the beverage types and quantities. Two dataset will be utilized: both contain a similar open-ended question and a set of closed questionsthe 1999 set was developed from the 1995 set to obtain a better coverage of different beverage types. More specifically, the research questions are:
![]() |
SUBJECTS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Stratified cluster sampling was used with European Union NUTS2 and urban/rural area division resulting in 10th strata. The Helsinki metropolitan area formed the 11th stratum. The sampling unit was a school class. All the pupils in the class answered the questionnaire. Swedish is a minority language in Finland. To facilitate language group difference analyses, the Swedish-language schools were over-sampled in 1995. Another deviation from a simple random sample was the over-sampling in the Helsinki metropolitan area. The over-sampling was corrected for in the analysis by using stratum weights.
In 1995 10 schools and in 1999 one school refused to participate; in both years, they were replaced by other randomly drawn schools from the same stratum. In line with the ESPAD project requirements, data from adolescents not belonging to the target cohorts (1979 and 1983) were deleted from the data set. The average age of the respondents was 15.3 years. The sample sizes were 121 classes with 2161 pupils in 1995 and 177 classes with 3109 pupils in 1999. The response rates were 94% in 1995 and 90% in 1999. None of the students actually refused to take part so the non-response was due to absence from school or class during the data collection.
The data collection took place at the end of March both in 1995 and 1999. The questionnaire was filled in during a lesson supervised by the pupils' own teacher using a normal written test procedure. The questionnaire was available in both Finnish and Swedish corresponding to their status as official languages in Finland. Confidentiality was emphasised in data collection; for example the anonymous questionnaires were sealed in individual envelopes to be sent to the research centre. After rigorous examination of the completed questionnaires, responses from 12 (0.6%) pupils in 1995 and 17 (0.5%) pupils in 1999 were removed because of poor quality data; in 1999, corrections were made to one or more items on 3% of all data records during this process. More detailed information on the data collection procedures has been published in Ahlström et al. (1997) and (2001)
.
Measurement
The amount of alcohol drunk on the latest drinking occasion was measured both using a set of closed questions and an open question. The closed questions were: The last time you had an alcoholic drink, did you drink any BEER (CIDER/LONG DRINK/WINE/SPIRITS). If so, how much? For cider and long drink (low-alcohol mixture of juice and spirits), and beer in 1999, the options were: I never drink X, I did not drink X on my last drinking occasion, About a bottle or less (<50 cl), 23 bottles or cans (50100 cl), 46 bottles or cans (101200 cl) and More than 6 bottles or cans (200 cl). In 1995, the options for beer drinking were: I never drink beer, I did not drink beer on my last drinking occasion, Less than a bottle or can, 12 bottles or cans, 34 bottles or cans and 5 or more bottles or cans. For wine, the options were: I never drink wine, I did not drink wine on my last drinking occasion, Less than a glass (<10 cl), 12 glasses (1020 cl), Half a bottle (37 cl) and A bottle or more (
75 cl). For spirits, the options in 1995 were: I never drink spirits, I did not drink spirits on my last drinking occasion, Less than a portion (<4 cl), 12 portions (48 cl), 35 portions (920 cl) and 6 portions or more. In 1999, the options for spirits were changed to: I never drink spirits, I did not drink spirits on my last drinking occasion, A portion or less (<5 cl), 2 portions (510 cl), 36 portions (1125 cl) and More than 6 portions (
30 cl). In 1995, the questions covered the use of beer, wine and spirits. The set was expanded in 1999 to include cider and long drinks. Long drinks are ready-made mixtures of spirits and juice with
4.5% ethanol content.
The open question for drinking was as follows: Think back on your latest drinking occasion and describe in your own words as accurately as you can what you drank and how much? (If you shared drinks with other people please try to tell us how much you personally drank). In 1995, the closed questions were on pages 5 and 6 of the 18-page questionnaire followed immediately by the open question. In 1999, the closed questions were on pages 4 and 5, and the open question on page 16 of the 17-page questionnaire.
Lifetime drinking experience was inquired using question On how many occasions (if any) have you had any alcoholic beverage to drink in your lifetime? The options were: Number of occasion: 0, 12, 35, 69, 1019, 2039 and 40 or more. The respondents were asked to estimate their degree of drunkenness on a scale using the question Use a scale below to estimate how drunk you were when you were drunk the last time. The scale consisted of 10 tick boxes arranged horizontally with the instruction on the left end stating Only a bit and on the right end stating Really drunk; I could hardly stand. Below the scale, there was an additional tick box for I've never been drunk.
Analysis
The relationships between the amounts of pure alcohol calculated from the set of closed questions and the open questions were studied using Pearson's correlation analyses. Linear regression lines were fitted into the data; the regression line equations are shown in the scatter diagrams. Spearman's rank order correlation analyses were used when studying the correlations between the beverage-specific closed questions and the corresponding figures derived from the open questions. Analyses were conducted using SAS statistical software taking account of the cluster sampling method and strata weights. Clustering may result in erroneously small deviations and thus SAS PROC SURVEYMEANS was used; it also produced 95% confidence intervals for the estimates. Correlation and regression analyses were done using SAS PROC SURVEYREG. However, when we compared the results with PROC REG and PROC CORR, which do not account for clustering, the results were the same. For technical reasons, clustering could not be accounted for in the analyses of rank order correlations. It is not likely, however, that clustering would have a considerable effect on comparisons of two different ways of measuring single occasion drinking amounts. Stratum weights were used in the analyses with the exception of the analyses of rank order correlations. Paired samples t-test was used to test the statistical significance of differences in mean amounts of alcohol. Comparisons of correlation coefficients were made utilizing the Fisher r-to-Z transformation (Hays, 1988, p. 591).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The estimated amounts of alcohol drunk on the latest drinking occasion derived from a set of closed questions differed from that derived in an open question in the 1995 survey. The 1999 survey suggests that discrepancy may be the result of insufficient coverage of the 1995 closed questionsit left out cider and long drinks (low-alcohol mixtures of juice and spirits). These were both popular beverages (Lintonen and Konu, 2001) that could not be placed in any of the categories offered (beer, wine, spirits). In 1999, with these two beverages added to the set, the two estimates (open/closed question) were the same on the population level. This result highlights the importance of prior theoretical or empirical knowledge on the phenomenon in question. As Pearson's and Spearman's correlation coefficients tend to display somewhat higher estimates of agreement than kappa and intraclass correlation coefficients (ICC), we also performed the analysis using kappa (for Table 1) and ICC (for Table 2) and the results were as expected; they gave a similar overall picture but were, on average, around 0.1 smaller than the corresponding correlations. However, it must be noted that the interpretation of kappa also slightly differs from that of a correlationaltogether, the agreements may be considered good (Cohen, 1960
; Fleiss, 1981
; McGraw and Wong, 1996
).
A noteworthy methodological issue in the survey has to do with the wording of the drunkenness question: ... how drunk you were when you were drunk the last time. With the other question inquiring the last drinking occasion, these two occasions may have been different. However, the meaning of being drunk has been shown to be quite wide-ranging and thus drunkenness is extremely common among Finnish adolescents (Rimpelä et al., 1999; Lintonen and Konu, 2001
)one third of boys and one fourth of girls reported having been really drunk and the majority of the rest slightly drunk on their latest drinking occasion. 76% of the respondents in this present survey answered the question inquiring into the degree of drunkenness on the last drunkenness occasion and the analysis was naturally limited to these respondents. It is, however, possible that the drinking and drunkenness occasions have not been the same, in which case our results would underestimate the accuracy of the adolescents' estimates of drunkenness.
Multiple form reliability indicates that two instruments actually measure the same concept (Bowling, 1991); a high correlation indicates a reliable test. Alternatively to the interpretation as multiple form reliability, the analyses could be seen as a test of concurrent validity (Bowling, 1991
, p. 15) or convergent validity (Bowling, 1991
, p.15; McDowell and Newell, 1996
, p.33). Convergent validity, in this case, comes very close to multiple form reliability in requiring the measure (set of closed questions) to correlate with another measure (the open question) of this same construct. The concurrent validity interpretation would yield information on the issue of whether the open question, hypothesized as having wider coverage of the issue but being more resource consuming could be replaced by a set of closed questions. Based on the analyses conducted in this present paper it can be postulated that the closed questions are at least as good as an open one in estimating group level consumption. It is even possible that the list of beverages shown in the case of a closed set of questions may aid in memorizing the drinking occasion and thus leading to improved coverage, especially when several drink types have been consumed. As already stated, the use of closed questions calls for good prior knowledge of the major beverage types, as was clearly shown in this present study when comparing the years 1995 and 1999.
Harrison (1997) pointed out that according to the social desirability hypothesis, the more stigmatized the drug is, the lower the validity in reporting its use. Possibly adolescents, being as they are in the alcohol experimentation phase, do not view the behaviour either as embarrassing or undesirable. In the case of alcohol in Finland, the norm experienced by the adolescents is actually pro-use (Lintonen and Konu, 2004
), implying no barriers to reporting drinking. At the age of fifteen, reporting substance use is likely to be influenced by peers, especially as the questionnaire is filled in the presence of classmates. A study on the influence of the response context, however, found no difference relating to whether the questionnaire was applied in a school or home context (Needle et al., 1983
). In a methodological study on the effect of administration method in school, Bjarnason (1995)
concluded that it did not make a difference whether the questionnaire was administered by the teacher or a research assistant.
Methodological rigidity has been a key issue in the ESPAD survey system at all levels. Separate methodological studies have been conducted (Johnston et al., 1994; Hibell et al., 2000
) and methodological analyses have been presented in both national (Ahlström et al., 1997
, 2001
) and international reports (Hibell et al., 1997
, 2000
). In the case of the set used in this present study, the data were well representative of all Finnish adolescents belonging to the target cohorts. The response rate, taken as an indication of validity (Single et al., 1975
), has been very good in the Finnish ESPAD survey. Adolescent survey non-respondents have been shown to have a higher likelihood of heavier substance use than those that respond (Grube and Morgan, 1989
; Lintonen et al., 2000
). The analysis of the effect of drinking experience on reporting reliability seems to indicate that our results may be over-estimates of the reliability in the whole adolescent population. Inconsistent answering in the case of alcohol drinking and drunkenness has been shown to be very low in the Finnish ESPAD data set (Ahlström et al., 2001
); in fact, the lowest among all ESPAD countries (Hibell et al., 2000
). Also, according to teachers' assessments, the disturbances during the data collection were scarcepupils were interested in the survey and generally worked seriously (Ahlström et al., 2001
).
The cultural context varies greatly between the ESPAD countries and probably affects the willingness to answer honestly. In Finland, methodological studies on adolescent alcohol surveys have indicated good testretest reliability (Lintonen, 2001). Some country-specific adjustments were made in the ESPAD questionnaire in Finland (as well as in other countries). Since alcopops are virtually unheard of in Finland and long drinks (low-alcohol mixtures of juice and spirits) are rather popular, the corresponding change was made in the set of closed questions. In the closed set of questions, the numbers of bottles, glasses and drinks were adjusted to Finnish standard sizes. In their study comparing Nordic countries with Mediterranean and Eastern European countries, Hibell and co-workers (2000)
observed that while the prevalence estimates remained stable in a 35 day testretest, the self-reports of drunkenness related to amounts drunk were distinctly different across the NorthSouth axis in Europe. Thus the reliability for adolescent self-report on drunkenness shown in this present study may not apply to adolescents in the Southern European countries. Regular surveys on alcohol use among Finnish adolescents have been conducted in Finland since the 1970s, and school surveys are a common phenomenon in Finland. Adolescents in countries less accustomed to surveys and alcohol surveys in general may exhibit quite different patterns of reliability and validity. Care should be exercised when generalizing the findings of this present study to cultures different from the one prevailing in Northern Europe.
Survey self-reports are, and will likely continue to be, the most widely used method of obtaining alcohol use data among adolescents. Designing the measurement tools is a lengthy process involving both theoretical and empirical, possibly qualitative, work. Also, the requirements for correct procedures in sampling, care in data coding and handling must be borne in mind. The main contribution of the present paper to the study of measuring alcohol use is that a well-designed set of closed questions may result in a better coverage of drinking than an open-ended question. As processing open-ended questions leads to more work on behalf of the investigators, efforts would be better targeted at designing rigorously thought-out closed questions.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Ahlström, S., Metso, L. and Tuovinen, E. L. (2001) ESPAD 1995 and 1999 Country Report, Finland, Themes 1/2001. National Research and Development Centre for Welfare and Health, Helsinki, Finland.
Ahlström, S., Eskola, A., Honkala, E., Kannas, L., Paronen, O., Rajala, M., Rimpelä, M. and Telama, R. (1979) The Juvenile Health Habit Study. Study design, methods and material. University of Tampere, Tampere, Finland.
Bailey, S. L. (1999) The measurement of problem drinking in young adulthood. Journal of Studies on Alcohol 60, 234244.[ISI][Medline]
Barnea, Z., Rahav, G. and Teichman, M. (1987) The reliability and consistency of self-reports in substance use in a longitudinal study. British Journal of Addiction 82, 891898.[ISI][Medline]
Bjarnason, T. (1995) Administration mode bias in a school survey on alcohol, tobacco and illicit drug use. Addiction 90, 555559.[CrossRef][ISI][Medline]
Bowling, A. (1991) Measuring health. A review of quality of life measurement scales. Open University Press, Milton Keynes, UK.
Brener, N. D., Collins, J. L., Kann, L., Warren, C. W. and Williams, B. I. (1995) Reliability of the Youth Risk Behavior Survey Questionnaire. American Journal of Epidemiology 141, 575580.[Abstract]
Campanelli, P., Dielman, T. and Shope, J. (1987) Validity of adolescents' self-reports of alcohol use and misuse using a bogus pipeline procedure. Adolescence 22, 722.[ISI][Medline]
Cohen, J. (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 3746.[ISI]
Edwards, G., Anderson, P., Babor, T. F. et al. (1994) Alcohol Policy and the Public Good. Oxford University Press, Oxford, UK.
Feinstein, A. R. (1988) Scientific Standards in Epidemiologic Studies of the Menace of Daily Life. Science 242, 12571263.[ISI][Medline]
Fleiss, J. L. (1981) Statistical Methods for Rates and Proportions. John Wiley, New York, NY.
Grube, J., Morgan, M. and Kearney, K. (1989) Using self-generated identification codes to match questionnaires in panel studies of adolescent substance use. Addictive Behaviors 14, 159171.[CrossRef][ISI][Medline]
Harrison, L. (1997) The Validity of Self-Reported Drug Use in Survey Research: An Overview and Critique of Research Methods. In The Validity of Self-Reported Drug Use: Improving the Accuracy of Survey Estimates, eds L. Harrison and A. Hughes, pp. 1736. National Institute of Drug Abuse Research Monograph 167. NIDA, Rockville, MD.
Hays, W. L. (1988) Statistics. Holt, Reinhardt & Wilson, Chicago, IL.
Hibell, B., Andersson, B., Bjarnason, T., Kokkevi, A., Morgan, M. and Narusk, A. (1997) The 1995 ESPAD report. Alcohol and Other Drug Use Among Students in 26 European Countries. CAN, Stockholm, Sweden.
Hibell, B., Andersson, B., Ahlström, S., Balakireva, O., Bjarnason, T., Kokkevi, A. and Morgan, M. (2000) The 1999 ESPAD report. Alcohol and Other Drug Use Among Students in 30 European Countries. CAN, Stockholm, Sweden.
Jaatinen, J. (2000) Viattomuuden tarinoita. Nuoret päihdekulttuurinsa kuvaajina [Stories of Innocence. Young People as Depicters of Their Alcohol Culture], Reports 251. National Research and Development Centre for Welfare and Health, Helsinki, Finland.
Johnston, L. D., Driessen, F. and Kokkevi, A. (1994) Surveying Student Drug Misuse: A Six-Country Pilot Study. Council of Europe, Strasbourg.
Kokkevi, A. and Stefanis, C. (1991) The epidemiology of licit and illicit substance use among high school students in Greece. American Journal of Public Health 81, 4852.[Abstract]
Leigh, B. C., Gillmore, M. R. and Morrison, D. M. (1998) Comparison of Diary and Retrospective Measures for Recording Alcohol Consumption and Sexual Activity. Journal of Clinical Epidemiology 51, 119127.[CrossRef][ISI][Medline]
Lemmens, P. H. (1994) The alcohol content of self-report and standard drinks. Addiction 89, 17031706.[Medline]
Lemmens, P. H. (1998) Measuring Lifetime Drinking Histories. Alcoholism: Clinical and Experimental Research 22, 29S36S.[ISI][Medline]
Lintonen, T. (2001) Drinking patterns among Finnish fourteen year-olds from 1977 to 1999. Available at http://acta.uta.fi/english/teos.phtml?5561 (accessed 22 October 2003).
Lintonen, T. and Konu, A. (2001) Drunkenness-related alcoholic beverage choices among adolescents. Journal of Substance Use 6, 1621.[CrossRef]
Lintonen, T. and Konu, A. (2004) The misperceived social norm of drunkenness among Finnish adolescents. Health Education Research, 19, 6470.
Lintonen, T. P. and Rimpelä, M. K. (2001) The validity of the concept of "self-perceived drunkenness" in adolescent health surveys. Journal of Substance Use 6, 145150.[CrossRef]
Lintonen, T. P., Rimpelä, M. K., Ahlström, S., Rimpelä, A. H. and Vikat, A (2000) Trends in drinking habits among Finnish adolescents from 1977 to 1999. Addiction 95, 12551263.[CrossRef][ISI][Medline]
Martin, C. S. and Nirenberg, T. D. (1991) Alcohol content variation in the assessment of alcohol consumption. Addictive Behaviors 16, 555560.[CrossRef][ISI][Medline]
McDowell, I. and Newell, C. (1996) Measuring Health A Guide to Rating Scales and Questionnaires, 2nd edition. Oxford University Press, Oxford, UK.
McGraw, K. O. and Wong, S. P. (1996) Forming inferences about some intraclass correlation coefficients. Psychological Methods 1, 3046 (erratum 1, 390).[CrossRef][ISI]
Midanik, L. T. and Hines, A. M. (1991) Unstandard ways of answering standard questions: Protocol analysis in alcohol survey research. Drug and Alcohol Dependence 27, 245252.[CrossRef][ISI][Medline]
Morgan, M. (1997) EMCDDA Instrument Bank: Core Scales, Sources and Guidelines. EMCDDA, Lisbon, Portugal.
Mustonen, H., Metso, L., Paakkanen, P., Simpura, J. and Kaivonurmi, M. (1999) Finnish Drinking Habits in 1968, 1976, 1984, 1992 and 1996. Tables and Publications Based on Finnish Drinking Habit Surveys. Themes 7/1999. National Research and Development Centre for Welfare and Health, Helsinki, Finland.
Needle, R., McCubbin, H., Lorance, J. and Hochhauser, M. (1983) Reliability and Validity of Adolescent Self-Reported Drug Use in a Family-Based Study: A Methodological Report. The International Journal of the Addictions 18, 901912.[ISI][Medline]
O'Callaghan, F. V. and Callan, V. J. (1992) Young adult drinking behaviour: a comparison of diary and quantityfrequency measures. British Journal of Addiction 87, 723732.[ISI][Medline]
O'Malley, P. M., Bachman, J. G. and Johnston, L. D. (1983) Reliability and consistency in self-reports of drug use. The International Journal of the Addictions 18, 805824.[ISI][Medline]
Pernanen, K. (1974) Validity of survey data on alcohol use. In Research Advances in Alcohol and Drug Problems, Vol. 1, Gibbins, R. et al. eds, pp. 355374. Wiley, New York, NY.
Plant, M. A., Peck, D. F. and Samuel, E. (1985) Alcohol, Drugs and School-leavers. Tavistock, London, UK.
Poikolainen, K. and Kärkkäinen, P. (1983) Diary gives more accurate information about alcohol consumption than questionnaire. Drug and Alcohol Dependence. 11, 209216.[CrossRef][ISI][Medline]
Rimpelä, A., Vikat, A., Rimpelä, M., Lintonen, T., Ahlström, S. and Huhtala, H. (1999) Nuorten terveystapatutkimus. Tupakoinnin ja päihteiden käytön muutokset. Aiheita 7/1999. National Research and Development Centre for Welfare and Health, Helsinki, Finland.
Single, E., Kandel, D. and Johnson, B. (1975) The reliability and validity of drug use response in a large-scale longitudinal survey. Journal of Drug Issues 5, 426443.[ISI]
Smith, G. T., McCarthy, D. M. and Goldman, M. S. (1995) Self-Reported Drinking and Alcohol-Related Problems among Early Adolescents: Dimensionality and Validity over 24 Months. Journal of Studies on Alcohol 56, 383394.[ISI][Medline]
Strunin, L. (2001) Assessing alcohol consumption: developments from qualitative research methods. Social Science and Medicine 53, 215226.[CrossRef][ISI][Medline]
Swadi, H. (1990) Validating and Improving the Validity of Self-Reports in Adolescent Substance Misuse Surveys. The Journal of Drug Issues 20, 473486.
Torsheim, T., Wold, B., Samdal, O. and Haugland, S. (1997) Testretest reliability of survey indicators measuring adolescent health and health behaviour. University of Bergen, Bergen.
Wagenaar, A. C., Komro, K. A., McGovern, P., Williams, C. L. and Perry C. L. (1993) Effects of a saliva test pipeline procedure on adolescent self-reported alcohol use. Addiction 88, 199208.[ISI][Medline]