Intervention-related Bias in Reporting of Food Intake by Fifth-Grade Children Participating in an Obesity Prevention Study

Lisa Harnack1 , John H. Himes1, Jean Anliker2, Theresa Clay3, Joel Gittelsohn4, Jared B. Jobe5, Kimberly Ring6, Pat Snyder1, Janice Thompson7 and Judy L. Weber8

1 Division of Epidemiology, School of Public Health, University of Minnesota, Minneapolis, MN.
2 Department of Nutrition, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA.
3 Prevention Research Center, Health Sciences Center, University of New Mexico, Albuquerque, NM.
4 Department of International Health, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD.
5 Division of Epidemiology and Clinical Applications, National Heart, Lung, and Blood Institute, Bethesda, MD.
6 Department of Biostatistics, Collaborative Studies Coordinating Center, University of North Carolina, Chapel Hill, NC.
7 Office of Native American Diabetes Programs, Health Sciences Center, University of New Mexico, Albuquerque, NM.
8 Department of Pediatrics/Center for Applied Research and Evaluation, University of Arkansas for Medical Sciences, Little Rock, AR.

Received for publication March 11, 2004; accepted for publication June 25, 2004.


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Data collected as part of Pathways, a school-based trial for the primary prevention of obesity in American Indian children conducted between 1997 and 2000, were analyzed to examine possible intervention-related bias in food reporting. The authors hypothesized that children in the intervention schools may have systematically underreported their dietary intake relative to children in the control schools. Nutrient intake estimates for lunch derived from record-assisted 24-hour dietary recalls were compared with intake estimates from observed lunch intakes. Reported nutrient intakes were included in regression analyses as the dependent variables; observed intake, intervention condition, and age were included as independent variables. Results indicated that, among females, intervention condition was a significant predictor of reported energy, fat, and saturated fatty acid intakes. Independently of observed intake, reported lunch energy intake among females in the intervention schools was 66.8 calories lower than reported intake among females in the control schools (p = 0.03). These findings suggest that investigators should consider bias in reporting of dietary intake by intervention condition when conducting diet-focused intervention studies. Specifically, enhancing measures that rely on self-reports with objective measures of dietary intake would help investigators to evaluate whether differential reporting by treatment group has occurred.

bias (epidemiology); intervention studies; nutrition assessment


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Pathways, a school-based trial for the primary prevention of obesity in American Indian children, was one of the first clinical trials in the United States to evaluate a comprehensive program for childhood obesity prevention. The effectiveness of the Pathways program was evaluated by comparing average percentages of body fat among children in the control and intervention schools after 3 consecutive years of intervention. Secondary outcomes included the children’s dietary intake (assessed by direct observation at school lunch and record-assisted 24-hour dietary recall), menu analysis of school lunches, physical activity (assessed by activity monitors and questionnaire), and knowledge, attitudes, and behaviors (assessed by questionnaire).

Pathways Study results showed that the intervention program produced significant reductions in total energy intake. Average total energy intake was 263 kcal/day lower in the intervention group than in the control group (1). However, there were no differences in average percentage of body fat between the intervention and control groups (2), raising concerns over possible intervention-related bias in reporting of food intake with the 24-hour dietary recall method (3). More specifically, it is possible that children in the intervention schools may have been more likely than children in the control schools to underreport their food intake. Several other nutrition intervention studies in children have found changes in self-reported dietary intake in the absence of changes in adiposity (46), which raises further suspicion of intervention-related bias in reporting of dietary intake.

Data collected as part of the Pathways Study included both subjective (24-hour dietary recall) and objective (direct observation at school lunch) measures of participants’ dietary intake. This presented us with a unique opportunity to examine possible intervention-related bias in reporting of food intake. To our knowledge, no previous investigations have examined bias in food intake reporting by children in clinical trials. We analyzed data collected as part of the Pathways Study to test the hypothesis that bias in reporting of dietary intake occurred with the record-assisted 24-hour dietary recall method, with children in the intervention schools systematically underreporting their dietary intake relative to children in the control schools.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Subjects
The design and content of the Pathways intervention have been described in detail elsewhere (7, 8). In brief, 41 elementary schools in seven American Indian communities were randomized to intervention and control conditions. The intervention included a classroom curriculum, physical education, school food service, and family components. The nutritional aspects of the curriculum and the family component emphasized making lower-fat food choices and learning healthy eating behaviors. The food-service intervention included nutrient and behavioral guidelines targeted toward reducing the amount of fat in school meals. The intervention began in the fall of the third grade (1997) and continued through the fifth grade (2000).

The institutional review boards at each participating university and tribe approved the study protocol, as well as the manner in which parental informed consent and children’s assent were obtained.

Measurements
Children were enrolled in the study and baseline measurements were completed at the end of the second grade, with selected measurements being conducted each spring through the fifth grade (follow-up). The impact of the Pathways intervention on diet was assessed in two ways: direct observation of children eating school lunches and record-assisted 24-hour dietary recalls. School lunch observation was chosen because it provides an objective estimate of intake unbiased by self-reporting (9, 10). Record-assisted 24-hour dietary recalls were used because they provide assessments of dietary intake for the entire day (11).

School lunch observation was conducted each spring on a sample of approximately 15 children in each school. At baseline, the children for the lunch observations were randomly chosen within each school. In subsequent years, as many of the same children were observed as possible, with additional children being chosen at random when the cohort children were not available. Lunch observations were made on at least two separate days in each school.

The approach of Gittelsohn et al. (9) served as the model for lunch observations. Observers evaluated each child’s initial serving relative to a standard tray with expected foods and serving sizes. Each data collector observed up to three children at a time as inconspicuously as possible, recording portion sizes, food choices, food brought from home, food trading, food spillage, and second servings. Following lunch, amounts of food remaining on the children’s plates were measured using standardized utensils and recorded, thus allowing for calculation of food consumed. Data collectors were centrally trained and certified by two of the authors. To ensure a high skill level, observers were trained until they could visually estimate portions with less than 20 percent error, and they were graded against observations of the same children by expert observers.

Record-assisted 24-hour dietary recalls were administered during the spring of the fifth grade. The decision to use this posttest-only design for the recalls was based on anticipated difficulties in obtaining valid recalls from second-grade children at baseline (12, 13) and because of additional expense. Record-assisted dietary recalls were collected from random samples of approximately 15 children per school.

The recalls were collected on the day following the observation of the child’s eating; hence, the recall information included observed lunch intake. Prior to the 24-hour recall period, children were trained to complete an abbreviated food record for use as a memory prompt during their recall interviews the next day.

Recalls were conducted in person using the Nutrition Data System for Research (version 4.02_30), a computer-based software application that allows for direct entry of dietary data in a standardized fashion (14). The multiple-pass interview technique was used to prompt children for complete food recall and descriptions. A variety of food-portion visual aids were available for use by participants in reporting portion size. Centralized training, certification of data collectors, and quality control were provided by staff at the University of Minnesota Nutrition Coordinating Center. To minimize the likelihood of bias in recording or reporting of food intake, intervention staff were not allowed to serve as data collectors.

Vendor products, menus, and recipes for food served in each school were collected for 5 days, including those days on which lunch observations and 24-hour dietary recalls were conducted. The menus and recipes were entered into the Nutrition Data System for Research and used as appropriate in the lunch observations and dietary recalls. Thus, nutrient intake calculations for the lunch observations and record-assisted 24-hour dietary recalls reflect as closely as possible the actual recipes and food preparation practices used at the child’s school, rather than fixed default values from the food database. This aspect was important because of the school food-service intervention activities and the varying recipes and food preparation practices among schools (15).

Data analysis
The cohort included in the analyses presented in this paper comprised study participants for whom 24-hour dietary recall and lunch observation measurements were collected concurrently during the final follow-up measurement period (n = 608).

Because of the sampling frame and study design, appropriate estimates of mean values, variances, and tests of differences required restricted maximum likelihood solutions for mixed models. Conceptually, the models allowed for comparisons between nutrient estimates derived from the two types of intake measures (observed and recall) in the same children. Children were nested within the 41 schools, schools within intervention conditions, and conditions within the four field sites. This structure allowed for correlations within nested groups and accounted for differences among groups. The resulting mixed models were constructed with PROC MIXED in the Statistical Analysis System (version 6.12; SAS Institute, Inc., Cary, North Carolina).

Regression parameters were derived from mixed models accommodating the study design features as described above. Of particular interest was the association of intervention conditions with systematic bias in reporting of nutrient intake, as estimated from recalled intake adjusted for observed intake in multiple regression analysis. For these regression analyses, nutrient intakes from recalls were the dependent variables; nutrient intakes from observations, child age, child gender, and intervention condition were fixed effects; and school and field site were random effects.

Because the distributions of intakes for specific nutrients departed substantially from a Gaussian distribution, power transformations were applied to approximate normality, and tests of significance were conducted on the transformed variables. For the analyses, intakes of saturated fatty acids, iron, vitamin C, and ß-carotene were log-transformed, and intakes of carbohydrate, calcium, and vitamin A were square-root-transformed.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
To examine the extent to which differences between reported and observed nutrient intakes for the lunch meal could be explained by intervention condition, we fitted linear regression models for each nutrient as follows: recall = observed + condition + sex + school(random) + site(random) = e. We included an intervention x sex interaction variable in the model to consider the possibility that bias in reporting might differ by sex. For all nutrients examined, the partial regression coefficients for intervention condition were not statistically significant; however, a significant intervention x sex interaction was found in the models predicting reported protein intake (p = 0.05) and reported iron intake (p = 0.05). In addition, a marginal intervention x sex interaction was found in the models predicting energy intake (p = 0.08) and total fat intake (p = 0.07). Accordingly, we reran the regression analyses by sex to further examine the possibility that intervention-related bias in reporting of intake may differ by sex (see tables 1 and 2 for p values). Results of these analyses (data not shown) indicated that among females, intervention condition was a significant predictor of reported energy, fat, and saturated fatty acid intakes. More specifically, independently of observed intake, reported energy intake among females in the intervention schools was 66.8 kcal lower than reported intake among females in the control schools (p = 0.03). With respect to total fat, independently of observed intake, reported fat intake among females in the intervention schools was 3.8 g lower than reported intake among females in the control schools (p = 0.03). Reported saturated fatty acid intake among females in the intervention schools was also significantly lower than reported intake among females in the control schools.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Average nutrient estimates from observed and recalled lunch intakes in boys, by treatment condition, Pathways Study, 1997–2000*
 

View this table:
[in this window]
[in a new window]
 
TABLE 2. Average nutrient estimates from observed and recalled lunch intakes in girls, by treatment condition, Pathways Study, 1997–2000*
 
Tables 1 and 2 present the differences in observed and recalled lunch intake estimates by treatment condition among boys and girls, respectively. The findings presented in these tables are consistent with the results of the linear regression analyses.


    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
These results suggest that intervention-related bias in reporting of dietary intake may be of concern among girls. More specifically, girls in intervention schools were found to systematically underreport energy, total fat, and saturated fatty acid intake relative to girls in the control schools. Among boys, no evidence of intervention-related bias in reporting of intake was found. The most likely explanation for the differential reporting of intake by intervention condition relates to social desirability bias in reporting, which may be greater among children in the intervention schools, where healthy eating and low-fat food alternatives were emphasized as part of the classroom curriculum. Indeed, the finding of underreporting by girls is consistent with the results of Hebert et al. (16), who found that adult women underreported total energy intake: A one-unit increase in social desirability score was associated with underestimation of 19.2 kcal/day in energy intake and 0.8 g/day in total fat intake. Hebert et al. have also reported that women generally rank higher than men in measures of social desirability. Moreover, conflict and guilt regarding food consumption occur more in girls than in boys (17).

Several limitations of this study should be noted. Because participants in this study were American Indian children from seven American Indian communities, strictly interpreted, these findings may be generalized only to similar children. Second, because participants provided a 24-hour dietary recall subsequent to being observed eating lunch, reporting may have been influenced by the observation, with participants providing more accurate reporting of intake because of increased attention to their diet.

In conclusion, these findings suggest that investigators should consider bias in reporting of dietary intake by intervention condition when conducting diet-focused intervention studies, in addition to the equally valid concerns about cognitive and social desirability biases in dietary reporting in epidemiologic studies. Intervention-related bias in reporting of dietary intake could potentially yield artifactual study results. In designing intervention studies, enhancement of measures that rely on self-reports with objective measures of dietary intake would help investigators to evaluate whether differential reporting by treatment group has occurred.


    NOTES
 
Correspondence to Dr. Lisa Harnack, Division of Epidemiology, School of Public Health, University of Minnesota, 1300 South 2nd Street, Suite 300, Minneapolis, MN 55454 (e-mail: harnack{at}epi.umn.edu). Back


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Himes J, Ring K, Gittelsohn J, et al. Impact of the Pathways intervention on dietary intakes of American Indian schoolchildren. Prev Med 2003;37:S55–61.[CrossRef][ISI][Medline]
  2. Caballero B, Clay T, Davis S, et al. Pathways: a school-based, randomized controlled trial for the prevention of obesity in American Indian schoolchildren. Am J Clin Nutr 2003;78:1030–8.[Abstract/Free Full Text]
  3. Byers T. On the hazards of seeing the world through intervention-colored glasses. Am J Clin Nutr 2003;78:904–5.[Free Full Text]
  4. Luepker RV, Perry CL, McKinlay SM, et al. Outcomes of a field trial to improve children’s dietary patterns and physical activity: The Child and Adolescent Trial for Cardiovascular Health. JAMA 1996;275:768–76.[Abstract]
  5. Stolley M, Fitzgibbon M. Effects of an obesity prevention program on the eating behavior of African American mothers and daughters. Health Educ Behav 1997;24:152–64.[ISI][Medline]
  6. Resnicow K, Yaroch A, Davis A, et al. GO GIRLS!: results from a pilot nutrition and physical activity program for low-income overweight African American adolescent females. Health Educ Behav 2000;27:633–48.
  7. Davis CE, Hunsberger S, Murray DM, et al. Design and statistical analysis for the Pathways Study. Am J Clin Nutr 1999;69(suppl):760–3S.
  8. Davis SM, Going SB, Helitzer DL, et al. Pathways: a culturally appropriate obesity-prevention program for American Indian schoolchildren. Am J Clin Nutr 1999;69(suppl):796S–802S.
  9. Gittelsohn J, Shankar AV, Pokhrel RP, et al. Accuracy of estimating food intake by observation. J Am Diet Assoc 1994;94:1273–7.[CrossRef][ISI][Medline]
  10. Simons-Morton B, Forthofer R, Huang I, et al. Reliability of direct observation of school children’s consumption of bag lunches. J Am Diet Assoc 1992;92:219–21.[ISI][Medline]
  11. Buzzard M. 24-hour dietary recall and food record methods. In: Willett W, ed. Nutritional epidemiology. 2nd ed. New York, NY: Oxford University Press, 1998:50–73.
  12. Hess R, Torney J. The development of political attitudes in children. Chicago, IL: Aldine Publishing Company, 1967.
  13. Mack K, Blair J, Presser S. Measuring and improving data quality in children’s reports of dietary intake. In: Warnecke R, ed. Proceedings of the 6th Conference on Health Survey Methods. Hyattsville, MD: National Center for Health Statistics, 1996:51–5. (DHHS publication no. (PHS) 96-1013).
  14. Feskanich D, Sielaff BH, Chong K, et al. Computerized collection and analysis of dietary intake information. Comput Methods Programs Biomed 1989;30:47–57.[CrossRef][ISI][Medline]
  15. Snyder P, Anliker J, Cunningham-Sabo L, et al. The Pathways Study: a model for lowering the fat in school meals. Am J Clin Nutr 1999;69(suppl):810S–15S.
  16. Hebert JR, Ma Y, Clemow L, et al. Gender differences in social desirability and social approval bias in dietary self-report. Am J Epidemiol 1997;146:1046–55.[Abstract]
  17. Wardle J, Beales S. Restraint, body image and food attitudes in children from 12 to 19 years. Appetite 1986;7:209–17.[ISI][Medline]




This Article
Abstract
FREE Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Disclaimer
Request Permissions
Google Scholar
Articles by Harnack, L.
Articles by Weber, J. L.
PubMed
PubMed Citation
Articles by Harnack, L.
Articles by Weber, J. L.