From the Beltsville Human Nutrition Research Center, Diet and Human Performance Laboratory, Agricultural Research Service, US Department of Agriculture, Beltsville, MD
Correspondence to Dr. David R. Paul, Beltsville Human Nutrition Research Center, Diet and Human Performance Laboratory, Agricultural Research Service, US Department of Agriculture, 307B Center Road, Beltsville, MD 20705 (e-mail: pauld{at}ba.ars.usda.gov).
Received for publication January 27, 2005. Accepted for publication May 25, 2005.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
bias (epidemiology); energy intake; energy metabolism; epidemiologic methods; nutrition assessment; questionnaires
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
There are three quantities to consider when corrections are generated for a food frequency questionnaire: 1) the food frequency questionnaire estimates, 2) the reference method, and 3) true (habitual) intake. Reference methods used in food frequency questionnaire validation studies include weighed-food records (8, 9
), diet records (10
13
), and 24-hour recalls (1
, 9
, 12
, 14
16
). Problems arise if the reference method is not close to true intake, for then one must understand how it differs from true intake to compensate for its deficiencies. Such deficiencies include correlated errors with the food frequency questionnaire (e.g., if there was underreporting of energy in both the food frequency questionnaire and the reference method), measurement bias (which may differ among subjects and food types), and if the relation between reference measurement error and true intake depended on the magnitude of intake (e.g., if measurement error was greater in those with higher intakes). With these limitations, the difficulty with using food and diet records and 24-hour recall techniques for validation studies is the measurement error that is well known (4
, 9
, 17
, 18
). Furthermore, maintaining diet records tends to interfere with the daily lives of subjects, altering dietary habits so that they no longer represent true habitual intake (19
).
Reference methods also include biomarkers, such as doubly labeled water and/or urinary nitrogen excretion, because they reduce the potential for correlated error and do not interfere with the measurement of food intake (1, 9
, 16
). However, a single 1- or 2-week mean value for energy or protein intake may not fairly represent the period of time covered by a food frequency questionnaire (several months to a year), since there is high variability in daily food intake (coefficient of variation of approximately 27 percent) (20
). Moreover, doubly labeled water and nitrogen excretion can provide estimates for only energy and protein intakes, respectively, and not for the other macronutrients or specific foods.
The underlying difficulty with calibrating a food frequency questionnaire is not knowing how well the reference technique reflects true intake for the same period of time measured during the food frequency questionnaire. Not only does this lack of knowledge affect the ability of investigators to gauge the amount of measurement error, it also makes it difficult to understand the effect of applying energy corrections to uncorrected food frequency questionnaire data. Our study is based on a 16-week period when the true intake was known for all subjects, thus allowing us to determine which method or combination of methods best adjusts food frequency questionnaire estimates toward their true values. We report on biases of the different adjustment methods and estimate the reduction in variance achieved by each. Our intent is not to provide specific coefficients to adjust food frequency questionnaire results but rather to demonstrate that food frequency questionnaire correction is possible using energy expenditure estimates and body weight.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Menus
Food items offered in the morning (breakfast and lunch) were presented in a cafeteria-style setting as three different rotating menus, each lasting 7 days (table 2). Some food items remained on all three menus (e.g., milk and orange juice). In the evening, breakfast and lunch items were also available. A typical dinner was presented cafeteria style as one- or two-entrée selections, with optional gravies or sauces and a minimum of three vegetables and side dishes. A garden salad with a variety of additional toppings and dressings was also available. Fifteen different dinner menus were rotated daily during the 16-week period (table 2).
|
Recording and tracking of food intake
After each subject selected his desired foods, he presented them to a staff member who recorded the identity and weight of each food item by hand and on a computer (combination of bar code recognition of the food item and hand entering of the weight). Upon termination of feeding, each subject presented his tray to a staff member who weighed any uneaten food. The accuracy of the food item recording process was verified by comparing the information on the computer with the hand-entered logs. This verification procedure was followed daily and repeated at the end of the study with all food records. Energy and macronutrient composition were determined by consultation with the US Department of Agriculture National Nutrient Database for Standard Reference (22).
Body weight and composition
Before breakfast and after voiding, body weight was determined weekly on an electronic balance to the nearest 0.01 kg. Body composition was measured by a QDR 4500A dual-energy x-ray absorptiometer (Hologic, Inc., Waltham, Massachusetts) during weeks 1, 8, and 16.
Estimated energy intake
In the absence of weight change over a defined period of time, energy intake and expenditure are assumed to be equal (23). Therefore, to obtain a surrogate estimate of energy intake, we used doubly labeled water to estimate energy expenditure. The doubly labeled water technique (24
) was used to provide an estimate of energy expenditure for 7 days, twice during the study (weeks 8 and 16). Subjects reported to the BHNRC between 6:30 and 9:00 a.m., at which time they received an oral dose of
(0.16 g/kg of body weight) and 2H2O (0.30 g/kg of body weight). Urine samples were collected immediately before the dose and on every morning (second void) for the last 7 days of the treatment period. The first sample was collected approximately 24 hours after the dose. Enrichments of 2H and 18O in urine samples were measured by infrared spectroscopy and isotope ratio mass spectrometry, respectively. The total energy expenditure and, in turn, energy intake were calculated using the equations of Weir (25
).
Physical activity
Physical activity was estimated using a daily recording log method, modified from the method of Bouchard et al. (26). Briefly, subjects recorded their daily activities in a log every 15 minutes for 7 consecutive days, the same days as the doubly labeled water dosing period. Activities were recorded as a value from one to nine, corresponding to example activities listed in the log. Each activity assumed a predetermined energy expenditure score. Thus, energy expenditure was calculated as the time spent in that activity times the energy expenditure rate.
Food frequency questionnaire
The Block '98 Food Frequency Questionnaire (Berkeley Nutrition Services, Berkeley, California) was used to assess usual dietary intake during the study period. The eight-page scannable booklet asks for frequency (times per day, week, month) of consumption of 109 food items. Portion sizes were estimated with the aid of a "serving size choices" page containing photos of simple abstract three-dimensional models representing four different amounts of food. Gram amounts were then assigned on the basis of the gram weight of the volume of the chosen model, for that particular food.
During the final week of the study, subjects received instructions on how to fill out the food frequency questionnaire, which was then completed at home. Although the questionnaire was designed to capture usual intake during the past year, subjects were instead instructed to answer questions based on what they ate and drank during the study. Returned questionnaires were reviewed for omissions, implausible responses, and other errors. When necessary, forms were reviewed with the subject for clarity of answers. The completed food frequency questionnaires were sent to Berkeley Nutrition Services (now known as NutritionQuest, Berkeley, California) for coding and analysis. Two of the questionnaires were "flagged" because the subjects reported consuming more than 17 food items per day (17.8 and 18.1 items). However, these values agreed with the actually measured number of food items. No other errors were detected.
This study was part of a broader investigation of the effects of covert manipulation on energy intake. To manipulate macronutrient composition, subjects consumed two of three drinks that provided a total of approximately 2.1 MJ/day of carbohydrate, protein/carbohydrate, and/or fat. Thus, solely for the purposes of comparing the food frequency questionnaire and actual intake with doubly labeled water and physical activity in terms of energy and macronutrient intake, 2.1 MJ were subtracted from doubly labeled water and physical activity (table 3). Subjects were unaware of the composition of the drinks and were instructed to not include them when filling out the food frequency questionnaire.
|
Energy correction
Since it is well accepted that data derived from food frequency questionnaires are subject to measurement error and should be adjusted for energy intake (21) and differences in body size or physical activity (7
), we based our adjustments on these variables. We applied two types of energy corrections: 1) group corrections, where a single adjustment factor based on the average of the subjects was used to correct individual subjects' food frequency questionnaires, and 2) individual corrections, where a subject-specific adjustment factor was developed and applied to each subject. Correction factors were created using energy intake estimated by either doubly labeled water or physical activity as the numerator and food frequency questionnaire intake as the denominator.
Our food frequency questionnaire measurement error analysis is based on the assumptions that 1) measurement error of actual food intake is negligible and 2) errors in the measurement of actual food intake and the food frequency questionnaire are unrelated. Potential errors in the first assumption are related to the misidentification of food items in the recording process, subjects' failing to report foods not directly measured in the Human Studies Facility, and database errors. Errors caused by misidentifying food items are assumed to be quite small, because any lack of agreement between the daily intakes recorded by hand and the bar-coding system was reconciled daily and again at the end of the study. Errors due to subjects' failing to consume all foods were minimized by direct observation of food intake in 10 of 21 weekly meals. The risk of violating the second assumption was minimized since actual food intake was not reported by the subjects (it was measured by the investigators), and there is little error in the actual food intake measure (assumption 1).
Bias analysis
We calculated the means and standard deviations for each of the three macronutrients and energy intake for actual intake, food frequency questionnaire, and the various food frequency questionnaire adjustments. We tested the reduction in bias (determining if there was a statistical difference with actual measurements) using paired t tests (pairing within subjects).
Variance analysis
To measure how much adjustments to the food frequency questionnaire reduce measurement error in macronutrient intake, we looked at reductions in the variance of the difference between actual measurements and food frequency questionnaire intakes averaged over the 12 subjects. Variances were calculated for the difference between actual measurements and unadjusted food frequency questionnaire and for group and individual corrections of the food frequency questionnaire by use of doubly labeled water and the combination of doubly labeled water and body weight. Variances were calculated as (actual FFQcorr)2/12, where FFQcorr = energy- and possibly body weight-corrected food frequency questionnaire macronutrient intake. The same procedure was carried out for the variance calculations based on physical activity.
Body weight adjustment
In a preliminary analysis, we found that the size of the difference between actual intake and energy-corrected food frequency questionnaire fat intake was correlated with body weight (r = 0.67, p < 0.02). Thus, the energy-corrected food frequency questionnaire estimate of fat intake could be further improved. This additional adjustment for fat intake requires estimation of two regression coefficients and also makes use of the actual fat intake measurement, which is the target of our adjustments to the food frequency questionnaire and is ordinarily not available. Still, one can conceive of a smaller, preliminary study using a sample of the population of interest to estimate these coefficients, which can then be applied to data obtained on the larger sample. An alternative is to use results from another study with a similar population from which these coefficients could be calculated. In our study, we regressed body weight on the difference between actual fat intake and the energy-corrected food frequency questionnaire fat intake, where doubly labeled water or physical activity was used for the energy correction.
Statistical tests and analysis
Comparisons between mean macronutrient intake for actual intake and food frequency questionnaire (tests of bias) were made using paired t tests. Differences in (ratios of) variances were tested using F statistics. Changes in body weight and composition were measured in a mixed-linear models framework, using the Proc Mixed procedure in SAS, version 9.1, software (SAS Institute, Inc., Cary, North Carolina). Linear relations between variables were calculated using Pearson's product-moment correlation. Attenuation factors were calculated using equation 4 in the paper by Kipnis et al. (3).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Energy intake
Actual energy intake was 10.5 (standard deviation (SD): 1.3) MJ/day, compared with 8.3 (SD: 2.9) MJ/day as measured by food frequency questionnaire, 10.5 (SD: 1.6) MJ/day for doubly labeled water, and 11.8 (SD: 2.0) MJ/day for physical activity. Actual energy intake and that predicted by doubly labeled water were significantly correlated (r = 0.75, p < 0.005), but the correlations between food frequency questionnaire and actual intake (r = 0.51, p > 0.09) and between food frequency questionnaire and doubly labeled water (r = 0.48, p > 0.10) were not significant. Energy intake, as measured by physical activity, was not correlated with actual intake (r = 0.17, p > 0.60), doubly labeled water (r = 0.37, p > 0.20), or food frequency questionnaire (r = 0.20, p > 0.50).
General bias and variance results
Absolute macronutrient intakes (g/day) predicted by food frequency questionnaire were significantly lower than actual intakes for carbohydrate and protein, but not for fat (table 3). Relative macronutrient intakes as a percentage of total energy were significantly different for carbohydrate and fat, but not for protein.
Group and individual energy adjustments to the food frequency questionnaire were equivalent corrections to bias. Including the body weight adjustment to fat intake reduced the fat bias to zero (as it must, since we used the actual fat intake measure to estimate the regression coefficients). While there was little overall bias in the unadjusted food frequency questionnaire measure of fat, variances were large.
The linear relations between food frequency questionnaire macronutrient and daily energy intakes were strong and significant, while the relations observed for actual intake were not as strong (figure 1). Figure 1A shows that the true increase in energy is due primarily to increases in carbohydrate intake. Unfortunately, this relation is not preserved using the food frequency questionnaire (figure 1B). Subjects with higher energy intake tended to overreport their intakes of protein and fat, relative to subjects with lower energy intake.
|
Applying energy correction factors tended to reduce variances, particularly for individual corrections (table 4). Using doubly labeled water, we found that there was a reduction for all macronutrient error variances. By use of physical activity, there was a substantial reduction in error variance for carbohydrate intake (overall, the correction removed about one third of the error variance).
|
Overall, corrections based on the combination of individual body weight and doubly labeled water/physical activity were the best combination for reducing variances, removing more than two thirds of the error for the physical activity correction (1 1.67/5.89) and even more for the doubly labeled water correction.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Considering the extensive documentation of measurement error in the food frequency questionnaire, the fact that intakes of energy and macronutrients were not accurate is not surprising. Proponents of the food frequency questionnaire have described methods for correcting for measurement error since the early 1990s (5) and have criticized authors who do not perform energy corrections on raw food frequency questionnaire data (21
, 28
). An interesting result of this study is that, not only are absolute and relative intakes not accurate, applying energy adjustment to correct the data, while helpful, is not entirely effective. In the case of fat intake, applying the energy adjustment actually increased error. The reason for the problem using energy corrections for our subjects can be observed using figure 1. Figure 1B indicates that the relations between food frequency questionnaire macronutrient and energy intakes in our subjects were linear and strong (r = 0.87 0.94). The strength of these relations forms the basis for using energy correction. However, figure 1A demonstrates that the relations between true macronutrient and energy intakes are strong for carbohydrate only. Thus, applying a single adjustment factor to all macronutrients does not correct all the macronutrient measurement error.
Given our homogeneous study population relative to the multitude of factors that may affect the misreporting of food intake (gender, age, physical activity, education, and body composition), there was surprisingly large subject-to-subject variation in measurement error, so one would expect even greater intersubject variation in more heterogeneous populations. The lack of effectiveness for the group versus individual corrections illustrates this. Since our subjects were relatively homogeneous, the specific results of this study (e.g., parameter estimates) are not universally applicable. Nevertheless, the discrepancies between actual intake and the food frequency questionnaire seen in this study and the adjustment methods we used to bring the food frequency questionnaire into better agreement with the actual intake should be applicable to other populations. In particular, individual energy adjustments using physical activity and a body weight correction for fat intake removed more than two thirds of the error in the food frequency questionnaire.
Another issue to be considered is that food frequency questionnaires are not intended to be used "in studies with small numbers of subjects; for surveillance and monitoring of current levels where accurate absolute intakes are required; ... and in some clinical work when precise intakes are required" (19, p. 568). However, if food frequency questionnaires do not produce accurate individual food intake estimates and introducing energy correction does not correct the errors, the usefulness of food frequency questionnaires to quantitatively measure the relation between disease and food intake in any size study is questionable. One other study utilized a similar approach to ours (comparison of a food frequency questionnaire with controlled feeding in a relatively small sample of 19 persons) (27
), and both studies indicate that the food frequency questionnaire was not a reliable indicator of true intake. The agreement between the statistics we calculated on our subjects and those given in Kipnis et al. (3
) and Subar et al. (1
) provides additional evidence that our conclusions are not unduly influenced by sample size.
Energy was underreported by about 21 percent (table 3), similar to 20 percent in 19 subjects consuming the "usual American diet" (27) and less than 31 percent in 261 male subjects (1
, 3
). The overall bias for protein intake in our study and that of Schaefer et al. (27
) was small compared with that reported by Subar et al. (1
). Fat and carbohydrate intakes were underreported by 30 percent and 10 percent, respectively, in the study by Schaefer et al. (27
) compared with 2 percent and 30 percent in our study. We found an attenuation factor of 0.192 for energy, between the 0.080 and 0.230 reported by Kipnis et al. (3
) (calculated using different models); and for protein, our value was 0.212, slightly greater than the 0.156 and 0.177 they calculated. We calculated the variance of true intake as 0.025 (kcal scale), compared with 0.026 and 0.044 for their two models (3
). It is likely that the differences between these studies are related to the composition of the subject pools (such as age and body composition) and/or the methods used to obtain the criterion measure (type of monitored feeding or doubly labeled water).
In conclusion, applying energy adjustments to food frequency questionnaire data can significantly reduce overall measurement error, but this has the potential to increase errors for some macronutrients. Although individual doubly labeled water energy expenditure adjustments are clearly effective to correct overall energy intake, doubly labeled water is not practical in large epidemiologic studies. Assuming that good estimates of the regression coefficients that relate body weight to the error in fat intake are available, an additional questionnaire (similar to physical activity) given during the administration of food frequency questionnaires and body weight measurements would provide sufficient information to markedly reduce error in individual food frequency questionnaire estimates.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|