1 National Cancer Institute, Bethesda, MD.
2 University of Connecticut Medical School, Farmington, CT.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
diet; epidemiologic methods; food habits; nutrition assessment; nutrition surveys; questionnaires
Abbreviations: AARP-FFQ, American Association of Retired Persons food frequency questionnaire; CSFII, Continuing Survey of Food Intakes by Individuals; FFQ, food frequency questionnaire; NCI, National Cancer Institute.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The Block and Willett FFQs, or variations of either, are two examples of FFQs that are widely used in nutritional epidemiology research. Block et al. (1) pioneered a data-driven approach, which used food consumption data to develop an FFQ and its associated analytical software. Dietary intake data from a large, nationally representative sample (19761980 National Health and Nutrition Survey II) were used to decide which foods to include in the FFQ and to assign nutrient composition and portion sizes. To create a food list for an FFQ, Willett et al. (2
) used regression methods (using food record data) and judgment to prepare an extensive list of commonly consumed foods containing nutrients pertinent to the prevention of cancer and heart disease. In the Willett FFQ, portion size is not specifically asked of respondents, but within the frequency question, respondents are asked how often a particular standard portion size is consumed. Judgment is used to establish both the size of these standard portion sizes (in common household units) and the nutrient content of the FFQ line items (using current food composition databases) (3
). Various other methods of assigning nutrient composition values to line items on an FFQ are documented in the literature (4
6
). However, there has been no evaluation of the relative performance of the methods.
Several scientific bodies have made strong recommendations for improving dietary assessment methods (7, 8
). In this paper, we investigate whether one aspect of the overall validity of an FFQ, the assignment of nutrient values, is sensitive to different assignment methods. The research was motivated by a need to provide a nutrient database for new FFQs developed and used in research at the National Cancer Institute (NCI). Specifically, we calculated nutrient values for food items queried on the FFQ used in the National Institutes of Health American Association of Retired Persons Diet and Health Study. However, the results are relevant to FFQs in general.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Eighty percent of the eligible respondents participated in the first interview, and 95 percent of those who responded to the first interview participated in the second interview, resulting in an overall response rate of 76 percent for both days of data collection. This analysis includes 10,019 adult respondents aged 19 years or older who completed either 1 or 2 days of dietary recall.
FFQ line items
We categorized the 5,261 individual food codes found in the CSFII database and consumed by these adults into 182 food groups similar in usage and nutrient content. For example, a ready-to-eat cereal group was created from the 111 individual food codes for all types of ready-to-eat cold breakfast cereals reported. Analyses of important food sources of nutrients, similar to those reported by others (1014
), were conducted to assess which of the 182 food groups were important food sources of energy, fat, percentage of energy from fat, vitamin C, beta-carotene, dietary fiber, vitamin A, calcium, and vitamin E. The 124 groups that contributed 90 percent or more of the total intake for each of these nutrients/food constituents were selected to create the final food list for the AARP-FFQ. The other 58 food groups were excluded because they contributed little to nutrient intake in the United States, usually because of infrequent consumption. For some food groups, subgroups were created to better assess varying nutrient contents within the broader food group, especially for fat and fiber. For example, the ready-to-eat cereals food group consisted of four subgroups to differentiate the cereals more clearly with respect to fiber and other nutrients (highly fortified, very high fiber, moderate fiber, and other). The FFQ first queries frequency of total cereal intake and usual portion size and then presents the embedded questions regarding the proportion of the time each of the four different varieties is consumed. Our decision to create subgroups for a large food group, as opposed to creating separate food groups, was based on cognitive research indicating that respondents had difficulty when asked to complete the frequency of each of many related items (15
). Including subgroups, nutrient content estimation was needed for 170 food groups.
FFQ portion size
The AARP-FFQ developed at the NCI retains questions about portion size. In specifying the portion size options for FFQ line items, we adopted line item-specific ranges, such as "less than 1 cup," "12 cups," and "greater than 2 cups," rather than "small," "medium," and "large." We did this because results from cognitive testing for the new FFQ (15) suggested that study participants were more able to answer questions about portion size when quantified range options were provided.
To establish portion size ranges using CSFII, we first looked at nationally representative data from a 68-item NCI Block FFQ administered in the 1992 National Health Interview Survey (16). The data showed that, across all food items, portion size was answered as "medium" about 66 percent of the time. We therefore decided that the range for the middle portion size should be broader than the middle third of the CSFII portion size distribution to better represent the tendency for individuals to select a middle portion size on FFQs. We selected the approximate 25th and 75th percentiles of gram weight portion sizes for each food group as cutpoints to define our three portion size ranges. This created a broad medium portion size, but left enough CSFII respondents in the small and large portion size groups to provide stable estimates of the amounts consumed.
Because portion size and types of foods consumed varies by age and gender, we separated respondents into three age groups (1930, 3150, and >50 years) by gender to assess age-specific portion sizes. We chose these age groups because they are similar to those upon which the Recommended Dietary Allowance energy requirements are based (1924, 2550, and >50 years) (17), yet allow for adequate numbers of CSFII respondents for analyses in the youngest age group.
Deriving nutrient estimates: alternative approaches
In general, the average daily intake of a given nutrient for an FFQ respondent is derived as follows: For each line item, the reported daily frequency is multiplied by a nutrient value specific to the respondents age, gender, and reported portion size. The calculated line item-specific values are then summed across all the line items, yielding a total nutrient intake for that respondent.
The source of methodological variability we sought to investigate was the derivation of the line item-specific nutrient values. Because the general method of calculating nutrient intake (outlined above) requires a nutrient value (age-, gender-, and portion size-specific) for each line item, the challenge is to assign a single nutrient value, given that respondents eat a variety of different foods subsumed by that line item. For example, people eat a variety of cheeses, each with its own dietary fat content. What fat value should one assign for the line item "cheese"? We used the CSFII, which provides detailed, nationally representative food consumption data regarding the intake of many varieties of food (cheese types, for example) to evaluate alternative methods of assigning nutrient values for a single line item. In all analyses, we used 24-hour recall data from CSFII; no FFQ nutrient data were analyzed.
We first categorized food consumptions of all the CSFII respondents as reported on the 24-hour recalls into 3 x 2 x 3 = 18 age-, gender-, and portion size-specific cells for each of the 170 food groups being evaluated. We computed means and medians for each of these 18 categories for each of seven nutrients or dietary constituents representing a variety of macro- and micronutrients (energy, fat, carbo-hydrate, fiber, vitamin A, vitamin C, and iron). We investigated a series of methodological variations. First, to investigate the influence of small numbers, we examined the effect of collapsing the data over adjacent age groups within gender and portion size when there were fewer than 10 individuals in one of the 18 cells. Second, to investigate the importance of age in determining nutrient intake, we combined the three age groups by gender and portion size before computing means and medians, thereby reducing the number of cell sizes from 18 to six. Third, we examined a simple gender-specific regression approachnutrient = age effect + portion size effectand then repeated the regression excluding outliers to address the problem of dietary data being skewed by high intakes. Fourth, we computed estimates by using the method developed by Block et al. (1), in which a single median nutrient density (weighted by frequency of consumption of individual foods within a food group) is multiplied by a median age- and gender-specific gram weight portion size.
We thus compared 10 nutrient estimation methods, by gender, characterized as follows: 1) mean nutrient by portion size and age; 2) mean nutrient by portion size and age, with collapsing; 3) mean regression of nutrient on portion size and age; 4) mean nutrient by portion size; 5) mean regression of nutrient on portion size and age, excluding outliers; 6) median nutrient by portion size and age; 7) median nutrient by portion size and age, with collapsing; 8) median regression of nutrient on portion size and age; 9) median nutrient by portion size; 10) median nutrient density (weighted by frequency of consumption of individual foods within a food group) x median age- and gender-specific portion size (Block method).
For all regressions, the model was: nutrient intake = ß0 + ß1 Age2 + ß2 Age3 + ß3 Size2 + ß4 Size3 + , where Agei is an indicator for age group i and Sizei is an indicator for portion-size group i. A regression model was fit separately for each gender group. In model 5, outliers from the model 3 regression were defined as observations with squared errors greater than three times the mean squared error.
We then sought to compare the 10 methods described above to determine which performed best. For each of 170 food groups corresponding to FFQ line items, we now had 10 different nutrient values from our estimation methods based on the grouped CSFII data analyses (described above) for each of the seven nutrients. We compared these nutrient values with those for each specific food item reported on 24-hour recalls (among our 170 food groups) for all CSFII respondents. Specifically, we calculated the difference between each nutrient value for each food reported on an individual's 24-hour recall (observed value) and the nutrient value for each of our 10 methods (estimated values). We then created an error term by summing these differences by nutrient for all foods reported in each individual's 24-hour dietary recall to evaluate error in terms of total daily nutrient intake.
We evaluated our 10 methods on the basis of three measures of error estimation: mean error, mean squared error, and mean absolute error. Mean error measures the magnitude and direction of the possible bias of the estimate, while mean squared error and mean absolute error measure the precision of the estimate (bias + variation). Estimators that minimize mean absolute error are sometimes preferred over those that minimize mean squared error because the latter can be sensitive to outliers.
To compare methods, we examined the associated errors of each one. Generally, when the same data are used both to fit a model and to estimate errors, the usual estimated errors, or residuals, tend to be biased toward zero. This bias depends on the complexity of the model and the method of fitting it. Therefore, we used cross-validation to obtain unbiased error estimates that could be compared across methods. This applied cross-validation method leaves one subject out of the data, predicts that subject's nutrient intake based on the remaining n - 1 subjects, and calculates error for that subject by subtracting the actual value from the predicted value. This was done for each subject one at a time.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The findings show that using either the mean or the median nutrient intakes of all reports within a given portion size for a given food group is an improvement over the current Block approach. Because there is little or no documentation on how nutrient databases are constructed for other FFQs, we are unable to evaluate how our newer methods might compare with them. Further, the findings indicate clearly that data-driven methods that use mean versus median approaches are superior with respect to mean bias, absolute error, and mean squared error of total daily nutrient intake.
An unexpected finding was that age was not a critical variable in nutrient estimation. Although nutrient intake and food choices are known to vary by age and gender, these data show that, once portion sizes are defined by the 25th and 75th percentiles of gram weight intakes for all adult men and women, age group has no appreciable impact on nutrient estimation. This suggests that investigators could simplify approaches to FFQ nutrient database development by excluding age as a factor. Further, grouping individuals who consume a food or foods into gender- and portion size-specific versus age-, gender-, and portion size-specific categories leads to cell sizes that are more likely to provide stable nutrient estimates.
An obvious question is, "Which is the optimal data-driven method for creating a nutrient database for an FFQ?" These data suggest that mean methods are best, but among the mean methods, none is clearly superior. The mean regression method excluding outliers was best overall in terms of mean absolute error, but performed less well with respect to mean error (bias) and mean squared error. Therefore, it is difficult to pick any single mean method over another because it is unclear whether it is better to have many estimates off by a little or a few estimates off by a lot when relating nutrients to disease outcomes. However, differences between any of the mean methods were small at best, and all performed quite well. This being the case, it makes sense to consider which method is the simplest and easiest to use, and that, we conclude, is the portion size x gender method.
Many FFQs do not query portion size. The data from this research suggest that in developing a nutrient database for such an FFQ, a mean rather than a median method should be used (excluding portion size). Further research is necessary, however, to clarify whether or not age is a more important factor when portion size is not considered, since age may be a proxy for portion size.
FFQs, like other dietary assessment instruments, continue to be based on self-report. Investigators using these instruments are well aware of the errors associated with them, such as under- and overreporting, misreporting, missing data, and so forth. The question is whether FFQ nutrient estimates can be improved even with the inevitable measurement error in reporting. Investigators frequently try to improve FFQs through changes in wording, formatting, ordering, and other cognitive aspects (15, 18
21
). This research provides data to show that the methods used to create a nutrient database for an FFQ may offer another means of improving FFQs. The reduction in measurement error accompanying each such improvement will result in an improvement in our ability to measure diet and disease associations.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|