1 Department of Nutrition, Harvard School of Public Health, Harvard University, Boston, MA.
2 Salud Coronaria, Instituto de Investigaciones en Salud, Universidad de Costa Rica, San Pedro, Costa Rica.
3 Department of Epidemiology, Harvard School of Public Health, Harvard University, Boston, MA.
4 Department of Biostatistics, Harvard School of Public Health, Harvard University, Boston, MA.
ABSTRACT
Little is documented about the performance of the food frequency questionnaire (FFQ) in US minority groups and in populations in developing countries. The authors applied a novel technique, the method of triads, to assess the validity and reproducibility of the FFQ among Hispanics. The subjects were men (n = 78) and women (n = 42) living in Costa Rica. Seven 24-hour dietary recalls and two FFQ interviews (12 months apart) were conducted between 1995 and 1998 to estimate dietary intake during the past year. Plasma and adipose tissue samples were collected from all subjects. Validity coefficients, which measure the correlation between observed and "true" dietary intake, were also estimated. The median validity coefficients for tocopherols and carotenoids estimated by dietary recall, the average of the two FFQs, and plasma were 0.71, 0.60, and 0.52, respectively. Compared with adipose tissue, plasma was a superior biomarker for carotenoids and tocopherols. Adipose tissue was a poor biomarker for saturated and monounsaturated fatty acids but performed well for polyunsaturated fatty acids (validity coefficients, 0.451.01) and lycopene (validity coefficient, 0.51). This study also showed that biomarkers did not perform better than the FFQ and that they should be used to complement the FFQ rather than substitute for it.
biological markers; diet; epidemiologic methods; Hispanic Americans; methods; questionnaires; recall; reproducibility of results
Abbreviations: FFQ, food frequency questionnaire; USDA, US Department of Agriculture; VC, validity coefficient
High intake of certain nutrients and total energy is a recognized risk factor for conditions such as diabetes, stroke, and cardiovascular disease (1, 2
), whereas intake of others (e.g., tocopherols, carotenoids, folic acid, vitamin C, and dietary fiber) is thought to be protective (3
5
). Assessment of long-term dietary intake, the essential exposure factor for disease, is complicated by a lack of accurate methods (6
, 7
). Traditionally, intake of micro- and macronutrients has been estimated by the use of semiquantitative food frequency questionnaires (FFQs), dietary records, and dietary recalls (7
, 8
). Use of biomarkers of dietary intake is a recent addition to these techniques (3
, 9
, 10
). Assessment of true long-term dietary intake is fraught with a number of problems (11
13
), and biomarkers may not be better than traditional methods because they can be affected by factors other than diet and are not available for all nutrients (10
, 14
). Regardless of the technique used, actual food intake and its reporting are affected by factors such as culture, education, type of food consumed, age, gender, and body weight (11
, 12
, 15
17
). Furthermore, menus and recipes tend to vary across sociocultural groups.
Although a questionnaire is an excellent tool for assessing long-term dietary intake, an FFQ valid for one population may be invalid when applied to another. For this reason, a validation study is necessary whenever an FFQ is used. The performance of FFQs in describing food consumption patterns (18) and intake of individual nutrients has been evaluated extensively in European and US Caucasian populations (8
, 14
, 19
). Little is known about the performance of the FFQ in other ethnic groups, particularly in developing countries and in US minority populations. In addition, FFQs have been validated mainly against one dietary assessment method such as dietary recalls or dietary records. The problem attending this approach is that the same factors that affect the reference method may also affect the FFQ. This problem would make it impossible to assume independent random errors in the two methods, which could lead to overestimation of the correlation between the reference method and the FFQ (20
).
Even if this error did not exist, a bias leading to underestimation of this correlation could occur as a result of correlated random errors in repeated measurements when the reference method is used (14, 20
). Furthermore, in most FFQ evaluation studies, additional information on biomarkers of intake is usually not available to augment data from the reference method. When such information is available, Kaaks (6
) has recommended that the correlation of the estimate by using the dietary assessment method and a person's "true" long-term intake (referred to as the validity coefficient (VC)) be estimated from three pairwise correlations between the FFQ, the reference method, and the biomarker (figure 1). The technique for this estimation, named the "method of triads," is an application of a factor analysis model and does correct for bias due to correlated errors in the repeated measurements from the reference method (6
, 20
). Briefly, if Q, R, and M are the measurements from the FFQ, the reference method, and the biomarker, respectively, VCs can be estimated as follows (6
, 14
, 20
):
![]() |
![]() |
|
MATERIALS AND METHODS
Study population and design
The subjects in this 19951998 validation study were recruited from the control group of an ongoing case-control study of myocardial infarction in Costa Rica. The 78 men and 42 women in this study are of Mestizo background and culturally are Hispanic Americans. All subjects had no history of myocardial infarction or physical or mental disabilities prior to enrolling in the study. The study was approved by the Use of Human Subjects in Research committees at the Harvard School of Public Health (Boston, Massachusetts) and the University of Costa Rica (San Pedro, Costa Rica). Subjects randomly selected from the control group were asked to participate in the FFQ validation study. All subjects were visited at their homes by a dietitian and were interviewed about their dietary intake during the past year (FFQ1). In addition, seven 24-hour dietary recall assessments were conducted at each participant's home and served as a reference dietary assessment method. To account for seasonal variations in nutrient intake, the dietary recall assessments for each subject were distributed across 7 months of the year. For each subject, the months and days on which dietary recall assessments were conducted were chosen at random. All 7 days of the week were represented, except for a few subjects (n = 9) who could not be contacted on certain days. All dietary recall assessments were completed within 1 year of the FFQ1 interview.
To assess the reproducibility of FFQ1, we conducted a second interview (FFQ2) among all validation study subjects about a year later. The same 135-item FFQ (modified from that of Willett et al. (19)) was used for both interviews. Each subject provided a plasma sample (for assessment of tocopherol and carotenoid intake) and an adipose tissue biopsy (for assessment of tocopherol, carotenoid, and fatty acid intake).
Intake of energy and nutrients was computed by multiplying the consumption frequency of each food by the nutrient content of the specific portion; food composition values from the US Department of Agriculture (USDA) database (19, 21
) and data from manufacturers and published reports were used. Carotenoid intake (including
-carotene, ß-carotene, ß-cryptoxanthin, lycopene, and zeaxanthin + lutein) was calculated by using the new carotenoid database that includes more than 2,400 fruits, vegetables, and selected multicomponent foods (22
, 23
). The carotenoid content of tomato-based products was updated with values from the USDA, which were derived recently by reverse-phase high-performance liquid chromatography (24
).
"Food Processor" nutrient analysis software (ESHA Research, Salem, Oregon), which is based on USDA food composition data, was used to compute nutrient intakes from dietary recalls. This database was enhanced with nutrient composition data for foods and recipes specific to Costa Rica.
Sample collection and biochemical analyses
All biologic specimens were collected at the subjects' homes on the morning after an overnight fast. A subcutaneous adipose tissue biopsy was collected from the upper buttock with a 16-gauge needle and disposable syringe, as described previously (25). Blood samples were collected into tubes containing 0.1 percent ethylenediaminetetraacetic acid (EDTA). Both samples were stored in a cooler with ice packs at 4°C and were transported to the fieldwork station within 4 hours. Blood was then centrifuged at 2,500 rpm for 20 minutes at 4°C to obtain plasma. Plasma and adipose tissue samples were stored at -80°C and, within 6 months of collection, were transported over dry ice to the Harvard School of Public Health for analysis.
Concentrations of - and
-tocopherol were measured with a dual wavelength Hitachi high-performance liquid chromatography system and data station (26
). The L-4200 detector was set at a wavelength of 300 nm, and injections were performed by a programmable AS-4000 Auto-Sampler. One large sample of adipose tissue was maintained as a stock for quality assurance in subsequent runs. A portion of the same weight (2060 mg) as the study sample was taken from the core of the quality assurance sample and was included in each run to adjust for between-run variation. Every run also included a pooled plasma sample. The coefficients of variation for plasma
- and
-tocopherol were 9.4 and 10.1 percent, respectively. The respective coefficients of variation were 20.4 and 17.8 percent for adipose tissue
- and
-tocopherol. The minimum detection limits in a 30-mg adipose tissue sample were 1.189 and 0.718 µg/g for
- and
-tocopherol, respectively, while they ranged from 0.023 to 0.035 µg/g for the carotenoids. Samples (n
4) below the detection limits were set to missing. The same procedure was used to measure concentrations of
-carotene, ß-carotene, ß-cryptoxanthin, lycopene, and zeaxanthin + lutein in adipose tissue and plasma samples, except that the L-4200 detector was set at a wavelength of 445 nm. Lutein and zeaxanthin were grouped because they coelute on the chromatogram. The between-run coefficients of variation for
-carotene, ß-carotene, ß-cryptoxanthin, lycopene, and zeaxanthin + lutein were 7.2, 7.5, 12.6, 8.1, and 8.6 percent, respectively, in plasma samples and 18.6, 21.8, 26.4, 16.7, and 19.6 percent in adipose tissue samples.
The proportion of total fat in adipose tissue contributed by each fatty acid was determined by gas-liquid chromatography (27). In brief, about 2 mg of adipose tissue was added to a hexane:isopropanol (3:2) mixture. To 200 µl of this mixture was added methanol and acetyl chloride, which formed methyl esters. After esterification, the adipose tissue was evaporated and the fatty acid methyl esters were redissolved in iso-octane. The fatty acid concentrations were determined according to the following specifications: fused-silica capillary cis/trans column, SP2560, 100 m x 250 µm internal diameter x 0.20 µm film (Supelco, Belefonte, Pennsylvania); splitless injection port at 240°C; hydrogen carrier gas at 1.3 ml/minute, constant flow; Hewlett-Packard (now Agilent, Palo Alto, California) Model GC 6890 flame ionization detector (FID) gas chromatograph with 7673 autosampler injector; 1 µl of sample injected; and a temperature program of 90170°C at 10°/minute, 170°C for 5 minutes, 170175°C at 5°/minute, 175185°C at 2°/minute, 185190°C at 1°/minute, 190210°C at 5°/minute, 210°C for 5 minutes, 210250°C at 5°/minute, and 250°C for 10 minutes. We used known standards (NuCheck Prep, Elysium, Minnesota) and Agilent Technologies ChemStation A.08.03 software to identify peak retention times and the relative quantity of each fatty acid. For the 16 blind duplicates included with the test samples, the coefficients of variation were 24.9 percent for myristic, 5.4 percent for palmitic, 14.0 percent for palmitoleic, 16.0 percent for stearic, 3.0 percent for oleic, 5.5 percent for linoleic, and 8.5 percent for linolenic acids.
Plasma triacylglycerol, cholesterol, and high density lipoprotein cholesterol concentrations were measured with enzymatic reagents (Boehringer-Mannheim, Indianapolis, Indiana) and a Cobas Mira Plus autoanalyzer (Roche Diagnostics, Somerville, New Jersey). Cholesterol measurements were standardized to guidelines from the Centers for Disease Control (Atlanta, Georgia) and the National Heart, Lung, and Blood Institute (Bethesda, Maryland) (28, 29
).
Statistical analysis
SAS software (SAS Institute, Inc., Cary, North Carolina) was used for all statistical analyses. The mean of seven dietary recalls was computed and was compared with intakes estimated from FFQs, plasma, and adipose tissue. We also determined the mean of the two FFQ (average FFQ) assessments in an attempt to obtain a better estimate of long-term dietary intake (30), and we compared it with the mean of dietary recalls. The significance of the differences between the mean of dietary recalls and the average from the two FFQs was assessed by using Wilcoxon's signed rank test (31
). Descriptive statistics for the validation study population and for the overall control population were computed.
Because there were high correlations between total energy intake and individual nutrient intakes estimated by dietary recalls and the FFQs, variables were adjusted for total energy intake by regressing the nutrient against total energy intake, as described previously (7, 30
). Briefly, a nutrient was loge or square-root transformed to improve normality. The transformed nutrient was then regressed on total energy intake obtained from the same dietary assessment method. The regression coefficient, the intercept, and residuals were obtained. Although by definition the residuals are independent of the explanatory variable, some are negative and difficult to interpret. Therefore, we multiplied the regression coefficient by the mean of energy intake for the whole study population; to this product we added the intercept and the residual (7
). The result was back-transformed to obtain energy-adjusted nutrient intake for each subject. Plasma tocopherol and carotenoid concentrations correlated highly with plasma triacylglycerol and cholesterol, respectively. Accordingly, plasma tocopherol and carotenoid measurements were adjusted for triacylglycerol and cholesterol concentrations by the methods used for energy adjustment. The same approach was used to adjust adipose tissue tocopherols and carotenoids for the quantity of the adipose biopsy. Because some individual fatty acids are somewhat related to the total area of the chromatogram, we adjusted adipose tissue fatty acids for the total area of the lipid chromatogram by using the method described for energy adjustment.
All correlations among dietary assessment methods were computed from intakes adjusted for total energy intake, plasma triacylglycerol, cholesterol, quantity of adipose tissue, or area of the chromatogram. Since these data are from a matched case-control study, we computed Pearson's partial correlation coefficients on normalized nutrient intake estimates by coding the matching variables (i.e., age, gender, and county of residence in Costa Rica) as numeric regressors and used them as partial variables in the CORR procedure of SAS (32). Because of the established negative relation between smoking and amounts of carotenoids and tocopherols in the plasma and diets of smokers (30
, 33
), and the observed associations between these nutrients, smoking status, and body mass index in this study, we used regression methods to compute Pearson's partial correlation coefficients for these nutrients (32
).
Random within-subject error in estimating the intake of a nutrient tends to attenuate correlations toward zero (7). Therefore, we performed analysis of variance on the data from dietary recalls to estimate within- and between-subject variation and used the ratio of the two variances to correct correlations for day-to-day variation (7
). Briefly, the following formula was used:
, where Cr is the corrected correlation, r is the observed correlation,
x is the ratio of within to between subject variation, and nx is the number of dietary recall replicates for each subject. Only subjects whose data were not missing were included in the computation of
x, leaving at least 110 subjects for whom data on each of the nutrients were complete.
We used the method of triads (6, 14
, 20
) to estimate the VCs between "true" nutrient intakes and those estimated from using dietary assessment methods. Because data on biomarkers were available for carotenoids, tocopherols, and fatty acids only, VCs were computed for these nutrients only. The correlation between the FFQ and the biomarker, and the deattenuated correlations of the dietary recalls with the biomarker and the FFQ, were computed and were used to estimate the VCs (14
, 20
). Three measurements, a biomarker (e.g., plasma or adipose tissue), the mean of dietary recalls, and the estimate from the FFQ (e.g., Q1 or Q2 or their average, Q3) were used to estimate the VCs (figure 1). The VCs obtained through the various dietary assessment methods were then compared by assuming that they should be about equal if the assessment method was reproducible (e.g., FFQ1 vs. FFQ2) or if all methods were relatively valid. We used bootstrap sampling to construct confidence intervals around the VCs (14
, 20
). A total of 1,000 bootstrap samples of equal size (n = 120) were obtained from 120 study subjects by random sampling with replacement. For each bootstrap sample, VCs for the FFQ, dietary recalls, and the biomarker were obtained and were merged into a single data set. The UNIVARIATE procedure of SAS software, which uses the empirical distribution function (32
), was applied to compute the 5th and 95th percentiles that we used as the nonparametric confidence interval for the VC.
RESULTS
Descriptive statistics
The subjects in this validation study were similar to the overall control group with respect to most variables, including biologic parameters (table 1). However, this study included a larger proportion of women (35 percent) than did the total control population (27 percent). The mean absolute intakes estimated by dietary recalls and the FFQs are shown in table 2. For most nutrients, intake estimates obtained by FFQ tended to be higher than those obtained by dietary recall. The average FFQ values also tended to be higher than the mean of the dietary recalls, and the differences were significant (p < 0.05) for all nutrients but vitamin K, iron, and caffeine. On average, the FFQ overestimated daily energy intake by 477 kcal.
|
|
|
|
|
|
DISCUSSION
Unlike most validation studies, in which data from only one FFQ interview and one reference method are available, our study estimated nutrient intake by using three methods (FFQ, dietary recalls, and biomarkers). The sample size of 120 subjects, coupled with seven dietary recall replicates per subject, allowed for reasonable precision in estimating the correlations (7). In addition, potential seasonal variation in food intake, especially of fruits and vegetables, was considered in the study design. The analysis also produced a better estimation of long-term intake because the two FFQs were averaged.
In this study, the correlations between estimates obtained by the FFQ and the dietary recall are similar to those observed between FFQs and dietary records in other studies (8, 19
). The median of the correlations between the dietary recalls and average FFQ measurements was 0.55, again indicating good validity of the FFQ. Relative to the dietary recalls, the FFQ also performed well in estimating intake of fatty acids, irrespective of whether they were saturated or unsaturated. The highest correlations were found for linoleic (r = 0.73) and myristic (r = 0.70) acids. The FFQ was highly reproducible, as shown by Pearson's partial correlation coefficients ranging from 0.33 to 0.77. Intraclass correlations were also high (data not shown).
In comparison with dietary recall estimates, lycopene seemed to be equally well estimated from both plasma (r = 0.45) and adipose tissue (r = 0.42). Plasma was also a good biomarker for ß-cryptoxanthin (r = 0.43) and zeaxanthin + lutein (r = 0.43), and adipose tissue was a good biomarker for ß-carotene (r = 0.43). The correlation between plasma and adipose tissue -tocopherol measurements (r = 0.41) was similar to the one (r = 0.39) reported in the European Community Multicenter Study on Antioxidants, Myocardial Infarction and Cancer (EURAMIC) of the Breast (34
). However, our correlations between plasma and adipose tissue ß-carotene (r = 0.50) and lycopene (r = 0.43) were higher than those reported in the EURAMIC study (ß-carotene, r = 0.39; lycopene, r = 0.24).
Measurements of saturated fatty acids and monounsaturated fatty acids from adipose tissue correlated poorly with those estimated by the FFQ. This poor correlation may reflect the fact that other than dietary sources, saturated fatty acids can be synthesized endogenously; furthermore, other physiologic processes may influence adipose tissue concentrations. Compared with plasma, higher coefficients of variation were observed for nutrients measured from adipose tissue. We determined that this high variation is due to nonuniform distribution of the nutrient in adipose tissue and could in part explain the poor correlations between adipose tissue measurements and those obtained by the FFQ and dietary recalls. Estimates of saturated fatty acids from dietary recalls correlated better with those from the FFQ than the measurements from adipose tissue, suggesting that although biomarkers are informative, there are nutrients for which the FFQ remains a better dietary assessment method.
The VCs estimated for the FFQ were high, suggesting that the FFQ is a valid dietary assessment method. We also estimated VCs for the dietary recall and the biomarker. Doing so enabled us to correct correlations for potential attenuating effects of correlated errors in repeated dietary recalls (6) and therefore to simultaneously compare the relative performance of the three methods. For some of the nutrients from the average FFQ (e.g., ß-cryptoxanthin and lycopene), the VCs for the questionnaire were higher or similar to those for the biomarker, suggesting that the FFQ and the biomarker are comparable. Published data on VCs are scarce, especially those from studies in which dietary recalls are used as a reference method. In one study (n = 61) in which serum was used as a biomarker for ß-carotene and dietary recall as a reference dietary assessment method, the VCs for the FFQ, dietary recall, and biomarker were 0.44, 0.58, and 0.32, respectively; in our study, the VCs were 0.76, 0.71, and 0.50, respectively (20
). In another study (n = 87) in which dietary records were used as a reference method (14
), the VCs for the FFQ, dietary records, and plasma ß-carotene were 0.39, 0.52, and 0.85, respectively. The disparity between estimates obtained from these studies could be due to the different reference methods used, random error resulting from differences in sample sizes, differences in the populations studied, or the structure and size of the questionnaires used in these studies.
The VCs for FFQ1 and FFQ2 were very similar, further suggesting that the FFQ is a reproducible dietary assessment method. Random error could explain the Heywood cases found in our analysis. Thus, sample sizes of >120 may be necessary to stabilize the variances in validation studies. Efforts to correct for random error and to constrain VC estimates between 0 and 1 are needed to perfect this method. We cannot rule out the potential for correlated errors between the FFQ and dietary recalls; thus, the VCs reported in this study could have been overestimated and should be regarded as upper limits of the true VC.
This study has shown that the FFQ is a valid and reproducible instrument for assessing dietary intake among Hispanics in Costa Rica. Furthermore, the VCs suggest that plasma could be used as a biomarker for -tocopherol, ß-carotene, ß-cryptoxanthin, lycopene, and zeaxanthin + lutein, while adipose tissue could be used as a biomarker for lycopene and polyunsaturated fatty acids. This study also shows that biomarkers did not perform better than the FFQ and that they should be used to complement the FFQ rather than substitute for it.
ACKNOWLEDGMENTS
This study was supported by National Institutes of Health grant HL49086.
The authors thank the field workers of Proyecto Salud Coronaria, San José, Costa Rica, for their help with data collection.
NOTES
Correspondence to Dr. Hannia Campos, Department of Nutrition, Room 353A, Harvard School of Public Health, Harvard University, Boston, MA 02115 (e-mail: hcampos{at}hsph.harvard.edu).
REFERENCES