1 Cancer Etiology Program, Cancer Research Center of Hawaii, University of Hawaii, Honolulu, HI.
2 Department of Preventive Medicine, School of Medicine, University of Southern California, Los Angeles, CA.
Received for publication November 30, 2001; accepted for publication May 24, 2002.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
data collection; dietary supplements; epidemiologic methods; ethnic groups; nutrition surveys; questionnaires; vitamins
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
When a substantial proportion of a study population is taking dietary supplements, total intake of vitamins and minerals of interest cannot be determined unless supplement use, as well as food consumption, is measured. If supplement use is not considered in nutritional analyses, observed associations between diet and health may be attenuated or even misleading. Although nutritional epidemiologists have urged that accurate data on supplement use be collected (4, 5), there is a paucity of information on both the validity and the reliability of current collection methods.
The baseline questionnaire for the Hawaii-Los Angeles Multiethnic Cohort Study contained several questions about the use of vitamin and mineral supplements. Because of the large number of participants, information on brand names was not requested. However, a subsample of participants in a calibration study also completed three 24-hour recalls which included detailed information on supplements consumed and a second questionnaire. In this paper, we present the results of a validation study comparing reported usage from the two instruments and the results of a reproducibility study comparing reported usage over a 2- to 4-year period.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Questionnaires
The dietary questionnaires were self-administered and included questions about the use of eight vitamin and mineral supplements. Subjects were asked to indicate whether any of the following supplements had been used at least weekly during the past year: multivitamins or multivitamins with minerals; vitamin A; vitamin C; vitamin E; ß-carotene; calcium; selenium; and iron. If a supplement had been used, subjects were asked to indicate one of five categories of use: 13 tablets per week, 46 tablets per week, one tablet per day, two tablets per day, or three or more tablets per day. Use of a supplement less often than once a week was defined as never use. For all categories but the multivitamin category, subjects were also asked to indicate the approximate dosage per tablet, choosing from several dose ranges. Nutrient levels in a multivitamin were calculated as composites of levels in the two multivitamin brands most frequently reported in the dietary recalls (Centrum Silver and Centrum Hi Potency (Wyeth Consumer Healthcare, Madison, New Jersey) were reported by 33 percent of persons who gave a brand name). Previous analyses had shown that this default choice minimized errors in intake estimates (8). If a subject did not know the nutrient level of a particular vitamin or mineral supplement consumed, the amount in the lowest amount category was assumed.
Recalls
Three 24-hour recalls were administered by registered dietitians during telephone interviews. Days of the week were randomly assigned for the interviews in order to obtain a balance of all seven days. Recalls were completed only on days that subjects considered typical of their usual intake. Subjects were asked whether they had used any dietary supplements on the day of the recall. If they had, the type of supplement and the brand name, place of purchase, dosage, number of tablets per dose, and number of tablets taken were recorded. Supplement intakes were then quantified using a supplement composition table compiled and maintained by the Cancer Research Center of Hawaii. Default values based on the most commonly consumed product in the same category were used when no matching composition data could be located (approximately 4 percent of the recall days).
To match the six categories of frequency of supplement use that appeared on the questionnaire, we defined approximately equivalent categories for the 3 days of recall: never = not reported on any of the days; 13 tablets per week = one tablet reported across all three recalls; 46 tablets per week = two tablets reported on the recalls; one tablet per day = 34 tablets reported on the recalls; two tablets per day = 57 tablets reported on the recalls; and three or more tablets per day = eight or more tablets reported on the recalls.
Several assumptions were made when categorizing vitamin and mineral supplement data from the recalls to match the eight types of supplements on the questionnaire. Multiminerals, cod liver oil, and all herbal products were excluded. On the basis of preliminary descriptive analyses, calcium supplements that included vitamin D or magnesium were categorized as calcium supplements, and three products that contained vitamin E and selenium were considered vitamin E supplements.
Calculation of amounts consumed
For the questionnaire data, frequency of use was multiplied by the dosage reported (or assumed, for multivitamin use) to obtain an estimate of the amount consumed daily for each supplement. For the recall data, the daily amount consumed was the average across the 3 days of intake for each product. For both instruments, the total amount consumed per day was then calculated as the sum of the nutrient levels of multivitamins, if reported, plus the nutrient levels of all single vitamin or mineral supplements reported. For the questionnaire, amounts of the B vitamins and zinc reflect only those amounts contained in multivitamins, because intake of these nutrients in other forms was not assessed. To correspond with current recommendations, vitamins A and E in supplements were converted from International Units to equivalent weight measures (9, 10).
Statistical analyses
Agreement between the two questionnaires and between the second questionnaire and the recalls was measured in three ways: frequency of use in two categories (ever use, never use); frequency of use in six categories (ranging from never to three or more per day); and nutrient amount. Kappa coefficients were calculated to correct for chance agreement between frequency-of-use categories (11). Weighted kappa coefficients ( w) were used to weight by the level of agreement when frequency of use was divided into six categories. The weighting factors for disagreement were proportional to the square of the distance between the cells in the 6 x 6 table. Confidence intervals were computed for
w assuming normality. The homogeneity of kappa statistics between age, sex, and ethnic subgroups was tested using a
2 statistic (11).
Because the large number of zero values precluded the use of even nonparametric statistical tests, agreement in nutrient amounts was determined only for persons who reported use of a supplement on both instruments. Before these comparisons were conducted, outlier values were excluded, reducing the sample size by less than 1 percent for each nutrient. Pearsons correlation coefficients were calculated for the log values of these continuous variables.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Agreement for frequency of use
Agreement between the 24-hour recalls and the second questionnaire for frequency of use is shown in table 2. When data were dichotomized into users versus nonusers (never use vs. ever use), agreement was very good ( > 0.6) for the most commonly used supplements (multivitamins, vitamin C, and vitamin E) and moderately good for the remaining supplements, with the exception of vitamin A. Agreement was similar when use across six frequency-of-use categories was evaluated. The proportion within one category was high (0.800.98).
|
|
To examine whether consistency of use varied with time between administrations of the questionnaire, we divided the elapsed time into tertiles. Weighted kappas declined slightly as the time between administrations increased, but the differences were not large, nor were they consistent across nutrients (data not shown). Averaged across the eight types of supplements, kappa was 0.54 when the elapsed time was less than 704 days (lowest tertile) and 0.48 when the time was greater than 1,016 days (highest tertile).
Agreement for supplement amount
For subjects who reported use of a supplement on both the recalls and the questionnaire, the amounts consumed are presented in table 4. Among the seven nutrients with dosage amounts from the questionnaire, mean intake on the questionnaire tended to be greater than mean intake from the recalls (or, for ß-carotene, approximately equal). As expected, intakes of most of the B vitamins were higher on the recall reports, because use of single supplements of these vitamins was not captured on the questionnaire.
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The short questionnaire also had limitations, including the need to assume a default value for the composition of multivitamins. The dosage of a supplement was likely to be measured more accurately with the dietary recall method, because participants were asked to give the actual name of the supplement used and, in some cases, to read the name from the label to the interviewer. However, although there was substantial variation in nutrient levels among the multivitamin and mineral products commonly reported during the recalls, correlations with the questionnaire were similar when two different default options were examined. Likewise, Patterson et al. (12) examined the effect of imputing values for three nutrients in multivitamins (vitamin E, folic acid, and iron) and found good correlations with actual intake levels for vitamin E (0.84) and folic acid (0.61), although the correlation for iron (0.29) was poor.
In addition, many participants appeared to report complexes of nutrients as single supplements on the questionnaire (e.g., calcium and vitamin D complex was often reported as a calcium supplement). As a result, amounts of other nutrients in complexes were not included in the calculation of total intake. It is also possible that dosages reported on the questionnaire were less accurate because there were only 35 dose categories available, and the lowest category may have been higher than the subjects usual dose.
Absolute differences in dosage levels may not be important for many epidemiologic studies, as long as intakes are ranked similarly by the two instruments. However, correlations between the nutrient intake levels were low (<0.4) for three nutrients (vitamin A, selenium, and iron) (table 4), implying that intakes would be ranked differently. With correlations of approximately 0.7 for vitamins C and E, rankings appeared more similar, though absolute nutrient intake levels from the questionnaire were higher.
When intake of a nutrient from single supplements is not captured on the questionnaire, it is likely that total intake from supplements will be underestimated (see lower half of table 4). Surprisingly, this was not always the case; intakes of both folate and zinc were higher using the questionnaire data, perhaps because supplemental intake of these nutrients is primarily from multivitamins. Correlations were very poor, however, implying that individual intakes were not ranked similarly by the two instruments. For the remaining B vitamins, the average intake was lower on the questionnaire, but correlations were somewhat higher than those for zinc and folate (ranging from 0.29 to 0.35). Thus, because the questionnaire lacked questions about use of these nutrients as single supplements, nutrient intakes that are based only on multivitamin and mineral supplements may not be useful in analyses of disease outcomes; associations are likely to be severely attenuated.
Because more than 2 years elapsed between the two administrations of the questionnaire, this study could not measure true instrument reliability (i.e., the repeatability of responses, assuming no change in supplement intakes). As a result, the agreement between the two responses shown in table 2 is a measure of consistency of supplement use, because change in use would be expected over a period of years. The kappa statistics shown in table 2 for multivitamins, calcium, and vitamins C and E indicate relatively high long-term stability in the use of these supplements. Kappa values were lower for iron and vitamin A, which could reflect changes over time in use of these supplements, as might be expected if iron is taken therapeutically. Agreement tended to decline slightly as the time between administrations increased, supporting the hypothesis that agreement would have been higher if the elapsed time had been shorter (months rather than years).
One of the few studies to report on the accuracy of supplement-use data in the United States was conducted by Patterson et al. (5). These authors compared the results of a detailed in-person interview with a short questionnaire similar to the one used in our study. On average, they found no evidence of bias (either underreporting or overreporting) on the questionnaire. Correlations between intakes from the two instruments were remarkably similar to ours, with the highest agreement being observed for vitamins C and E. Unlike dietary recalls, this interview method attempted to capture usual usage. However, it still did not provide a completely accurate measure, because it was necessary for participants to provide a single estimate of use (times per week) over the past year, even though usage may have varied over time. In addition, although current supplement labels were examined, other brands that had been used during the past year may not have been captured.
In another study, Patterson et al. (13) examined differences between long-term intake (over the past 10 years) and current intake of three dietary supplements and found substantial disagreement, particularly in dosage levels. We also found that reproducibility of intake was low for some nutrients, particularly those that were not frequently reported. Thus, it appears that individuals change their frequency of use of supplements as well as the dosages.
Ishihara et al. (14) compared the ability of a questionnaire versus multiple dietary recalls (for four 7-day periods) to classify participants in Japan as users or nonusers of supplements. Having up to 28 days of recalls should allow more accurate classification of users than having only 3 days as in the current study. However, the kappa value calculated from Ishihara et al.s data for use of any supplement was 0.66, which is slightly lower than our kappa of 0.69 for use of multivitamins.
If the true measure of interest is long-term exposure, administration of daily or even periodic dietary recalls or records of supplement use is not practical for most studies. However, these analyses indicate that asking more questions on the type of multivitamin used, as well as giving more detailed instructions regarding the classification of multinutrient formulations, would increase the accuracy of questionnaire intake estimates. For nutrients of particular interest in a study, specific questions regarding use of single nutrient supplements should be included.
Neither instrument in our study could be considered a "gold standard" measure of dietary supplement intake. Data from the recalls covered too short a time period to represent usual supplement use, while data from the questionnaire covered a longer time period but did not reflect specific product use. Although our study yielded information about the types of measurement error incurred with the use of questionnaires, more research is needed. A correct assessment of associations between dietary intakes and disease outcomes requires that we better understand how to collect accurate data on supplement use and dosages. Substantial effort has been devoted to understanding the error structure of dietary intake data. A similar effort is needed for supplement intake data.
![]() |
ACKNOWLEDGMENTS |
---|
The authors thank Lucy Liu Shen for programming assistance.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|