1 Department of Community Medicine, Malmö University Hospital, Malmö, Sweden.
2 The NEPI Foundation, Malmö, Sweden.
3 Department of Medicine, Surgery and Orthopedics, Malmö University Hospital, Malmö, Sweden.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
bias (epidemiology); estrogens; pharmacoepidemiology; progestational hormones; questionnaires; women
Abbreviations: BMI, body mass index; MDCS, Malmö Diet and Cancer Study.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
One aim of the present study was to perform a reliability analysis, that is, to study the ability to replicate results whether or not they were correct, by assessing the agreement (kappa value) between the questionnaire and the diary. We also explored the association between discrepancy and eight individual characteristics. By conducting a validity analysis, we aimed to determine whether the questionnaire achieved an accurate result, that is, one that lacked systematic error; the personal diary was considered the gold standard. We calculated specificity, sensitivity, and positive and negative predictive values.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Information on current use of hormone therapy was obtained from 1) a self-administered questionnaire that included the open-ended question, "Which medicines do you use on a regular basis?" and 2) a structured, 7-day personal diary that included a specific, open-ended item for recording drug use. Each participant completed both information sources at home within a 12-week period between the first and second visits to the study center at baseline.
All pharmacologic agents reported in the personal diary or the questionnaire were classified according to the 1997 version of the Anatomic Therapeutic Chemical classification system (3). Hormone therapy was defined by the classification codes G03C (estrogens), G03D (progestagens), and G03F (progestagens and estrogens in combination); use of anxiolytics/hypnotics or opiates was defined by codes N05B (anxiolytics), N05C (hypnotics and sedatives), N03AE (benzodiazepines), N02A (opioids), N02B51C (in this study, paracetamol in combination with codeine), and N02AA59 (in this study, aspirin in combination with codeine).
The population was divided into four groups by quartile of body mass index (BMI), expressed as weight (kg) divided by height (m) squared (kg/m2). Height and weight were measured at the study center at baseline. High alcohol consumption was defined as an intake of 25 g or more of alcohol per day as reported in the personal diary. The rest of the information was obtained from the self-administered questionnaire.
Smoking status was classified as regular, infrequent, former, or never. Age at entry into the study was aggregated into the four categories of 4549, 5059, 6069, and 7073 years. Marital status was defined as married, unmarried, divorced, or widowed. Country of origin was dichotomized into Sweden and foreign. Finally, educational level was categorized as less than 9 years, 911 years, or more than 11 years of education.
Reliability analysis
We conducted a reliability analysis to assess to what extent the questionnaire and the personal diary agreed (i.e., the ability to replicate results whether or not the information was correct). We calculated the percentage of agreement and the related kappa coefficient (4) for dichotomous (yes vs. no) self-reported current use of hormone therapy. The kappa coefficient is a measure of the degree of nonrandom agreement between two measurements of the same categorical variable (5
). A p value of <0.05 was required for rejection of the null hypothesis of no agreement other than by chance.
Prevalence of current use of hormone therapy (i.e., the percentage of women reporting current use) was computed according to information from the questionnaire, from the diary, from the questionnaire or the diary, and from the questionnaire and the diary. Discrepancy was considered present if a woman reported in the questionnaire but not in the diary, or vice versa, that she currently used hormone therapy. We observed whether women tended to report hormone therapy use more in the questionnaire than in the diary (or vice versa) by computing the difference between the percentage of women reporting in the questionnaire only and the percentage reporting in the diary only. A positive difference indicated use reported more often in the questionnaire.
Prevalence of hormone therapy use in the general population
We compared the prevalence of hormone therapy use in the general population (in the southwest part of the county of Skåne, where the city of Malmö is located) according to the National Prescription Survey (6) with the prevalence according to the questionnaire, according to the diary, and according to the questionnaire or the diary. We restricted our analysis to women aged 5660 years (mean age in the MDCS was 58 years) and to the years 19921995 (when the MDCS was being conducted).
Characteristics associated with discrepancy
The odds ratios of discrepancy (i.e., reporting in the questionnaire but not in the diary, or vice versa, of current use of hormone therapy) were calculated as a function of age, smoking status, alcohol consumption, use of anxiolytics/hypnotics or opiates, BMI, marital status, country of origin, and educational level (refer to Materials and Methods) by using logistic regression analysis. The analyses were performed unadjusted for each of the characteristics and were adjusted by entering all eight characteristics as covariates in the same model.
Validity analysis
In the validity analysis, we studied whether the questionnaire produced an accurate result (one that lacked systematic error) by calculating its sensitivity, specificity, and positive and negative predictive values; the personal diary was considered the gold standard. Computations were performed by using the on-line calculator at the Canadian Centres for Health Evidence (Internet site: http://www.cche.net/).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Reliability analysis
Agreement between the two methods was high at 95.5 percent (kappa = 0.840, p < 0.001), and similar values were obtained for specific groups of hormones (table 1). The lowest discrepancy was found for women aged 7073 years (2.8 percent) and the highest (6.3 percent) for women with high alcohol consumption (table 2). The women tended to report more current hormone therapy use according to the questionnaire, especially those with high alcohol consumption (2.33). However, users of anxiolytics/hypnotics and opiates reported more use according to the diary (-1.23).
|
|
The adjusted odds ratios (table 2) were similar to the unadjusted odds ratios. However, the adjusted point estimates moved slightly toward the null value except those for use of anxiolytics/hypnotics or opiates and for age 5059 years, for which the estimates moved slightly away from the null value.
Validity analysis
The outcome of the validity analysis is presented in table 1. The specificity and sensitivity of the questionnaire were 97.7 and 84.5 percent, respectively. Sensitivity was lowest (80.2 percent) for progestagens and highest (89.6 percent) for progestagens and estrogens in combination (table 1).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Note that our study analyzed only current hormone therapy use. Questionnaires may provide information about past use of medication, but diaries are suitable for current use only. Recall of exposure in the past may be less accurate and less complete (7).
Women who participated in the MDCS, especially those included in the present study who provided complete drug use information in both the questionnaire and the diary, may be a select, more motivated population (e.g., because of family or personal history of disease) for whom agreement is higher. Compared with the women who provided complete information, those whose information was incomplete were older (2 vs. 10 percent in the group aged 7073 years), more of them used anxiolytics/hypnotics or codeine (4 vs. 9 percent), and more were born in Sweden (83 vs. 88 percent). In addition, compared with the women who completed both the questionnaire and the diary, those who completed only the questionnaire used more anxiolytics/hypnotics or codeine (8 vs. 12 percent), more were within the higher BMI quartile (25 vs. 42 percent), and fewer were married (61 vs. 49 percent). Therefore, there is a risk of selection or response bias (e.g., if results for the excluded women were more discrepant, the lower discrepancy found for the highest BMI quartile might be an overestimation, as these women were found more often in the excluded group with incomplete information).
Furthermore, the MDCS may not be representative of the general population. The MDCS questionnaire was also used in a mailed health survey carried out in Malmö in 1994 (8). The response rate for the mailed survey was 74.6 percent, twice the MDCS participation rate. Compared with the mailed survey, the MDCS included a higher proportion of employed persons and subjects born in Sweden and a lower proportion of retired subjects. However, in both studies, participants were similar with regard to educational level, type of employment, marital status, smoking habits, and weight distribution. Therefore, MDCS participants could be regarded as reasonably representative of the general population. Moreover, data from the National Prescription Survey (6
) suggest that the MDCS rather reliably estimated the prevalence of hormone therapy use in the general population.
Differences in payment or prescription practices might have influenced hormone therapy use. However, payment practices probably did not influence the results; in Sweden, the government reimburses a major part of the cost of prescription drugs. Therefore, during the baseline period, the total cost per year for hormone therapy could not exceed SEK 450 (US $56). On the other hand, different local therapeutic traditions might have led to differences in hormone therapy use between women from different areas.
Nondifferential misclassification of covariates may bias the estimate either toward or away from the null value (9) and may lead to a partial loss of ability to control confounding (10
). Therefore, it is necessary to consider the degree of misclassification when deciding whether to control a covariate (11
). Of the covariates studied, high alcohol consumption may be more prone to misclassification (12
). Also, continuous age and BMI variables were categorized in our analysis, which may have introduced misclassification. Excluding the alcohol variable or using continuous age and BMI did not affect point estimates in multivariate analyses.
As the questionnaire and the personal diary were completed during the same 2-week period, it is possible that recording of information in the diary may have influenced memory when the questionnaire was being completed. This bias would have overestimated the agreement. However, as neither the questionnaire nor the personal diary focused on drug use, this potential source of bias probably had a limited effect on the results.
Compared with the women aged 4549 years, the discrepancy for the group aged 5059 years was higher; for older women in our study, the discrepancy was similar or even lower. Other studies (13) have observed a higher agreement for younger subjects, but these studies compared self-reported questionnaire data with pharmacy databases. Older women are prescribed more drugs; therefore, according to database information, they appear to use more drugs. However, underuse is particularly common in the elderly, and pharmacy databases may overreport current hormone therapy use (14
).
The higher discrepancy for users of anxiolytics/hypnotics or opiates could be due to impaired memory recall (15), which also could explain why these women reported in the personal diary more current use of hormone therapy (table 2). Higher educational level also was associated with a higher discrepancy. In the literature, results conflict when interviews are compared with physicians' records (16
) or with prescription databases (17
). Nevertheless, our study was based on two self-reported information sources, and highly educated women may have reported in the questionnaire the drugs they purchased rather than the drugs they actually used.
There could be a reasonable explanation for the observed discrepancy. Those women who reported in the questionnaire but not in the diary that they currently used hormone therapy might have been reporting during a period of low compliance.
In summary, compared with a 7-day personal diary, a more convenient, self-administered questionnaire that includes an open-ended question on drug use seems to be a valid method for assessing current hormone therapy use in large epidemiologic studies.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|