Comparison of Assessment Methods for Pesticide Exposure in a Case-Control Interview Study

Julie L. Daniels1, Andrew F. Olshan2, Kay Teschke3, Irva Hertz-Picciotto2, David A. Savitz2 and Julie Blatt4

1 Epidemiology Branch, National Institute of Environmental Health Sciences, Research Triangle Park, NC.
2 Department of Epidemiology, School of Public Health, University of North Carolina, Chapel Hill, NC.
3 Department of Health Care Epidemiology, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada.
4 Division of Pediatric Hematology/Oncology, School of Medicine, University of North Carolina, Chapel Hill, NC.


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
In epidemiologic studies, much of the variation in disease risk estimates associated with occupational pesticide exposure may be due to variation in exposure classification. The authors compared five different methods of using interview information to assess occupational pesticide exposure in a US-Canada case-control study of neuroblastoma (1992–1994). For each method, exposure assignment was compared with that of a reference method, and neuroblastoma effect estimates were calculated. Compared with the reference method, which included a complete review of occupation, industry, job tasks, and exposure-specific activities, the use of occupation-industry groups alone or in combination with general job task information diluted the exposed group by including individuals who were unlikely to have been truly exposed. The effect estimates representing associations between each exposure method and neuroblastoma were different enough to influence the study's conclusions, especially when the exposure was rare (for maternal occupational pesticide exposure, the odds ratio was 0.7 using the reference exposure assessment method and 3.2 using the occupation-industry group exposure assessment method). Exposure-specific questions about work activities can help investigators distinguish truly exposed individuals from those who report exposure but are unlikely to have been exposed above background levels and from those who have not been exposed but are misclassified as exposed because of their employment in an occupation-industry group determined a priori to be exposed.

case-control studies; child; epidemiologic methods; interviews; neoplasms; neuroblastoma; occupational exposure; pesticides

Abbreviations: CI, confidence interval; IH, industrial hygiene; OR, odds ratio


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Assessment of occupational exposures continues to be a challenge in community-based case-control studies. The varying magnitude of exposure misclassification has been a likely contributor to the inconsistent results and conclusions of studies evaluating health effects of occupational pesticide exposure. While studies carried out within a specific industry can sometimes assess pesticide exposure using biologic or environmental measurements, retrospective population-based studies that assess diverse occupations have been limited to indirect methods of exposure assessment, namely telephone or in-person interviews or mailed questionnaires completed by study participants or proxies. Although in recent studies investigators have begun to collect more detailed information on occupational exposure, historically only job title and industry or self-reported exposure has been used to classify pesticide exposure. Compared with expert assessment of job and exposure histories, both of these methods can misrepresent the exposure experience of the individual (1GoGo–3Go). Industrial hygienists are becoming more involved in designing data collection instruments and reviewing responses; yet even industrial hygiene review of detailed job information, which is the currently preferred method and is often referred to as the "gold standard," is imperfect and remains unvalidated (4Go). Since there is no way to appropriately correct measures of association that are biased by exposure misclassification if a true gold standard is unavailable or exposure assessment methods are correlated (5Go, 6Go), it is important to consider how differences in exposure classification can affect the interpretation of effect estimates.

In this study, we evaluated and compared different methods of utilizing interview information to classify occupational pesticide exposure, using data from a population-based case-control study of neuroblastoma. Neuroblastoma is an early childhood tumor of the peripheral nervous system (7Go). Previous studies of parental occupational exposures during the periods before and during pregnancy have suggested a possible association between parental employment in agriculture and increased risk of neuroblastoma; however, methods of occupational exposure assessment have varied and results have been inconsistent (8GoGoGoGo–12Go). In the present study, parental occupational pesticide exposure was classified according to five different methods that used different combinations of interview data, including self-reported exposure, information on occupation and industry, and industrial hygiene review of job tasks and exposure details. We compared the proportions of parents classified as exposed by the different exposure classification methods and calculated pesticide exposure odds ratios associated with neuroblastoma to evaluate how much each exposure assessment method could affect the results and interpretation of a study.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Initial study
Details on the study population have been provided elsewhere (13Go). Briefly, children with neuroblastoma diagnosed between May 1, 1992, and April 30, 1994, at hospitals in the Children's Cancer Group and the Pediatric Oncology Group throughout the United States and Canada were eligible to participate if the biologic mother was available for telephone interview and spoke English or Spanish. A total of 537 case mothers provided interviews that included information on occupational history (73 percent of the 741 mothers eligible). Fathers were eligible to participate only if the mothers also provided interviews. A total of 405 case fathers provided interviews (76 percent of those eligible). Controls were identified through telephone random digit dialing and were individually matched to cases on the child's date of birth (±6 months for cases aged <=3 years at diagnosis and ±1 year for cases aged >3 years at diagnosis). A total of 503 control mothers (72 percent of the 703 eligible) and 304 control fathers (61 percent of those eligible) provided interviews that included information on occupational history.

Trained interviewers conducted structured telephone interviews with each mother and father. A complete occupational history was collected for all jobs held by the parents for more than 6 months from the age of 18 years to the date of the child's diagnosis (or the reference date for controls); this included information on job title, employer, the employer's product or service, and typical job tasks. For each job held during the 2 years prior to the child's birth, the parents were also asked about chemicals they had been exposed to on the job, including the type of chemical, the activity being performed when exposure occurred, and the amount of time spent working with the chemical. To enhance recall, each parent was referred to an interview guide sent prior to the interview that included an occupational history form, a list of chemicals, and a list of work activities. The chemical list included broad pesticide categories (pesticides, insecticides, herbicides, fungicides, and fumigants). Work activities included specific operations performed with chemicals, such as mixing, spraying, and applying, as well as more vague operations such as "using" and "working near exposure." Copies of the interview instruments are available from the authors upon request.

For both mothers and fathers, occupational pesticide exposure during the 2 years prior to the child's birth was classified according to five distinct methods, without knowledge of case status (table 1). Method 1, referred to as "self-report," classified parents solely on the basis of their own reports of occupational pesticide exposure (ever/never). Method 2, referred to as "occupation-industry," classified parents solely on the basis of their occupation and industry (ever/never), independently of self-reported exposure. Occupation and industry were coded using the 1980 US Bureau of the Census classification (14Go) and paired into occupational groups using a previously developed scheme that aggregates jobs with historically similar tasks and potential exposures (15Go). The original scheme was modified according to the distribution of the specific occupations and industries in this study. Occupation-industry groups considered potentially exposed to pesticides on the basis of industrial hygiene (IH) literature and previous studies included animal caretakers, farmers and agricultural workers, florists and nursery workers, landscapers, and pest control operators (table 2). Individuals who were considered exposed according to both the self-report and occupation-industry group methods comprised the exposed group for method 3, which is referred to as the "self-report/occupation-industry" group.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Interview information used for different methods of pesticide exposure assessment among parents of children with neuroblastoma, United States and Canada, 1992–1994

 

View this table:
[in this window]
[in a new window]
 
TABLE 2. Occupational groupings from 1980 US Census occupation and industry codes represented in a study of parental risk factors for children's neuroblastoma, United States and Canada, 1992–1994

 
For the final two methods, two members of the study team (K. T. and J. D.) reviewed records of subjects who had been 1) classified as exposed by either method 1 or method 2 or 2) employed in an occupational group that included at least one subject self-reporting pesticide exposure in this study. Method 4, "partial IH review," reviewed job title, industry, employer's product or service, and daily job tasks but did not include information on self-reported pesticide exposure. The fifth method, "comprehensive IH review," reviewed all available interview information and served as our reference method. In addition to the information about industry and typical job tasks, comprehensive IH review considered self-report of pesticide exposure (ever/never), pesticide type (pesticide, insecticide, herbicide, fungicide, fumigant), chemical form (gas, dust, liquid, solid, etc.), exposure route (air, skin, clothing), work activity engaged in when exposure was incurred (mixing, spraying, working nearby, etc.), and average number of hours exposed per week. Both types of IH review were conducted by consensus, with masking as to case status. The exposed groups included individuals who, after review of the respective information, appeared likely to have had occupational pesticide exposure above background levels. The primary effect of the comprehensive IH review was reclassification of subjects considered exposed by either of the first two methods as unexposed, because either the duration of exposure was very low (i.e., <1 hour per week) or the exposure was indirect (e.g., "working near" exposure). Each exposure assessment method produced an ever/never classification representing the individual's exposure status, restricted to the period 2 years before the child's birth.

We then compared exposure assessment methods, treating the comprehensive IH review as the reference method. Although a comprehensive IH review is commonly considered the "gold standard" in epidemiologic studies (16Go), exposure classification by this method was not validated by biologic or environmental measurements, and it served as a reference only by our judgment, because all available information was used. Statistics calculated for the comparison included: kappa ({kappa})—the level of agreement between methods; sensitivity—the probability of being classified as exposed if truly exposed; specificity—the probability of being classified as unexposed if truly unexposed; positive predictive value—the probability that an individual classified as exposed is truly exposed (i.e., true positives); and negative predictive value—the probability that an individual classified as unexposed is truly unexposed (17Go).

Unconditional logistic regression was used to estimate odds ratios and 95 percent confidence intervals for the association between neuroblastoma and pesticide exposure according to each exposure classification method. Effect estimates were adjusted for the child's age (the matching factor), which met linearity requirements and was included as a continuous variable.

Qualitative follow-up study
We conducted a follow-up interview to qualitatively improve our understanding of the disagreement between self-reported pesticide exposure and IH assessment. We targeted a subset of fathers who reported either exposure to agricultural pesticides or employment in an occupation or industry considered likely to involve exposure to agricultural pesticides in the original interview. Forty-two fathers (67 percent of those targeted) provided both initial interviews and follow-up interviews.

The exposure-specific interview was administered by telephone 3–5 years after the original interview. It asked about the chemical or brand names of all pesticides used, the form of each chemical (e.g., liquid, granules, residue), the method of application (e.g., hand-held sprayer, tractor sprayer, postapplication work with treated crops or animals), use of protective equipment, personal washing practices, and the specific months in which the product had been used. An interview guide sent prior to the follow-up interview provided lists of common purposes for pesticide use, common pesticide application methods, and 95 chemical names along with their common trade names.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Initial study
The distributions of demographic characteristics among case and control parents were similar (data not shown). Mothers of children whose fathers did not participate reported lower income and education and were less likely to be White in comparison with mothers of children whose fathers provided personal interviews.

Twenty-seven fathers (3.8 percent) and seven mothers (0.7 percent) were classified as exposed to pesticides by comprehensive IH review, our reference method for the estimation of sensitivity, specificity, positive predictive value, and negative predictive value (table 3). By this method, neuroblastoma was associated with a modest but unstable increase in the odds of paternal occupational pesticide exposure during the 2 years before the child's birth (odds ratio (OR) = 1.5; 95 percent confidence interval (CI): 0.7, 3.4). No increase was seen for maternal exposure (OR = 0.7; 95 percent CI: 0.2, 3.2).


View this table:
[in this window]
[in a new window]
 
TABLE 3. Effect estimates for associations between parental occupational pesticide exposure and children's neuroblastoma and sensitivity, specificity, and predictive values for each method as compared with industrial hygiene review, United States and Canada, 1992–1994

 
The proportions of fathers and mothers exposed to pesticides were substantially higher when parents were classified by self-report: 53 fathers (7.5 percent) and 35 mothers (3.4 percent). Self-report was the most sensitive exposure classification method in comparison with comprehensive IH review for both parents (sensitivity = 100 percent). Although greater proportions of individuals were classified as exposed by self-report than by other methods, 49 percent of fathers and 80 percent of mothers who self-reported exposure were considered unlikely to have been exposed according to comprehensive IH review (positive predictive values were 51 percent and 20 percent for fathers and mothers, respectively). Sixty-eight percent of these fathers reported their only exposure activity to be "working near" pesticides or "moving" pesticides. In addition, all but one were in occupations not considered likely, a priori, to involve pesticide exposure. By comparison, only three (11 percent) of those classified as exposed by comprehensive IH review reported "working near" or "handling" pesticides as their only exposure activity; the other 89 percent reported activities such as "mixing," "pouring," or "spraying" pesticides. Neither paternal nor maternal self-reported pesticide exposure was associated with neuroblastoma (for fathers, OR = 1.0 (95 percent CI: 0.6, 1.7); for mothers, OR = 0.9 (95 percent CI: 0.5, 1.7)).

The sensitivity of exposure classification was lower for methods 2 (occupation-industry group) and 3 (self-report/occupation-industry group); with both methods, sensitivity was 70.4 percent for fathers and 57.1 percent for mothers (table 3). However, greater proportions of parents classified as exposed by method 3 were also classified as exposed by the reference method (for fathers, positive predictive value = 90.5 percent; for mothers, positive predictive value = 44.4 percent), and the overall concordance between the methods was higher (for mothers and fathers combined, {kappa} = 0.8; 95 percent CI: 0.7, 0.9). Effect estimates resulting from the self-report/occupation-industry group method were somewhat higher than those obtained from comprehensive IH review for fathers (OR = 1.9; 95 percent CI: 0.7, 4.9) and in the opposite direction of comprehensive IH review for mothers (OR = 1.9; 95 percent CI: 0.5, 7.6).

The sensitivity of partial IH review was slightly lower than that of the other methods. The paternal effect estimate, however, was similar to that of the reference method. The maternal effect estimate pointed in a direction opposite that of the reference method and was quite imprecise.

Qualitative results from follow-up study
In the original interview, 14 fathers who self-reported pesticide exposure were classified as unexposed by comprehensive IH review. In the follow-up interview, half (n = 7) of those fathers reported that they had not been exposed to pesticides during the relevant time period. Of those who maintained that they had been exposed, most explained that their exposure was very occasional or indirect.

Of the 31 fathers who reported pesticide exposure in the follow-up interview, 27 were able to recognize and report exposure to specific pesticides listed in the interview guide. The original interview did not provide a list of specific pesticides, and only four fathers reported exposure to specific chemicals. The original interview also queried about only the average number of hours exposed per week. Farmers, specifically, can have sporadic exposure patterns, being relatively unexposed most of the year but highly exposed during the planting and harvesting seasons (18Go). The follow-up interview asked about the specific months in which pesticides had been used and the frequency and level of use within those months, more clearly characterizing exposure patterns. This information allowed us to classify exposure by level and frequency rather than simply as exposure versus nonexposure. However, the small size of the substudy sample precluded calculation of effect estimates for more refined exposure categories and limited its use to qualitative explanation.


    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
This study illustrates how differences among exposure assessment methods can result in different effect estimates, especially when the overall exposure prevalence is low, as it was among mothers in this study. Misclassification of even a few people as exposed who are truly unexposed can greatly impact an effect estimate when exposure is rare (19Go). Evaluating the positive predictive value can be useful when comparing exposure classification methods and when interpreting differences in the resulting effect estimates, since positive predictive values incorporate information about the method's accuracy (number of false positives and false negatives) and the exposure prevalence (20Go).

Each method in this study used different pieces of interview information and produced clearly different proportions of individuals classified as exposed and unexposed. According to our IH review of all detailed occupational and exposure information, paternal occupational pesticide exposure was modestly (though imprecisely) associated with neuroblastoma, but maternal exposure was not. In contrast, the analysis based on job title suggested opposite patterns, with an association being seen for mothers but not for fathers. This demonstrates how the lack of a common, reliable exposure assessment method may yield inconsistent results among otherwise similar studies (9GoGoGo–12Go).

In general, exposure classification by occupation and industry is objective, standardized, and reliable (4Go). However, it is difficult to account for the exposure variability within an occupational group, even with a study-specific scheme (21Go, 22Go). We grouped occupations and industries that were likely to be similar with respect to pesticide exposure; however, exposure variability within these groups was possible because of the low prevalence of pesticide exposure and the diversity of job tasks. Although the paternal effect estimate for exposure by occupation-industry group was similar to estimates for other exposure methods, the maternal estimate was much higher and was possibly skewed by the small proportion of exposed mothers and the considerable uncertainty in the exposure definition. The self-report/occupation-industry method was better than either self-reports or occupation-industry groups alone, but all three methods produced small exposure groups and thus imprecise effect estimates.

Self-reported exposure can be highly sensitive to exposure variability between and within jobs, but it is not standardized and it relies on the individual's perception of and ability to recall exposure, both of which may be faulty (1Go, 23Go). In this study, self-reporting produced the most inclusive exposure group, but low specificity resulted in the greatest extent of misclassification in comparison with comprehensive IH review. Most individuals who self-reported being exposed but were classified as unexposed by comprehensive IH review reported ambiguous exposure activities such as "moving" or "working near" pesticides, rather than specific activities like "pouring," "mixing," or "applying." Although working near exposure could convey some exposure, in the follow-up interview we found that exposure among individuals reporting vague exposure activities seemed negligible in comparison with that among persons reporting specific exposure activities. Allowing response options like "working near exposure" can satisfy a respondent's need to report potential exposures of personal or public concern; however, by following up such responses with probing questions like, "What activities were you engaged in when you worked near pesticides?," the interviewer can collect descriptive information with which to construct a context for exposure and later assist in IH review.

Information on exposure-related activities may be more helpful than information on general job tasks. In our initial study, "farmers" often reported job tasks such as "managing the farm" or "raising grain crops and cattle." Our follow-up study revealed their exposure potentials to be very different: Some personally applied pesticides and harvested crops, while others acted as managers who contracted with pesticide application services and hired field workers. In another example, a "store clerk" at a "garden center" reported that his main job duty was "customer service," which would typically have classified him as unexposed, but he also reported having pesticide exposure. Follow-up revealed that he "fumigated plants" delivered to the store twice per week for 6 months of the year. Such examples further illustrate how exposure details can clarify exposure status. Computer-assisted telephone interviewing and other detailed data collection techniques that offer branching questions based on the interview-ee's response may be one way to collect necessary details in the initial interview and decrease the need for follow-up (24Go).

The ability of IH review to correctly classify exposure depends on the detail and quality of the information collected about the job and exposure, the experience of the hygienist, and the amount of a priori information about exposure variation within a particular job and industry (25Go, 26Go). Our IH assessments resulted in different exposure classifications according to which interview information was used in the review. Partial IH review, without exposure-specific information, forced us to make assumptions about exposure. The IH literature offered little guidance for assessing exposure probability, because IH studies have been limited to very specific exposure situations within industry settings, impeding extrapolation to the diverse job activities in this study and most population-based studies. Our comprehensive IH review functioned primarily to remove unexposed people from the exposed group by identifying exposure activities that were not likely to have involved direct pesticide exposure. Through inclusion of the exposure-specific information, the comprehensive IH review allowed exposure variability within occupational groups but decreased overrepresentation of exposure by self-report.

Although IH review is often considered the gold standard in validity studies, the extent to which IH review can be considered a true gold standard, regardless of the amount of information used, cannot be known without validity testing using direct measurement data (27Go). Direct measurements are generally considered almost impossible to carry out in retrospective population-based studies, but validity substudies that compare IH review with environmental or biologic assays are needed (3Go, 21Go, 27Go).

Our results do not reflect a true validity assessment, but they permit qualitative comparison of the effect estimates resulting from each method and illustrate how different exposure assessment methods may account for the inconsistencies between study results and conclusions within certain areas of epidemiologic research. We assessed these methods in a population with diverse occupations; the performance of these methods may differ for other chemicals or in industry-specific settings, where pesticide exposure prevalence is higher. These methods primarily corrected for overreporting of pesticide exposure, and we were unable to use them to determine the extent of underreporting. Nonoccupational exposures that could increase an individual's overall pesticide burden were not considered but would not have affected the classification of the reported occupational exposures or the comparison of exposure assessment methods.

Current exposure assessment practices in population-based studies will continue to produce varied results. The time and expense involved in collecting and using detailed exposure data could ultimately lead to a better understanding of exposure-disease relations. This may be particularly valuable when multiple studies have produced inconclusive results due to poor exposure data and a more definitive study could produce the information needed to impact public health or regulatory decisions. Until biologic or environmental exposure assessment methods are more efficacious, researchers in future studies should investigate how to efficiently collect and use detailed information, limit the use of crude exposure assessment methods, and validate IH review as a comprehensive exposure assessment method.


    ACKNOWLEDGMENTS
 
This study was funded in part by grant CA 57004 from the US National Cancer Institute.

The authors acknowledge the participating institutions of the Children's Cancer Group and the Pediatric Oncology Group.


    NOTES
 
Reprint requests to Dr. Julie L. Daniels, Epidemiology Branch, National Institute of Environmental Health Sciences, MD: A3-05, 111 T. W. Alexander Drive, Research Triangle Park, NC 27709 (e-mail: daniels1{at}niehs.nih.gov).


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Fritschi L, Siemiatycki J, Richardson L. Self-assessed versus expert-assessed occupational exposures. Am J Epidemiol 1996;144:521–7.[Abstract]
  2. Siemiatycki J, Dewar R, Richardson L. Costs and statistical power associated with five methods of collecting occupational exposure information for population-based case-control studies. Am J Epidemiol 1989;130:1236–46.[Abstract]
  3. Tielemans E, Heederik D, Burdorf A, et al. Assessment of occupational exposures in a general population: comparison of different methods. Occup Environ Med 1999;56:145–51.[Abstract]
  4. McGuire V, Nelson LM, Koepsell TD, et al. Assessment of occupational exposure in community-based case-control studies. Annu Rev Public Health 1998;19:35–53.[ISI][Medline]
  5. Spiegelman D, Schneeweiss S, McDermott A. Measurement error correction for logistic regression models with an "alloyed gold standard." Am J Epidemiol 1997;145:184–96.[Abstract]
  6. Wacholder S, Armstrong B, Hartge P. Validation studies using an alloyed gold standard. Am J Epidemiol 1993;137:1251–8.[Abstract]
  7. Brodeur GM, Maris JM, Yamashiro DJ, et al. Biology and genetics of human neuroblastomas. J Pediatr Hematol Oncol 1997;19:93–101.[ISI][Medline]
  8. Kristensen P, Andersen A, Irgens LM, et al. Cancer in offspring of parents engaged in agricultural activities in Norway: incidence and risk factors in the farm environment. Int J Cancer 1996;65:39–50.[ISI][Medline]
  9. Michaelis J, Haaf HG, Zollner J, et al. Case control study of neuroblastoma in West-Germany after the Chernobyl accident. Klin Padiatr 1996;208:172–8.[ISI][Medline]
  10. Spitz MR, Johnson CC. Neuroblastoma and paternal occupation. Am J Epidemiol 1985;21:924–9.
  11. Wilkins JR, Hundley VD. Paternal occupational exposure to electromagnetic fields and neuroblastoma in offspring. Am J Epidemiol 1990;131:995–1107.[Abstract]
  12. Bunin GR, Ward E, Kramer S, et al. Neuroblastoma and parental occupation. Am J Epidemiol 1990;131:776–80.[Abstract]
  13. Olshan AF, De Roos AJ, Teschke K, et al. Neuroblastoma and parental occupation. Cancer Causes Control 1999;10:539–49.[ISI][Medline]
  14. Bureau of the Census, US Department of Commerce. 1980 census of population: alphabetical index of industries and occupations. Final edition (PHC80-R3). Washington, DC: US GPO,1982.
  15. Schnitzer PA, Teschke K, Olshan AF. A classification scheme for aggregating U.S. Census occupation and industry codes. Am J Ind Med 1995;28:185–91.[ISI][Medline]
  16. Stewart WF, Stewart PA. Occupational case-control studies. I. Collecting information on work histories and work-related exposures. Am J Ind Med 1994;26:297–312.[ISI][Medline]
  17. Hennekens CH, Buring JE. Epidemiology in medicine. Boston, MA: Little, Brown and Company, 1987.
  18. Nieuwenhuijsen MJ, Noderer KS, Schenker MB. The relation between subjective dust exposure estimates and quantitative dust exposure measurements in California agriculture. Am J Ind Med 1997;32:355–63.[ISI][Medline]
  19. Dosemeci M, Stewart PA. Recommendations for reducing the effects of exposure misclassification on relative risk estimates. Occup Hyg 1996;3:169–76.
  20. Maclure M, Willett WC. Misinterpretation and misuse of the kappa statistic. Am J Epidemiol 1987;126:161–9.[ISI][Medline]
  21. Flegal KM, Brownie C, Haas JD. The effects of exposure misclassification on estimates of relative risk. Am J Epidemiol 1986;123:736–51.[Abstract]
  22. Stengel B, Pisani P, Limasset J, et al. Retrospective evaluation of occupational exposure to organic solvents: questionnaire and job exposure matrix. Int J Epidemiol 1993;22(suppl 2):S72–82.
  23. Joffe M. Validity of exposure data derived from a structured questionnaire. Am J Epidemiol 1992;135:564–70.[Abstract]
  24. Stewart PA, Stewart WF, Heineman EF, et al. A novel approach to data collection in a case-control study of cancer and occupational exposures. Int J Epidemiol 1996;25:744–52.[Abstract]
  25. Stewart PA, Stewart WF. Occupational case-control studies. II. Recommendations for exposure assessment. Am J Ind Med 1994;26:313–26.[ISI][Medline]
  26. Siemiatycki J. Exposure assessment in community-based studies of occupational cancer. Occup Hyg 1996;3:41–58.
  27. Siemiatycki J, Fritschi L, Nadon L, et al. Reliability of an expert rating procedure for retrospective assessment of occupational exposures in community based case-control studies. Am J Ind Hyg 1997;31:280–6.
Received for publication April 4, 2000. Accepted for publication October 23, 2000.