Recall Bias in Self-reported Melanoma Risk Factors

Myles Cockburn, Ann Hamilton and Thomas Mack

From the Department of Preventive Medicine, University of Southern California, Los Angeles, CA.


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
The evidence implicating sun exposure in the etiology of melanoma derives largely from case-control studies in which the retrospective assessment of sun exposure suggests potential for significant recall bias. Previous attempts at characterizing and quantifying that bias have had significant methodological limitations. In the International Twin Study, a case-control study of melanoma risk factors in twins conducted from 1980 to 1991, the authors asked melanoma cases and their co-twins to quantify their own exposures and asked which twin had the greater exposure. Recall bias was investigated by assuming that, if bias had occurred, the odds ratio based on the case's response would differ significantly from the odds ratio based on the co-twin's response. Case-derived odds ratios were higher than the odds ratios for the controls for sunbathing in childhood and adulthood and for mole frequency and freckling in childhood, suggesting some recall bias. The odds ratios for ease of burning and tanning appeared unbiased. The belief that sunlight was a cause of melanoma appeared related to an increased odds ratio for sunbathing as a child. There is a continuing need to carefully assess recall bias in the study of melanoma risk factors.

bias (epidemiology); melanoma; recall; risk factors; sunlight; twin studies

Abbreviations: CI, confidence interval;; OR, odds ratio


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
The evidence implicating sun exposure in the etiology of melanoma derives largely from case-control studies that assess sun exposure retrospectively in people already diagnosed with melanoma (1Go). There is the potential for significant recall bias in the estimation of past exposures relevant to melanoma, especially when there has been a great deal of public information about the relation between sun exposure and melanoma (2Go).

Assessing the reproducibility of risk assessment questionnaires is one way in which sun exposure recall bias has been quantified (3GoGo–5Go). This method is limited to identifying random or systematic biases in repeated self-reports of sun exposure, and it cannot truly estimate recall bias unless one measure preceded the development of melanoma (6Go). In one study in which this was possible, women in the Nurses' Health Study cohort reported a reduced ability to tan after their diagnosis of melanoma in comparison with their ability to tan reported before the diagnosis of melanoma, but sun exposure variables were not investigated (5Go).

We present here a novel approach to investigating recall bias in exposures related to melanoma. In a case-control study of melanoma risk factors in twins, we asked melanoma cases and their co-twins quantitative questions about their own exposures and asked them to comparatively assess which twin had the greater exposure. We were able to investigate recall bias using these two types of questions and by comparing the twins' responses. First, we assumed that, if recall bias had occurred, the odds ratio based on the case's comparative response should have differed from the odds ratio based on the control's response for the specific exposure under study. We also created an odds ratio based on the joint comparative responses of the twins, taking as valid only those responses in which both agreed on which twin received greater exposure. We then compared the odds ratio derived from quantitative responses (e.g., "How much exposure did you have?") with those derived from comparative responses (case derived, control derived, and joint; e.g., "Which twin had the greater exposure?"), assuming that, in the presence of recall bias, the odds ratio based on the comparative responses would differ from the odds ratio based on the self-reported quantitative exposures. To evaluate if there was a precedent for recall bias, we also investigated whether the reported anatomic site of most frequent sunburn exposure was related to the anatomic site of the tumor and whether or not the participants' opinion of the cause of their melanoma appeared to influence their reports of exposure. Finally, we performed similar analyses on exposures not related to melanoma to determine if recall bias was specifically related to the widely publicized melanoma risk factors.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Case and control ascertainment
Cases of malignant melanoma and their disease-free co-twins (controls) were obtained from the International Twin Study. The purpose of this study that began in 1980 was the recruitment of pairs of twins by means of advertisements placed in newspapers throughout the United States and Canada, appealing for responses from twins with cancer or other chronic diseases. Over the course of 11 years, over 17,000 affected pairs were ascertained from all over North America, including roughly 12,000 pairs of twins affected by cancer. Among these were 569 pairs in which one or both twins were reported to have melanoma.

Participants were sent a detailed questionnaire on melanoma risk factors and demographic variables, sun exposure behavior and tanning or sunburn reactions, relevant constitutional variables, and sun protection behavior, with reference to both themselves and their twin. The questionnaire was initially sent to 921 individual twins in 553 pairs, after eliminating twins known to be deceased. The individual response rate was 71 percent. Among pairs, at least one questionnaire was received from 76 percent, and both twins from 212 pairs responded. Since this analysis requires the comparison of responses from both twins, it has been restricted to pairs where both twins responded.

Two methods were used to verify the diagnosis of the cases. The anatomic subsite, morphologic type, and behavior of melanomas were obtained from medical records with a signed authorization for release of records that was requested at the time of recruitment. In 27 of the 212 double-respondent pairs (14.6 percent), medical record review revealed that the case's diagnosis was not malignant melanoma but basal cell carcinoma (n = 16), squamous cell carcinoma (n = 10), or not cancer (n = 1), leaving 185 pairs of twins for analysis. Second, we obtained slides for the verification of histologic diagnosis for 133 (71.9 percent of 185) persons. In 128 cases (96 percent of 133), the original diagnoses were verified, in three cases (2 percent) a different diagnosis of melanoma morphology or behavior was obtained, and in two cases (2 percent) the pathologist was unable to provide a diagnosis. When there was a discrepancy, we used the diagnosis supplied by the pathologist. Hospital records provided an anatomic subsite of melanoma for 164 of the 185 cases. A comparison was made of the subsite reported on hospital records and the subsite reported by the cases. There were no instances in which the case report of subsite disagreed with the subsite given on his or her hospital record. For 19 of the 21 cases whose tumor site was unverified, the self-reported subsite was used. Two subsites remained unknown.

Twins were instructed to complete the questionnaire without conversing with their co-twin about the study. If participants wished to converse with their co-twin about the study, they were asked to complete the questionnaire a second time in a different color of ink if they wished to provide a different answer, rather than altering their original answers. Only the original answers were used in this analysis. The results of the case-control analysis of these data are being processed. Consent to participate in the study was given by participants when they completed their questionnaires. The Institutional Review Board of The University of Southern California/Los Angeles County General Hospital approved the study.

Comparative variables related to melanoma
Twins were asked whether they or their co-twin tanned more easily, burned more easily, had more freckles in childhood, had more large moles, sunbathed more as a child, and sunbathed more as an adult. Possible responses were "me," "my twin," "we were both the same," and "I don't know."

Comparative variables unrelated to melanoma
As a contrast, twins were asked whether they or their co-twin had more acne, had more warts, and perspired more. None of these exposures have been shown to be related to melanoma nor are they likely to be perceived by the public to be risk factors for melanoma, and they are therefore less likely to be subjected to recall bias with respect to melanoma.

Contrasts of comparative and quantitative questions
For two questions, there were both comparative and quantitative responses. That is, in addition to being asked which twin had more large moles and which twin had more freckles in childhood, respondents were also asked how many moles they had and how many freckles in childhood. It was therefore possible to compare the odds ratio determined using the case's or the control's report of who had greater exposure with the odds ratio comparing their self-reported exposures, based on the quantitative outcome.

Sun exposure-specific variables
Respondents were asked about anatomic site-specific sunburn and propensity for avoiding sun exposure at four anatomic sites (face and neck, back and chest, arms and shoulders, legs). The site-specific odds ratio for each sun avoidance behavior was calculated. In these odds ratios, the "disease" represented melanoma at the site in question versus melanoma at all other sites. The ratio of the odds ratio for melanoma at the site in question versus the odds ratio for all other sites combined was calculated, along with a corresponding 95 percent confidence interval.

Self-reported explanations for the genesis of melanoma
Cases were asked what they thought was the cause of their melanoma. Analyses were stratified by self-reported "cause" of melanoma. For example, results for analyses regarding sunburn exposures were stratified by the dichotomous variable reflecting whether or not the case thought his or her melanoma was caused by sunburn.

Statistical analyses
Agreement between the reports of cases and controls was assessed using the kappa coefficient, along with its 95 percent confidence interval, as a measure of correlation for comparative questions (7Go). In all assessments of agreement, only responses where the subject answered definitively ("I had greater exposure," "my twin had greater exposure," "we both had the same exposure") were used.

Matched odds ratios for comparative responses were calculated, using first the case's response as the measure of exposure, second the control's response as the measure of exposure, and finally by taking the joint response as the measure of exposure. For the joint comparative response where only one twin answered "the same" and the other twin had a definitive response, this was considered an exposure-discordant pair. If one twin responded "don't know," the pair was eliminated from the joint response as well. For example, when the case reported more large moles and the co-twin (the control) also reported more large moles, the first odds ratio considered the pair to be discordant, case exposed; the second odds ratio considered the pair to be discordant, control exposed; and the third odds ratio considered the pair to be concordant for exposure, both exposed.

To assess if recall bias was present, we systematically made the following odds ratio comparisons:

  1. case-derived comparative odds ratio versus control-derived comparative odds ratio for melanoma and nonmelanoma risk factors;
  2. for selected melanoma risk factor variables case-derived, control-derived, and joint comparative odds ratios versus the quantitative odds ratio;
  3. self-reported odds ratio for sun exposure to the specific body area where melanoma occurred versus exposure to other body areas;
  4. stratified analysis by the cases' view on causes of melanoma.
To determine if the odds ratios differed, we conducted statistical tests on the ratio of the two odds ratios. Matched odds ratios and their 95 percent confidence intervals were calculated with conditional logistic regression using the SAS programming language (8Go).


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
The 185 pairs in the study consisted of 100 monozygotic and 85 dizygotic pairs (table 1). There were 114 like-sex female pairs, 38 like-sex male pairs, and 43 male-female pairs. This population was generally well educated (with over 60 percent of both cases and controls having more than 12 years of education) and therefore likely to be aware of the link between sunlight exposure and melanoma. While two of three previous investigations of bias in the reporting of melanoma risk factors were based on females (3Go, 5Go), the pres-ent study had information from 109 males and 261 females.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Descriptive characteristics of melanoma cases and controls, International Twin Study, 1980–1991

 
Kappa coefficients for agreement of case responses and control responses
Generally, a kappa coefficient greater than 75 percent represents good agreement over and above chance (9Go). Agreement within twin pairs was greater than 75 percent for the melanoma-related risk factors ease of tanning, ease of sunburn (table 2), number of moles, and freckling in childhood (table 3). Among those factors not thought to convey risk of melanoma, agreement was high for degree of perspiration (table 2). Agreement was poorer for sunbathing as a child or adult and for the extent of body hair, acne, and warts (table 2).


View this table:
[in this window]
[in a new window]
 
TABLE 2. Comparison of matched odds ratios for melanoma risk factors using case's response, control's response, and agreement of subjects' responses, International Twin Study, 1980–1991

 

View this table:
[in this window]
[in a new window]
 
TABLE 3. Comparison of matched odds ratios and kappa statistics for melanoma risk factors, using data from comparative and quantitative outcomes, International Twin Study, 1980–1991

 
Assessing agreement of the comparative odds ratios
For the melanoma-related risk factors of tanning and burning more easily, both of which had high agreement, the case-defined and control-defined odds ratios were similar, with ease of tanning found to be protective and burning more easily associated with an elevated risk. However, for sunbathing more as a child or adult (both with poorer agreement), the case-defined odds ratio was higher than that of the control (i.e., odds ratios (ORs) = 2.2 and 2.1 for the case-defined odds ratio vs. OR = 0.8 for the control-defined odds ratio). The case- and control-defined odds ratios were different from each other beyond that expected by chance for sunbathing more as an adult. Having more moles was a significant risk factor for melanoma based on all comparative odds ratios regardless of source. Although not significantly different from each other, the case-defined odds ratio was over twice as high as the control-defined odds ratio (OR = 5.7 vs. OR = 2.6). Similarly, a greater than twofold difference was seen for the odds ratios for freckling in childhood with the case-defined odds ratio significantly elevated (OR = 3.1) compared with the control-defined odds ratio (OR = 1.3).

Among factors not thought to confer the risk of melanoma, little difference was seen in the odds ratios for perspiring more, which also had high agreement between the twins. However, among those with poorer agreement, the case-defined odds ratio was significantly higher for having more acne and more warts, although no difference was seen for more body hair.

Comparison of comparative and quantitative questions
For the two variables in table 3, number of moles on the body and amount of freckling in childhood, quantitative and comparative responses were obtained. The jointly derived comparative odds ratio for the exposure, "who has more large moles on [their] body" (OR = 4.0; 95 percent confidence interval (CI): 2.1, 7.7), was similar to the quantitatively derived odds ratio (OR = 3.8; 95 percent CI: 2.0, 7.2), and both were between the control-defined odds ratio (OR = 2.6) and the case-defined odds ratio (OR = 5.7). Similarly, for freckling in childhood, the odds ratio using the case's comparative response was higher than the odds ratio reflecting the quantitative association between freckling in childhood and melanoma (OR = 3.1 vs. 2.8), while the control-defined and jointly derived comparative odds ratios were lower (OR = 1.3 and 1.4, respectively). The odds ratio using the control's comparative response for freckling was not raised beyond chance, but the odds ratio using the case's comparative response and the odds ratio using the quantitative response were.

Sun exposure site-specific variables
To determine if recall of sun exposure to a specific site on the body was more likely to be related to the site of the melanoma, we computed the ratio of odds ratios as follows: the numerator was the odds ratio for sun exposure to a specific site, and the denominator was the odds ratio for sun exposure to other sites. The ratio of these odds ratios for sunburn on the face/neck was significantly higher for cases whose melanoma occurred on the face/scalp/neck than for those cases whose melanoma occurred elsewhere, but only for childhood sunburn and not for adulthood sunburn (table 4). Otherwise, the relation between the site of sun exposure or coverage derived from comparative responses did not appear to be related to the anatomic site of the melanoma.


View this table:
[in this window]
[in a new window]
 
TABLE 4. Relative risks* for comparative responses for sunburn and exposure variables in relation to the site of melanoma, International Twin Study, 1980–1991

 
Respondents' opinions about the cause of their melanoma
Fifty-three cases (28.6 percent) did not offer an opinion about the cause of their melanoma. Of 132 cases who did, 85 (64.4 percent) said exposure to sunlight caused their melanoma, 40 (30.3 percent) said exposure to other ultraviolet sources caused their melanoma, and the remaining seven thought their melanoma was caused by genetic factors or the use of skin care products. The case- and control-derived odds ratios for sunbathing as an adult and as a child (table 2) appeared to show some recall bias as the case-derived odds ratios were higher than those of the controls. When stratifying the analysis on the case's view on the role of sunlight as a cause of melanoma, the case-derived odds ratio on sunbathing as a child was much higher among those who thought sunlight was a cause (OR = 8.0; 95 percent CI: 1.4, 14.5) than among those not expressing this view (OR = 0.8; 95 percent CI: 0.2, 3.0). Little difference was seen for sunbathing as an adult.


    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
In case-control studies of risk factors for melanoma, self-reported exposures are retrospectively gathered and are therefore subject to recall bias. This is especially true of exposures such as sunburn frequency and tanning ability, which are highly publicized risk factors for melanoma.

Although sunburn exposure has been reported to be free of recall bias (4Go), tanning ability appears to be recalled in a biased fashion (5Go). However, in previous attempts at estimating the effect of recall bias for melanoma risk factors, study design has been a limiting factor (6Go, 10Go), because the comparisons have usually been of exposure self-reports both occurring after the diagnosis of melanoma. This amounts to an investigation of the repeatability of recall-biased estimates, rather than an estimate of recall bias itself. In previous studies, the demonstration of limited recall bias in self-reported sunburn exposure (4Go) is far less compelling than the assessment of bias in self-reported tanning ability (5Go), because the latter involved the comparison of self-reported exposure before and after the diagnosis of melanoma.

The present study provided a unique opportunity to investigate recall bias because the reported (comparative) exposure of the case and control refers to the same event, a situation not ordinarily occurring in a case-control study. In addition, we have also obtained the same type of self-reported quantitative information usually obtained in other studies. This study is the largest reported to date that addresses bias in melanoma risk factors. Contrary to previous reports, we found little bias in the odds ratios related to sunburning more easily, but the report of sunbathing exposure, in both childhood and adulthood, appeared to be overestimated by cases. Ease of tanning did not appear to be overestimated by cases and, although a previous study demonstrated bias in self-reports of tanning, that study investigated response to tanning, not exposure resulting in tanning (5Go).

We have still not achieved the true assessment of recall bias in melanoma risk factors, which would require the self-assessment of exposures before and after the development of melanoma. In this study, we have compared two potentially biased estimates with each other. Whereas in previous studies the occurrence of recall bias could have eluded investigators because of the study design (6Go), the design of our study probably only underestimated recall bias. The controls, co-twins of the cases, who knew the cases had melanoma may also have overestimated the cases' exposure (or underestimated their own) when asked which twin had the greater exposure. If, for example, controls believed their co-twin's greater sun exposure in childhood caused the occurrence of melanoma in their co-twin, they might also recall the juxtaposition of their exposure in a biased fashion.

We found differences in the case- and control-derived comparative odds ratios for both mole prevalence and freckling (in childhood) with higher odds ratios from cases' reports. However, the joint comparative odds ratio, which eliminated pairs that disagreed on who had more moles and pairs in which one member was uncertain, closely approximated the quantitative odds ratio based on the self-reported number of moles. For freckling in childhood, the "truth" is more difficult to determine because the case-derived and quantitative odds ratios were similarly elevated with a threefold increased risk, whereas the control-derived and joint comparative odds ratios were only slightly (and nonsignificantly) elevated.

Mole prevalence is an established risk factor for melanoma, even when assessed objectively (11Go), and while it is not necessarily a widely reported risk factor for melanoma in lay circles, this was a well-educated population. In addition, biased recall of mole and freckling frequency may have occurred because of cases' intuition that a mole "turned into" melanoma. However, although 61 percent of cases in this study reported a normal-looking mole at the same anatomic location as their tumor before the diagnosis of melanoma, none reported that their melanoma was "caused" by the presence of a mole.

If cases recalled their sun exposures in a biased fashion related to their knowledge of subsequent risk for melanoma, we might expect them also to overestimate their sun exposure at the site of their melanoma, compared with exposure at other non-melanoma-affected anatomic sites. This certainly appeared to be the case for melanomas of the face/scalp/neck for the reporting of sunburn exposure in childhood.

There was a notable lack of agreement between the cases' and controls' reports of exposure to sunbathing in childhood and adulthood. There was some disagreement between cases and controls with respect to measures not related to melanoma, such as body hair coverage, acne coverage, and wart prevalence. For the latter two variables, the case-derived odds ratio was significantly higher than the control's. Perhaps this may reflect increased sensitivity on the case's part to any skin-related condition.

One drawback of previous studies is that they assume that the study population is well versed in melanoma risk factors and therefore likely to recall past exposures in a biased fashion. Although this may be true for the members of the Nurses' Health Study cohort (5Go), previous studies of recall bias have not examined the level of knowledge of melanoma risk factors in their population. More than half of the cases in this study attributed their melanoma to exposure to sunlight and may therefore have recalled their sun-related exposures in a biased fashion. Indeed, we found that, for sunbathing in childhood, cases who mentioned sunlight as a cause of melanoma were much more likely to report that they had greater exposure than cases who did not link sunlight to their melanoma.

In conclusion, we found evidence of bias in reports of sunbathing in childhood and adulthood and possibly of freckling in childhood. However, ease of tanning and burning appeared less subject to bias. Self-reports of sun exposure, and of other less publicized melanoma risk factors such as freckling and mole frequency, require careful scrutiny and assessment when used to elucidate the etiology of melanoma in the case-control setting. Perhaps the ideal way to estimate the effects of melanoma risk factors is from cohort studies that assess exposures before the development of melanoma.


    ACKNOWLEDGMENTS
 
This project was supported by grants CA32262, CA42581, and CA23927 from the US National Cancer Institute.

The authors acknowledge the assistance of Janice Schaefer in the design and maintenance of the International Twin Study database.


    NOTES
 
Correspondence to Dr. Myles Cockburn, USC/Norris Compre-hensive Cancer Center, USC Keck School of Medicine, 1441 Eastlake Ave., Los Angeles, CA 90033 (e-mail: cockburn{at}usc.edu).


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Elwood JM. Melanoma and sun exposure. Semin Oncol 1996;23:650–66.[ISI][Medline]
  2. Koh HK, Norton LA, Geller AC, et al. Evaluation of the American Academy of Dermatology's National Skin Cancer Early Detection and Screening Program. J Am Acad Dermatol 1996;34:971–8.[ISI][Medline]
  3. Westerdahl J, Anderson H, Olsson H, et al. Reproducibility of a self-administered questionnaire for assessment of melanoma risk. Int J Epidemiol 1996;25:245–51.[Abstract]
  4. Berwick M, Chen YT. Reliability of reported sunburn history in a case-control study of cutaneous malignant melanoma. Am J Epidemiol 1995;141:1033–7.[Abstract]
  5. Weinstock MA, Colditz GA, Willett WC, et al. Recall (report) bias and reliability in the retrospective assessment of melanoma risk. Am J Epidemiol 1991;133:240–5.[Abstract]
  6. Gefeller O, Brenner H. Re: "Reliability of reported sunburn history in a case-control study of cutaneous malignant melanoma." (Letter). Am J Epidemiol 1996;144:707–8.[ISI][Medline]
  7. Fleiss JL. Statistical methods for rates and proportions. 2nd ed. New York, NY: John Wiley & Sons, Inc, 1981.
  8. SAS Institute, Inc. SAS/STAT. Cary, NC: SAS Institute, Inc, 1990.
  9. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.[ISI][Medline]
  10. Fox BH. Re: "Reliability of reported sunburn history in a case-control study of cutaneous malignant melanoma." (Letter). Am J Epidemiol 1998;148:620–1.[ISI][Medline]
  11. Elwood JM, Whitehead SM, Davison J, et al. Malignant melanoma in England: risks associated with naevi, freckles, social class, hair colour, and sunburn. Int J Epidemiol 1990;19:801–10.[Abstract]
Received for publication April 11, 2000. Accepted for publication September 18, 2000.