1 Epidemiology Unit, Queensland Cancer Fund, Brisbane, Queensland, Australia.
2 School of Population Health, University of Queensland, Queensland, Australia.
3 School of Public Health, Queensland University of Technology, Brisbane, Queensland, Australia.
4 National Cancer Control Initiative, Carlton, Victoria, Australia.
5 Health Information Centre, Queensland Health, Brisbane, Queensland, Australia.
6 Department of Community and Behavioral Health, College of Public Health, University of Iowa, Iowa City, IA.
7 Office of Economic and Statistical Research, Queensland Treasury, Brisbane, Queensland, Australia.
Received for publication October 22, 2003; accepted for publication December 23, 2003.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
mass screening; melanoma; mental recall; sensitivity and specificity; skin
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In order to add to the available evidence, accurate information on the prevalence of screening is needed for assessment of the impact of any population-based skin screening program (9, 10). Skin screening examinations for melanoma commonly do not result in a laboratory or pathology report and are often not documented in the medical record (7); thus, almost all information about skin screening prevalence is based on patients self-reports (7, 1114). There is virtually no information available on the accuracy of such reports, in contrast to self-reports of breast, colorectal, and cervical cancer screening, which have been successfully validated against laboratory or medical records. For example, Norman et al. (15) reported sensitivities of over 0.90 and specificities of over 0.80 for self-reported mammography screening compared with facility records among cases and controls. Using a national breast screening database, Barratt et al. (16) found that 91 percent of women reported their mammography date accurately to within 12 months. Baier et al. (17) assessed the validity of self-reported colorectal cancer screening against health maintenance organization records and found 96 percent sensitivity for fecal occult blood testing, 95 percent for sigmoidoscopy, and 96 percent for colonoscopy. Another study that compared recall of six early cancer detection procedures with health maintenance organization records (18) found greater accuracy of recall for procedures which resulted in a test report, such as mammography (concordance of 84 percent), Papanicolaou smears (78 percent), or fecal occult blood testing (79 percent), than for those which generated only a physicians note, such as clinical breast examination (70 percent).
The present study was conducted as part of the first phase of a randomized controlled trial of melanoma screening in which whole-body skin examinations were offered to residents of intervention communities. Within the trial, records were kept of all such skin screening examinations carried out over a 3-year period, providing us with a unique opportunity to verify participants self-reported skin screening history.
The aim of this study was to assess the validity of recall of whole-body clinical skin examination in comparison with medical records and to determine factors associated with accuracy of recall.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Study subjects
The validation study was conducted in three of the nine intervention communities. Subjects were drawn from two sources within these three communities. First, we selected all 3,878 residents aged 30 years or more listed in the SkinWatch clinic database who had had a skin examination at a SkinWatch clinic in the past 3 years. Of these persons, we excluded 1,720 people who had participated in a recent survey of skin screening and whose accuracy of recall in this second survey may therefore have been better than normal. Those excluded were comparable to participants with regard to age, sex, and melanoma risk factors. A further 586 persons were uncontactable (their telephone had been disconnected (n = 238), they had moved (n = 187), there was no answer (n = 137), or the telephone line was nonresidential (n = 24)), 112 were unavailable within the study time frame, and 30 were unable to speak English. Of the remaining 1,430 residents, 1,304 (91.2 percent) agreed to take part, and 126 (8.8 percent) refused participation.
The second group of subjects was drawn at random from a commercially available telephone list (Desktop Marketing Systems Pty. Ltd., Boronia, Victoria, Australia) containing virtually all residential telephone numbers in the geographic areas involved in the study. From this list, we excluded the telephone numbers of all persons known to have had a skin examination at a SkinWatch clinic and all those who had participated in a recent skin screening survey. Of 3,164 residences contacted, 1,769 contained a potentially eligible subject (i.e., someone aged 30 years or more and of the required gender (we used a quota sampling scheme to ensure equal numbers of men and women) who was able to speak English and was available within the study time frame), and 1,400 (79.1 percent) agreed to take part.
Thus, the final sample comprised 2,704 participants, including 1,304 known to have had a skin examination at a SkinWatch clinic in the past 3 years and 1,400 known not to have had such an examination (overall consent rate = 84.5 percent).
Data collection
All participants completed a structured computer-assisted telephone interview administered by professional telephone interviewers.
Self-reported clinical whole-body skin examination
Participants were randomly assigned to receive one of two versions of a question assessing whole-body skin examination in the past 3 years. The two groups were similar in terms of their sociodemographic composition. In the total sample of 2,704 participants, 1,598 people were asked, "In the past 3 years, has a doctor deliberately checked the skin on your whole body? Usually this would involve taking your clothes off at least down to your underwear (i.e., bra and underpants)." For assessment of the accuracy of reports related to the more recent past, the same question was repeated in this group but asking only about the past 12 months. The remaining sample of participants (n = 1,106) received a slightly shorter question: "During the past 3 years, has a doctor deliberately checked the skin on all or nearly all of your whole body?" All those who reported having undergone a whole-body skin examination were asked for the location and date of this examination (SkinWatch clinic, primary care physicians or dermatologists office, or other venue).
Sociodemographic characteristics, attitudes toward skin cancer, and skin cancer risk factors
Other information collected during the interview included gender; age (3039, 4049, 5059, 6069, or 70 years); educational level (primary school (7 years of schooling), some high school (10 years of schooling), completion of high school (12 years of schooling), trade certification (e.g., plumber, electrician), or university degree (
13 years of schooling); employment (full-time, part-time/casual, or not working); and yearly household income before taxes (in Australian dollars: <$20,000, $20,000$60,000, >$60,000, or unwilling to answer). Participants were asked how concerned they were about skin cancer (very concerned, somewhat concerned, or not concerned at all); how likely they thought they were to get skin cancer in the future (not at all likely, somewhat likely, or very likely); whether they had had a mole or skin spot removed in the past; whether they had ever been diagnosed with melanoma; and whether they had been diagnosed with any other sort of skin cancer. Self-reported history of melanoma was compared with cancer registry records.
Validation of self-reported skin screening
We compared all participants self-reports with the SkinWatch clinic records. Where possible, we also checked the private medical records of all participants who indicated that they had received a whole-body skin examination from a physician outside the SkinWatch clinics. For each participant who reported a skin examination that was subsequently confirmed, we calculated the interval between the date of the interview and the actual date of the skin examination as recorded in the medical record. With regard to checking of negative reports, we were able to confirm reports of nonattendance at SkinWatch clinics, but we did not attempt to check negative reports in private physicians medical records; that is, we did not check medical reports for patients who stated that they had not visited their own physician for a skin examination.
Of the 2,704 total participants, 563 stated that they had visited a physician outside the SkinWatch clinic for a skin examination in the past 3 years; 296 (52.6 percent) gave written consent for us to contact their physician for confirmation of this. Thirty-eight physicians were contacted by mail to participate in the present study, of whom 28 (73.7 percent) agreed. Copies of participants consent forms were mailed to the nominated physicians. Physicians were asked to indicate on an attached form whether there was a record of a whole-body skin examination and the exact date (or dates) of the examination, or to indicate as much if there was no record of such an examination. Responses were received from the physicians of 252 (85.1 percent) participants. The remaining 311 participants for whom we were not able to check private physicians records were excluded from the sample, generating a final sample for analysis of 2,393. The 311 excluded participants were similar to the rest of the sample in terms of age, sex, and education.
Data analysis
We assessed concordance as the proportion of total self-reports (either positive or negative) that were accurate in comparison with medical records. Cohens kappa statistic was calculated as a measure of agreement between self-reports and medical records, correcting for chance agreement (19). Additionally, we calculated the sensitivity (percentage of "positive" medical records accurately reported by participants) and specificity (percentage of "negative" medical records accurately reported by participants) of self-reports (table 1). The sensitivity, specificity, and concordance of self-reported whole-body skin examination during the past 3 years were compared between sociodemographic groups and according to attitudes toward skin cancer and skin cancer risk factors using chi-squared tests for trend (table 2). Multiple logistic regression analysis was used to examine factors associated with concordance of self-reported whole-body skin examination within the past 3 years. All analysis was undertaken using the SPSS statistical package (version 11.5; SPSS, Inc., Chicago, Illinois).
|
|
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Sensitivity, specificity, and concordance: whole-body skin examination in the past 3 years
Among the 2,393 participants, 1,366 reported having had a whole-body skin examination carried out by a primary care physician within the past 3 years (1,163 in SkinWatch clinics and an additional 203 at a physicians office outside the clinics), and 1,027 reported that they had not received a skin examination in the past 3 years. We confirmed 1,147 (98.6 percent) reports of skin examinations at SkinWatch clinics and 184 (90.6 percent) reports of skin examinations outside the clinics, for a total of 1,331 (97.4 percent) true-positive reports (table 2). The percentage of correct reports was similar for version 1 and version 2 of the question (97.7 percent and 97.1 percent, respectively). Among the 1,027 people who reported having had no skin examination in the past 3 years, 116 had actually received an examination at the SkinWatch clinics, giving us a total of 911 (88.7 percent) true-negative reports (assuming no overreporting of private skin examinations).
Overall concordance between self-reports and medical records was 93.7 percent. The sensitivity of self-reports was 92.0 percent overall, and sensitivity was similar for both versions of the question (92.1 percent and 91.8 percent). Specificity was 96.3 percent overall and was also similar for both versions of the question (96.5 percent and 96.0 percent).
We repeated this analysis, assuming the same amount of underreporting of private skin examinations (116 false-negative reports) as we had observed for SkinWatch clinic examinations, that is, a total of 232 false-negative reports. Under this assumption, sensitivity was reduced to 85.2 percent and specificity was reduced slightly to 95.8 percent, with overall concordance of 88.9 percent.
Sensitivity, specificity, and concordance: whole-body skin examination in the past 12 months
Participants recall of whole-body skin examination within the past 12 months was less accurate (74.3 percent) than recall for the past 3 years. The majority of errors were false-positive reports (n = 352), that is, overreporting of skin examinations within the past 12 months. In most cases (98 percent), the errors were due to "telescoping"underestimation of the amount of time elapsed since the last examination (table 1). There was no association of telescoping with gender or age. Similar degrees of telescoping could be seen with all levels of education, ranging from 73.4 percent for persons with a primary school education to 79.2 percent for those who had a trade certificate (p = 0.82). No associations with telescoping were seen for other sociodemographic factors or for skin cancer risk factors.
Factors associated with accuracy of recall of skin examination
Concordance, sensitivity, and specificity according to sociodemographic characteristics, attitudes toward skin cancer, and skin cancer risk factors are presented in table 1. Women (95.2 percent) were more likely than men (92.1 percent) to recall a whole-body skin examination correctly (p = 0.002). There was a significant age trend, with persons aged 70 years or more recording the lowest concordance (90.5 percent). No other sociodemographic factors, such as income, education, or employment, were associated with concordance. Ninety-six participants reported a personal history of melanoma. Concordance for participants with a self-reported history of melanoma was 90.6 percent, as compared with 93.9 percent for those reporting no history of melanomaa difference that was not significant. Of the 96 participants who reported a history of melanoma, 76 (79.2 percent) consented to access to cancer registry records, and history of melanoma was confirmed for 28 (36.8 percent) of these persons. There was virtually no difference in sensitivity of recall of whole-body skin examination between participants with and without a confirmed history of melanoma (92.6 percent and 92.0 percent). Participants who reported a history of nonmelanoma skin cancer, removal of a spot or mole, or high susceptibility to development of skin cancer were less concordant in their recall of skin examinations than other participants. There was no association between level of concordance and concern about skin cancer (table 1).
In multivariate modeling, after adjustment for all other variables, women were more likely than men (p for trend = 0.006) to report a skin examination accurately, while there was a tendency for older participants to recall a skin examination less accurately than younger participants (p for trend = 0.09). No other factors were significantly associated with accuracy of recall of skin examination.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We compared participants self-reports of whole-body clinical skin examinations conducted within the past 3 years with records of such examinations collected during a randomized trial of screening for melanoma. The concordance between self-reports and medical records was very high (93 percent), as was the sensitivity (92 percent) and specificity (96 percent) of self-reports, indicating that participants are able to recall this event accurately. Men and participants aged 70 years or older were less likely to recall skin examinations accurately. Increasing age was also reported to be inversely associated with recall of other screening procedures such as sigmoidoscopy, fecal occult blood testing, and Papanicolaou smear (17, 18). However, Newell et al. (22), in their study of cervical cancer and cholesterol screening, did not find any associations between accuracy of recall and sex, age, or education.
Based on the current results, the validity of self-reported skin screening history within both skin clinics and private physicians practices compares favorably with that of self-reports of mammography and other screening procedures. Concordance between self-reports and mammographic records ranges from 83 percent (18, 23) to 94 percent (16). Warnecke et al. (24) summarized the validity of self-reported Papanicolaou smears (12 studies; overall concordance = 65 percent), mammograms (eight studies; overall concordance = 86 percent), and clinical breast examinations (three studies; overall concordance = 70 percent). Gordon et al. (18) reported sensitivities for self-reports of 98 percent for mammograms, 97 percent for Papanicolaou smears, and 91 percent for fecal occult blood testing, while Norman et al. (15) reported sensitivities of 93 percent for cases and 92 percent for controls as part of a case-control study on the efficacy of screening mammography. Somewhat lower sensitivities (7682 percent) have been reported for prostate-specific antigen testing (25, 26) and digital rectal examination (78 percent) (18).
Within the current study, concordance was significantly lower when participants were asked about the past 12 months than when they were asked about the past 3 years; this was largely due to participants underestimating the amount of time elapsed since the skin examination had actually been performed. This phenomenon of a patients recalling a test or procedure as occurring more recently than it actually did is referred to as "telescoping" (18, 24, 27). McGovern et al. (28) found that up to one third of participants recalled having their mammograms more recently than they actually did. Additionally, Caplan et al. (23) found that approximately 28 percent of women in their study underestimated the time that had elapsed since their mammogram.
Some limitations of our study must be noted. Our sample was drawn from residents living within intervention communities of a randomized controlled trial of melanoma screening. Therefore, our participants may have been more aware of and more interested in skin cancer and skin screening than could be expected in other populations because of publicity associated with the trial. This could have resulted in somewhat better recall of skin examinations than could be expected in the general population, though we have no evidence of this. The recall of whole-body skin examinations that we observed is well within figures reported for other self-reported health behaviors (1618, 23).
The response rate within our study was high, with 85 percent of all eligible residents agreeing to participate. This compares favorably with other validation studies, where response rates varied between 33 percent for a study of mammogram screening involving women in a county health system (29) to 7085 percent for studies on the accuracy of recall of Papanicolaou smears, cholesterol screening, and other screening procedures (16, 18, 22, 23). Although we had complete records of all whole-body skin examinations performed at SkinWatch skin clinics, we could only check medical records for approximately 45 percent of whole-body skin examinations reported to have occurred outside the clinics, in private practices. The accuracy of reporting was similar for self-reported SkinWatch examinations and examinations conducted in private practices, and the sociodemographic characteristics of participants for whom we were not able to access medical records did not differ from those of other participants in any measurable way. We have no reason to believe that their accuracy or reporting would have differed from that of the remainder of the sample. Although we were able to confirm negative reports of skin examinations through the SkinWatch clinic records, we did not attempt to check negative reports of skin examinations by private physicians. Our analysis suggests that had we done this, our results are unlikely to have changed significantly.
Accuracy of recall depends to a significant extent on the precise wording of survey questions (30). Baier et al. (17) have noted the importance of accurate wording in surveys on colorectal cancer screening. Following the results of their pilot work, these authors reworded their questions on fecal occult blood testing and were able to increase the accuracy of recall from 62 percent to 96 percent in their final sample (17). Within the present study, concordance, sensitivity, and specificity were similar for the two versions of the question on skin screening, perhaps because of a number of key words common to both questions, such as "has a doctor deliberately checked...". The greater detail given in version 1 of the question ("...this would involve taking your clothes off at least down to your underwear") does not seem to have added to the accuracy of recall. On this basis, we would recommend the use of the second, shorter version in future studies.
Confirmation of a self-reported history of melanoma was low within the present study (37 percent), in contrast to a recent study of US teachers (31). That study compared self-reported cancer histories with cancer registry records and found 73 percent sensitivity for persons reporting a history of melanoma and 54 percent for other types of skin cancer. The most likely explanation for overreporting of melanoma in this Queensland population is confusion between melanoma and other, more common actinic neoplasms (solar keratoses and basal and squamous cell carcinoma) and confusion with benign nevi, which are highly prevalent in this sun-exposed population and are commonly excised to exclude melanoma. These sources of error have also been shown to result in substantial overreporting of family history of melanoma in Queensland (32).
After adjustment for other variables such as sex and age, neither a self-reported history of melanoma nor a confirmed history of melanoma appeared to influence accuracy of recall of whole-body skin examinations. This finding is encouraging for case-control studies of melanoma screening utilizing self-reported history of skin examinations, since we found no evidence of bias in the way cases or controls reported their screening history.
To our knowledge, this is the first study to have examined the validity of patient recall of whole-body clinical skin examination. The validity of self-reported whole-body clinical skin examination during the past 3 years was high and compares well with results observed in studies of other screening examinations. "Telescoping" was observed when participants were asked about undergoing a whole-body skin examination during the past 12 months; this suggests that future studies should restrict questioning to longer time intervals, such as the past 3 years. Overall, self-reports of whole-body skin examination seem to be a reliable source of data for epidemiologic studies, if the time frame is sufficiently wide. Because self-reporting will continue to be the most widely used source of data for evaluating the prevalence of screening behavior, attention must be given to accurate wording of questions in order to clarify sufficiently the behavior in question and the time frame of recall.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|