1 Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD.
2 New Mexico Tumor Registry, University of New Mexico Health Sciences Center, Albuquerque, NM.
3 Medical Service, Department of Veterans Affairs Medical Center, Albuquerque, NM.
4 Fred Hutchinson Cancer Research Center, Seattle, WA.
5 Department of Preventive Medicine, University of Southern California Keck School of Medicine, Los Angeles, CA.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() |
---|
medical records; prostatic neoplasms; questionnaires; recall; therapeutics
Abbreviations: PCOS; Prostate Cancer Outcomes Study; SEER; Surveillance; Epidemiology; End Results
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() |
---|
In 1994, the National Cancer Institute (Bethesda, Maryland) initiated the Prostate Cancer Outcomes Study (PCOS) to investigate the patterns of cancer care and the effects of initial treatments on health-related quality-of-life outcomes in a large population-based cohort of newly diagnosed prostate cancer patients. For the PCOS, information about initial treatments was collected primarily from medical records, including both inpatient and outpatient sources. Treatment information was also collected in a self-administered patient survey designed primarily to ascertain health-related quality of life. In this study, we compare surgery, radiation, and hormonal therapies self-reported by men newly diagnosed with prostate cancer with treatment information obtained from medical records.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() |
---|
Among 11,137 eligible cases, 5,672 were randomly sampled from the six registries according to defined age and race/ethnicity strata. Of the sampled cases, 4,736 (83.5 percent) were contacted and were invited to participate, and 3,196 (56.3 percent of sampled cases and 67.5 percent of invited participants) returned a 6-month survey. Eligible sampled patients were contacted by mail (90.2 percent) or telephone/in person (9.8 percent) 6 months after the diagnosis date and were asked to complete a self-administered questionnaire and provide consent for access to medical records from all providers of cancer care (both institutions and specific physicians). Medical records were abstracted for 3,173 (99.3 percent) of the participants who finished the 6-month survey. This study included all 3,196 patients who completed the 6-month survey. For 23 patients whose medical records were not abstracted for the PCOS, we used data on initial treatment routinely abstracted from medical records by the SEER registries.
Variables of initial treatment
The 6-month PCOS patient survey collected information on demographics, treatment of prostate cancer, health-related quality of life, and quality-of-life outcomes. Participants were asked whether they had initially received a radical prostatectomy, radiation, orchiectomy, hormone shots, or hormone pills for prostate cancer (table 1). Because chemotherapy is rarely used for initial treatment of prostate cancer (it is used typically to treat hormone-refractory disease, which tends to occur in a fraction of patients after initial hormonal therapy fails), the 6-month PCOS survey did not ask about chemotherapy. Response options for each question were "yes" and "no." For those who did not give an answer, their responses were categorized as "unknown."
|
Statistical methods
Kappa statistics were used to obtain chance-corrected agreement when neither self-report nor medical record could be assumed to be the "gold standard." In general, values of kappa greater than 0.8 represent excellent agreement beyond chance, values between 0.61 and 0.8 substantial agreement, and values higher than 0.4 but 0.60 or lower moderate agreement (2).
Depending on the method of classifying unknown answers, we calculated three types of kappa statistics. Kappa1 used "yes," "no," and "unknown." Kappa2 considered only "yes" and "no" by treating "unknown" from medical record abstraction as "no" and excluding participants who did not answer the survey item. Kappa3 also considered only "yes" and "no" but counted both "unknown" from medical record abstraction and the survey items as "no." Sensitivity, specificity, positive predictive value, and negative predictive value were also estimated (based on categorical levels of "yes," "no," and "unknown") by assuming the medical record to be the gold standard.
Adjusted agreements were produced from the logistic regression models (3) to investigate the association between various patient characteristics and agreement between self-reported and medical record information. The dependent variable for these analyses was agreement between survey self-reports and medical record abstracts based on categorical levels of "yes," "no," and "unknown." For example, the dependent (indicator) variable was coded 0 (disagreement) if self-report was "unknown" and the medical record indicated "no" treatment. The independent variables included age (<60, 6069, 7079,
80 years), race (Black, White, Hispanic), education (less than high school, some college, advanced), income (
$10,000, $10,001$20,000, $20,001$30,000, $30,001$50,000,
$50,001, unknown), and registry area. All analyses were performed by using SUDAAN statistical software (4
) except when we calculated adjusted agreement, for which we used a customized computer program written in SAS (SAS Institute, Inc., Cary, North Carolina). Horvitz-Thompson sampling weights, which are the reciprocal of sampling probabilities, were used to take into account the study design. The sum of the weights, 6,031, was about double the number of subjects in the analysis, 3,196 (table 2).
|
On the basis of medical records, 1,582 patients received prostatectomy, 810 radiation, 188 orchiectomy, 538 hormone shots, and 299 hormone pills (table 3). Kappa1 (which included "unknown" as a separate category) had the lowest values, while kappa2 (which classified "unknown" from medical records as "no" and excluded respondents who did not complete the survey item) had similar or slightly larger values than kappa3 (which excluded "unknown" responses). All kappa statistics for prostatectomy and radiation were greater than 0.8, indicating excellent agreement between self-reports and medical records. The chance-corrected agreement for hormone pills was moderate (kappa statistics ranged from 0.47 to 0.57), while for hormone shots it was substantial (from kappa1 = 0.69 to kappa2 = 0.78). Kappa1 (0.61) indicated somewhat moderate agreement between self-reports and medical records for orchiectomy, whereas kappa2 (0.82) and kappa3 (0.81) showed substantial agreement.
|
The positive predictive value and negative predictive value of self-reports corresponded to the sensitivity and specificity of the medical records, assuming the survey self-reports were the gold standard. The estimated positive predictive value was greater than 90 percent for prostatectomy, radiation, and orchiectomy and was 77 percent for hormone shots. However, the estimated positive predictive value was only 54 percent for hormone pills, meaning that among the patients who reported taking hormone pills, 46 percent did not have any such evidence in their abstracted medical records. The estimated negative predictive value was high (90 percent or more) for all treatments.
Logistic regression analysis was used to examine whether the respondent's age, race/ethnicity, educational level, annual family income, and cancer registry confounded the likelihood of agreement about initial treatment. Table 4 displays information on adjusted agreement calculated from the regression models for the covariates and the p values from testing for the overall effect of each covariate by using the Wald test statistic. The overall test (with more than one degree of freedom) was not a trend test (with one degree of freedom), because we did not assume that the effect of a covariate is linear. Because income was not statistically significant (p > 0.05) for any of these treatments, it is not shown in this table. Cancer registry was also excluded, since its effect was not of particular interest in its own right but should be adjusted for as a study design variable. Age, race, and educational level did not affect agreement on radiation treatment. Age was statistically significantly associated with agreement for reporting prostatectomy, hormone shots, and hormone pills, with better agreement for younger compared with older patients. Race/ethnicity also significantly affected the agreement for prostatectomy, orchiectomy, and hormone pills, with the highest agreement for White patients followed by Hispanic patients. For orchiectomy and hormone shots, self-reports from patients with a higher educational level versus less than a high school education showed significantly better agreement with the medical record.
|
When a treatment was identified from medical records but not reported by patients, the disagreement might have been attributable to recall error. Inaccuracy in recalling medical history and medication history has been documented by test-retest evaluation (14). Results from test-retest evaluation indicate that a patient is more likely to recall the presence of a disease requiring a surgical or intensive pathologic/laboratory diagnostic procedure than a disease without such a procedure. We also observed that the extent of inaccuracy varied by treatment; that is, patients were more likely to recall a treatment that involved a surgical procedure but less likely to recall one that involved only taking pills. Not surprisingly, we found that age can affect recall. For example, agreement between self-report and medical records regarding hormone pills was less likely for men aged 80 years or more, compared with younger men, and the adjusted agreement was 63 percent versus 80 percent or more, respectively.
The disagreement between medical records and surveys might also have reflected biased self-reports or unclear wording of questionnaire items. The estimated 74 percent sensitivity of the self-reports on orchiectomy indicates that 26 percent of men who had the procedure did not respond positively to the question about that procedure. Because it is unlikely that men would forget an orchiectomy received within the past 6 months, some of the underreporting might have been due to the respondents' reluctance to acknowledge a potentially embarrassing treatment that negatively affected their body image. Another possibility is that the wording of the question may have confused some men. The question specifically asked about testicles, which may have been an unfamiliar term for some participants. For orchiectomy, educational level was a statistically significant predictor of agreement in our regression model: for self-reports from men with less than a high school education, adjusted agreement with medical records was lower. Therefore, future studies should design questions appropriate for subjects with lower educational levels.
Poor agreement could have arisen from incomplete medical records abstraction. Although the PCOS attempted to abstract records from all treating physicians, we might have missed some, particularly the office-based physicians. This omission could certainly account for much of the poor agreement on hormone pills, which are prescribed almost exclusively in physicians' offices. Our study suggests that medical record review may not necessarily be the gold standard for hormone pills and shots, unless investigators are certain that all physicians treating the patient were contacted and consented to record reviews. In our study, more than 40 percent of the men who reported taking hormone pills had no such evidence in their medical record abstracts (estimated positive predictive value given in table 3). Men might also have inaccurately reported hormone pill use because they confused hormone pills with other medications. However, a Swedish prospective cohort study of more than 16,000 women aged 4573 years confirmed the validity of self-reporting current use of hormone therapy (estrogens, progestogens, or their combination) when compared with a 7-day personal diary (15). Although the Swedish study might not be generalizable to prostate cancer, patient surveys may be more accurate than medical record abstracts regarding self-administered oral medications, especially when investigators cannot be certain they have accessed all relevant providers' records.
A fifth reason for disagreement could have been poor patient compliance with prescription regimens. For example, patients might not have taken their hormone pills or not even filled their prescriptions. This noncompliance might have led to lower sensitivity when the medical records were considered the gold standard. Poor patient compliance might partly explain why more than 30 percent of men did not acknowledge taking hormone pills despite medical records indicating that they had been given a prescription (estimated sensitivity shown in table 3). This finding suggests that patients should be asked whether hormone pills were prescribed for prostate cancer in addition to asking the current question, "Have you taken hormone pills for prostate cancer?"
Our results showed that agreement between self-reported and medical-record-based initial treatment (except for radiation) was lower for Hispanics and Blacks than for Whites after we controlled for tumor registry, income, age, and educational level. The adjusted agreement was also poorer for older men than for younger men.
We also performed logistic regression analyses that included indicator variables for clinical disease stage. The adjusted proportions of agreement generally did not change much between levels of clinical stages, although agreement tended to worsen somewhat with increasing stage for each type of therapy. However, the adjusted agreement for hormone pills was much worse for patients with stage T3 or T4 disease. One possible reason for this finding is that men with more advanced disease may be more likely than men with early-stage disease to be prescribed hormone pills by medical oncologists to manage their condition, because hormonal therapy is usually given alone for advanced disease but as adjuvant therapy to other local treatments (surgery or radiotherapy) for early-stage disease. In the PCOS, abstraction of records from specialists was not as comprehensive as abstraction from urologist or radiation oncologist records.
One limitation of our study is that we did not obtain the date of surgery, radiation, or orchiectomy on the patient survey, although this information was collected in medical record abstracts. This variable may be important for some quality-of-life studies. It is conceivable that agreement on timing of treatment will not be as high as agreement on treatment itself. Also, because PCOS nonresponders were slightly older and from areas with lower levels of educational attainment (1), the missing responses might have led us to slightly overestimate agreement between self-report and medical record, except for radiation therapy, for which agreement was not associated with age and education.
In summary, this study provides the first known population-based comparison of self-report with medical record abstractions regarding treatment of prostate cancer, and the results are possibly generalizable to other diseases and treatments. These results can serve as a useful guide to outcomes researchers contemplating using surveys instead of medical record abstraction to ascertain treatments.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() |
---|