Department of Medical Oncology and Hematology, Princess Margaret Hospital and University of Toronto, Ontario, Canada
*Correspondence to: Dr I. F. Tannock, Department of Medical Oncology and Hematology, Princess Margaret Hospital and University of Toronto, 610 University Avenue, Toronto, Ontario, Canada M5G 2M9. Tel: +1-416-946-2245; Fax: +1-416-946-2082;Email: ian.tannock{at}uhn.on.ca
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Key words: cancer care, evidence-based medicine, quality of treatment
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Is the right treatment being given? |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Judging the quality of evidence
The relevance of a clinical trial to a given treatment decision depends not only on the type of trial (as classified in Table 1), but also on its quality. Judgement of quality can be separated into internal validity (how the trial was designed, executed and analysed) and external validity (external factors, such as its concordance with the results of related studies).
Internal validity
Some of the factors that are important in determining the internal validity of a clinical trial are listed in Table 2. First and foremost, the research question that is being asked in a pragmatic phase III study should be relevant to clinical decision-making. It is equally important that the outcome measures reflect benefit to patients and that the authors define explicitly the primary end point(s).
|
Common end points in cancer trials are tumour response, survival and (increasingly) a measure of quality of life (QOL). While tumour response may be a valid measure for evaluating the biological effect of a treatment, patients may have a significant reduction in tumour size without clinical benefit. Tumour response is also dependent on patient selection (patients with higher performance status are more likely to respond) and on the criteria used to measure response. Response to a given drug or drug combination may vary markedly from one patient to another, even if they have the same tumour type and stage of disease.
Survival is an appropriate measure of outcome, but there are relatively few treatments that have improved survival in adult patients with metastatic disease from common solid tumours [6]. In view of this, a single trial claiming improved survival must be viewed sceptically as it is quite likely to be a false-positive result, especially if the P value is borderline. The factors that contribute to false-positive trials include publication bias in favour of positive trials (see below), the use of multiple significance tests in the analysis of the data (at least one may be positive by chance), and a low probability that a new treatment will be superior. It was shown by Parmar that if the true prevalence of clinical trials that compare strategies where there is a meaningful difference in survival is
10% (an arbitrary but not unreasonable estimate), and one designs an RCT with
=0.05 and with 80% power to detect a positive result, then about one trial in every three reported as positive will actually be a false-positive [7
]. For these reasons, any improvement in duration of survival needs to be verified in a second trial.
Improvement in symptom control and/or quality of survival is a realistic and achievable goal for clinical trials. QOL is particularly important where the aim of treatment is palliation, and the expectation of survival benefit is low. Realistically, this is the situation for almost all trials for patients with metastatic cancer. It is also of prime importance where the trial is examining a therapy aimed at prevention or control of symptoms (e.g. antiemetic agents for chemotherapy). A measure of QOL should also be used when comparing treatments that provide similar expectations of survival, as the focus should then be on comparison of the side-effects between the two treatments and their effects on QOL [8, 9
].
Many other aspects of trial design are important, and several authors have suggested methods for evaluating the quality of a clinical trial [1014
]. The sample size should be adequate to provide statistical power to detect a difference in the primary end point, if such a difference really exists, and to exclude a difference if it does not exist. Randomisation must be performed appropriately and statistical analysis executed in a thorough and rigorous manner. In particular, the investigators should refrain from making conclusions based on unplanned subgroup analyses.
External validity
The reported results of a trial are more likely to be valid if they are consistent with clinical experience and with the results of related clinical trials. As discussed above, if true-positive trials are rare then false-positive trials will be common. If possible, positive results should be confirmed in a second trial before changes to standards of care are made.
Patients in clinical trials are often highly selected, with good performance status and near normal blood parameters, and so are frequently not representative of the general cancer patient population. Therefore, benefits seen in patients recruited to clinical trials are not necessarily generalisable to a less selected sample of patients with the same tumour type and stage.
Publication bias can have a major influence on the evidence that is available from published studies. Even large RCTs are subject to publication bias. In a further study of the abstracts describing the 510 RCTs presented at an ASCO meeting, 26% remained unpublished 5 years later, with a significantly higher proportion of negative trials remaining unpublished (81% versus 68%; P<0.001) (see Figure 1) [15
]. An example of the bias that this creates is that five large RCTs evaluating the use of interferon-
(IFN-
) in patients with renal cell cancer have been presented at ASCO in recent years. Two of these RCTs (n=160 and 350) showed a survival benefit for patients with metastatic disease and were published 13 years post-ASCO [16
, 17
]. However, three RCTs (n=247, 283 and 270) evaluating the use of adjuvant IFN-
either showed no difference in survival or suggested a trend towards harm [18
20
]. Two of these were published 2 and 7 years after presentation [18
, 19
], respectively, and the third study remains unpublished after 12 years [20
].
|
To be successful guidelines need to be based on evidence, disseminated to all affected health professionals for critique, and implementation needs to include direct feedback on individual performance or general feedback on system performance. Finally, there needs to be accountability for performance [21, 22
].
Several organisations have conducted audits to determine whether guidelines promote adherence to treatment recommendations and greater use of evidence-based care. For example, a study by Ray-Coquard et al. [23] reported on the impact of clinical practice guidelines in a regional cancer network in France. Medical records of patients with localized breast and colon cancer were examined from 1994 and 1996 (before and after implementation of guidelines in the cancer network) in four network hospitals, and in three hospitals that were in a region that did not have clinical practice guidelines. Medical decisions were analysed to assess compliance with clinical practice guidelines and to determine use of evidence from clinical trials. After introduction of the guidelines, hospitals in the network had a substantially higher number of decisions made according to guidelines and/or based on medical evidence, whereas the control hospitals did not (Table 3
) [23
].
|
![]() |
Is cancer treatment being done well? |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Another way to improve quality of care may be to recruit patients to clinical trials. Several studies have suggested that patients treated in clinical trials have a better outcome than patients who receive similar treatment but who are not in a clinical trial [3840
]. Studies have shown that patients with non-small-cell lung cancer, breast cancer and myeloma treated in trials had better survival than patients treated off study, even though they received similar treatments. The reasons for this benefit from inclusion in a clinical trial are unknown and a recent review suggests that it might be due to the failure to correct for confounding factors [41
]: even after matching patients in a trial with others, there may be a bias for patients with better prognostic factors (e.g. performance status) to be recruited to trials (i.e. selection bias). Other possible benefits for patients on trials include better management of their disease, as they tend to have more frequent evaluation with potentially earlier detection of problems and better management of side-effects [38
40
]. They are also more likely to maintain the scheduled dose and frequency of treatment. It is of great concern that well intentioned individuals in administrative or political positions are making it more difficult to recruit patients into trials, based on very rare incidents where clinical research has led to harm. These efforts may lead to greater harm, both immediate and long-term, if fewer patients are recruited to well designed clinical trials. The outcome of patients who are denied access to a clinical trial may be poorer, and future patients may be adversely affected because of failure to generate new evidence that is important for optimising cancer treatment.
![]() |
Is the patient being treated in addition to the disease? |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Improving QOL and symptom control (e.g. pain, nausea) are important goals of cancer treatment, and important end points of clinical trials. The evidence from clinical trials must also be disseminated to as many patients as possible. Some centres have demonstrated how improvement in pain control can be achieved using institution-based programmes. For example, an education programme, the Zero Acceptance of Pain (ZAP) project, was implemented by Fortner et al. to educate clinical staff at the University of Memphis [43]. Experienced oncology nurses acted as ZAP coordinators in each clinic. Components included pocket pain management cards, self-report forms and patient pain journals. After participating in the ZAP programme, patients reported a decrease in their ratings for the worst pain in the last 24 h from 4.1 to 3.3 (P<0.05) and in the level to which pain interfered with activities (from 3.7 to 2.9; P<0.05), although the overall pain severity rating was not statistically significantly different [43
]. de Wit et al. also evaluated education of cancer patients with chronic pain. All patients received regular pain treatment and were then randomised to a pain education programme or no further intervention. Pain was significantly improved in the intervention group at 2 and 4 weeks, although the difference was not significant at 8 weeks [44
] (see Figure 2
).
|
Effective communication is an essential component of good cancer care. Several studies indicate that many patients leave consultations unclear about their diagnosis and prognosis, the management plan and the intent of treatment [51]. Not only do patients report dissatisfaction with the level of communication with their health professionals, but oncologists acknowledge that their own training is insufficient in this area, and this adds to their own stress, lack of job satisfaction and burnout [51
, 52
]. Studies have shown that communication skills can be taught and are not something you either have or you don't [53
, 54
]. An RCT was performed that compared oncologists before and after training in communication. Doctors who had undertaken this training showed a higher use of open-ended questions, greater empathy, gave more appropriate responses to patient's cues and used more psychosocial probing than they had before the course, or compared with a control group of doctors [53
, 55
]; all of these improvements were significant.
![]() |
Conclusion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Notes |
---|
Received for publication February 26, 2004. Revision received March 5, 2004. Accepted for publication March 8, 2004.
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2. Feinstein AR. Fraud, distortion, delusion, and consensus: the problems of human and natural deception in epidemiologic science. Am J Med 1988; 84: 475478.[ISI][Medline]
3. Krzyzanowska MK, Pintilie M, Brezden-Masley C et al. The quality of abstracts describing randomized trials in the Proceedings of ASCO meetings; guidelines for improved reporting. J Clin Oncol 2004; In press.
4. Tannock IF. False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons. J Natl Cancer Inst 1996; 88: 206207.
5. Mills JL. Data torturing. N Engl J Med 1993; 329: 11961199.
6. Chlebowski RT, Lillington LM. A decade of breast cancer clinical investigation: results as reported in the Program/Proceedings of the American Society of Clinical Oncology. J Clin Oncol 1994; 12: 17891795.[Abstract]
7. Parmar MK, Ungerleider RS, Simon R. Assessing whether to perform a confirmatory randomized clinical trial. J Natl Cancer Inst 1996; 88: 16451651.
8. Stephens RJ, Hopwood P, Girling DJ. Defining and analysing symptom palliation in cancer clinical trials: a deceptively difficult exercise. Br J Cancer 1999; 79: 538544.[CrossRef][ISI][Medline]
9. Fayers PM, Hopwood P, Harvey A et al. Quality of life assessment in clinical trialsguidelines and a checklist for protocol writers: the U.K. Medical Research Council experience. MRC Cancer Trials Office. Eur J Cancer 1997; 33: 2028.[CrossRef][Medline]
10. Hayward RS, Wilson MC, Tunis SR et al. Users' guides to the medical literature VIII. How to use clinical practice guidelines. A. Are the recommendations valid? The Evidence-Based Medicine Working Group. JAMA 1995; 274: 570574.[CrossRef][ISI][Medline]
11. Girling DJ. Clinical Trials in Cancer: Principles and Practice. New York: Oxford University Press 2003.
12. Begg C, Cho M, Eastwood S et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276: 637639.[CrossRef][ISI][Medline]
13. Altman DG, Schulz KF, Moher D et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001; 134: 663694.
14. Huwiler-Muntener K, Juni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic quality. JAMA 2002; 287: 28012804.
15. Krzyzanowska MK, Pintilie M, Tannock IF. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 2003; 290: 495501.
16. Medical Research Council Renal Cancer Collaborators. Interferon-alpha and survival in metastatic renal carcinoma: early results of a randomised controlled trial. Lancet 1999; 353: 1417.[CrossRef][ISI][Medline]
17. Pyrhonen S, Salminen E, Ruutu M et al. Prospective randomized trial of interferon alfa-2a plus vinblastine versus vinblastine alone in patients with advanced renal cell cancer. J Clin Oncol 1999; 17: 28592867.
18. Pizzocaro G, Piva L, Colavita M et al. Interferon adjuvant to radical nephrectomy in Robson stages II and III renal cell carcinoma: a multicentric randomized study. J Clin Oncol 2001; 19: 425431.
19. Messing EM, Manola J, Wilding G et al. Phase III study of interferon alfa-NL as adjuvant treatment for resectable renal cell carcinoma: an Eastern Cooperative Oncology Group/Intergroup trial. J Clin Oncol 2003; 21: 12141222.
20. Porzsolt F. Adjuvant therapy of renal cell cancer (RCC) with interferon alfa-2A. Proc Am Soc Clin Oncol 1992; 11: 202 (Abstr 622).
21. Smith TJ, Hillner BE. Ensuring quality cancer care by the use of clinical practice guidelines and critical pathways. J Clin Oncol 2001; 19: 28862897.
22. Browman GP, Levine MN, Mohide EA et al. The practice guidelines development cycle: a conceptual tool for practice guidelines development and implementation. J Clin Oncol 1995; 13: 502512.[Abstract]
23. Ray-Coquard I, Philip T, de Laroche G et al. A controlled before-after study: impact of a clinical guidelines programme and regional cancer network organization on medical practice. Br J Cancer 2002; 86: 313321.[CrossRef][ISI][Medline]
24. Hillner BE, Smith TJ, Desch CE. Hospital and physician volume or specialization and outcomes in cancer treatment: importance in quality of cancer care. J Clin Oncol 2000; 18: 23272340.
25. Glasgow RE, Mulvihill SJ. Hospital volume influences outcome in patients undergoing pancreatic resection for cancer. West J Med 1996; 165: 294300.[ISI][Medline]
26. Lieberman MD, Kilburn H, Lindsey M, Brennan MF. Relation of perioperative deaths to hospital volume among patients undergoing pancreatic resection for malignancy. Ann Surg 1995; 222: 638645.[ISI][Medline]
27. Begg CB, Cramer LD, Hoskins WJ, Brennan MF. Impact of hospital volume on operative mortality for major cancer surgery. JAMA 1998; 280: 17471751.
28. Hodgson DC, Zhang W, Zaslavsky AM et al. Relation of hospital volume to colostomy rates and survival for patients with rectal cancer. J Natl Cancer Inst 2003; 95: 708716.
29. Davis S, Dahlberg S, Myers MH et al. Hodgkin's disease in the United States: a comparison of patient characteristics and survival in the Centralized Cancer Patient Data System and the Surveillance. Epidemiology, and End Results Program. J Natl Cancer Inst 1987; 78: 471478.[ISI][Medline]
30. Feuer EJ, Frey CM, Brawley OW et al. After a treatment breakthrough: a comparison of trial and population-based data for advanced testicular cancer. J Clin Oncol 1994; 12: 368377.[Abstract]
31. Horowitz MM, Przepiorka D, Champlin RE et al. Should HLA-identical sibling bone marrow transplants for leukemia be restricted to large centers? Blood 1992; 79: 27712774.[Abstract]
32. Junor EJ, Hole DJ, Gillis CR. Management of ovarian cancer: referral to a multidisciplinary team matters. Br J Cancer 1994; 70: 363370.[ISI][Medline]
33. Twelves CJ, Thomson CS, Gould A, Dewar JA. Variation in the survival of women with breast cancer in Scotland. The Scottish Breast Cancer Focus Group and The Scottish Cancer Therapy Network. Br J Cancer 1998; 78: 566571.[ISI][Medline]
34. Campbell NC, Elliott AM, Sharp L et al. Rural factors and survival from cancer: analysis of Scottish cancer registrations. Br J Cancer 2000; 82: 18631866.[CrossRef][ISI][Medline]
35. Harding MJ, Paul J, Gillis CR, Kaye SB. Management of malignant teratoma: does referral to a specialist unit matter? Lancet 1993; 341: 9991002.[CrossRef][ISI][Medline]
36. Collette L, Sylvester RJ, Stenning SP et al. Impact of the treating institution on survival of patients with poor-prognosis metastatic nonseminoma. European Organization for Research and Treatment of Cancer Genito-Urinary Tract Cancer Collaborative Group and the Medical Research Council Testicular Cancer Working Party. J Natl Cancer Inst 1999; 91: 839846.
37. Aass N, Klepp O, Cavallin-Stahl E et al. Prognostic factors in unselected patients with nonseminomatous metastatic testicular cancer: a multicenter experience. J Clin Oncol 1991; 9: 818826.[Abstract]
38. Davis S, Wright PW, Schulman SF et al. Participants in prospective, randomized clinical trials for resected non-small cell lung cancer have improved survival compared with nonparticipants in such trials. Cancer 1985; 56: 17101718.[ISI][Medline]
39. Karjalainen S, Palva I. Do treatment protocols improve end results? A study of survival of patients with multiple myeloma in Finland. BMJ 1989; 299: 10691072.[ISI][Medline]
40. Mayers C, Panzarella T, Tannock IF. Analysis of the prognostic effects of inclusion in a clinical trial and of myelosuppression on survival after adjuvant chemotherapy for breast carcinoma. Cancer 2001; 91: 22462257.[CrossRef][ISI][Medline]
41. Peppercorn JM, Weeks JC, Cook EF, Joffe S. Comparison of outcomes in cancer patients treated within and outside clinical trials: conceptual framework and structured review. Lancet 2004; 363: 263270.[CrossRef][ISI][Medline]
42. Tannock IF. Treating the patient, not just the cancer. N Engl J Med 1987; 317: 15341535.[ISI][Medline]
43. Fortner BV, Okon TA, Ashley J et al. The Zero Acceptance of Pain (ZAP) Quality Improvement Project: evaluation of pain severity, pain interference, global quality of life, and pain-related costs. J Pain Symptom Manage 2003; 25: 334343.[CrossRef][ISI][Medline]
44. de Wit R, van Dam F, Loonstra S et al. Improving the quality of pain treatment by a tailored pain education programme for cancer patients in chronic pain. Eur J Pain 2001; 5: 241256.[CrossRef][Medline]
45. Stone P, Richardson A, Ream E et al. Cancer-related fatigue: inevitable, unimportant and untreatable? Results of a multi-centre patient survey. Cancer Fatigue Forum. Ann Oncol 2000; 11: 971975.[Abstract]
46. Manzullo EF, Escalante CP. Research into fatigue. Hematol Oncol Clin North Am 2002; 16: 619628.[ISI][Medline]
47. Dimeo FC. Effects of exercise on cancer-related fatigue. Cancer 2001; 92 (Suppl 6): 16891693.[CrossRef][ISI][Medline]
48. Dimeo FC, Stieglitz RD, Novelli-Fischer U et al. Effects of physical activity on the fatigue and psychologic status of cancer patients during chemotherapy. Cancer 1999; 85: 22732277.[CrossRef][ISI][Medline]
49. Schwartz AL. Daily fatigue patterns and effect of exercise in women with breast cancer. Cancer Pract 2000; 8: 1624.[CrossRef][ISI][Medline]
50. Bruera E, Driver L, Barnes EA et al. Patient-controlled methylphenidate for the management of fatigue in patients with advanced cancer: a preliminary report. J Clin Oncol 2003; 21: 44394443.
51. Fallowfield L, Jenkins V. Effective communication skills are the key to good cancer care. Eur J Cancer 1999; 35: 15921597.[CrossRef][ISI][Medline]
52. Ellis PM, Tattersall MH. How should doctors communicate the diagnosis of cancer to patients? Ann Med 1999; 31: 336341.[ISI][Medline]
53. Fallowfield L, Jenkins V, Farewell V et al. Efficacy of a Cancer Research UK communication skills training model for oncologists: a randomised controlled trial. Lancet 2002; 359: 650656.[CrossRef][ISI][Medline]
54. Baile WF, Lenzi R, Kudelka AP et al. Improving physicianpatient communication in cancer care: outcome of a workshop for oncologists. J Cancer Educ 1997; 12: 166173.[Medline]
55. Jenkins V, Fallowfield L. Can communication skills training alter physicians' beliefs and behavior in clinics? J Clin Oncol 2002; 20: 765769.