Consultant in Public Health Medicine. Bristol North Primary Care Trust, King Square House, King Square Bristol, BS2 8EE, UK. E-mail angela.raffle{at}bristolnorth-pct.nhs.uk
In 1991 George Knox published an unequivocal condemnation of misuses of case-control methods for evaluating screening.1 Noel Weiss countered some of Knoxs criticisms, arguing that carefully conducted and cautiously interpreted case-control studies may be the only option in circumstances where other evidence is unavailable.2 Whilst Weisss arguments are valid, Knoxs criticisms were justifiable given the poor scientific rigour of many publications that had claimed to elucidate the outcomes of screening. Faced with Hoffman et al.s paper3 it is hard not to have a sense of déjà vu.
Not only are there theoretical concerns. Experience with nationwide quality assured programmes shows that mortality reduction achieved by true life screening is less than predicted by randomized controlled trials (RCT), which in turn is less than the estimates from case-control comparisons. The breast cancer mortality reduction in women aged 5569 attributable to the England and Wales mammographic screening programme introduced in 1989, is estimated by 1998 based on cohort specific trends, as 7%.4 When the service was planned the predictions from RCT evidence were far more optimisticthat breast cancer deaths in the screened age band would be reduced by a third or more.5 Estimates from case-control studies of breast screening are higher still.6,7 The conclusion of the authors of the Health Technology Assessment report6 are that case control estimates of effectiveness should therefore be interpreted with extreme caution.
A case-control study of screening takes cases who suffer the adverse outcome that screening is intended to avert, and controls drawn from the population giving rise to the cases, then compares past participation in screening amongst both groups. If screening makes a difference, by virtue of positive tests correctly identifying those destined to become cases, coupled with intervention that reduces risk of the adverse outcome, then some potential cases will have become non-cases because of participation in screening and will have switched to the control group. The screening rate amongst controls is essentially the same as that in the total population, since the number of averted cases joining the controls is too small to make any impact on the overall participation rate.
The South African study3 found that 50% of 524 cases of invasive cervical cancer reported prior participation in cervical screening compared with 73% of 1540 controls. The authors interpret this as showing that even the limited screening service in the Western Cape, with no quality standards or assurance processes, leads to a reduction in cervical cancer incidence of 70%. This assumption that the association between being a control and reporting past screening is explained by the efficacy of screening alone is invalid. In Weisss paper2 about maximizing the reliability of case-control studies of screening he stresses the importance of validating screening histories from records. Hoffman et al.s study does not do this but instead depends totally on womens recollection. The unreliability of these data is betrayed by the implausibly high 73% reported participation amongst the controls, who the authors describe as disadvantaged women. It is highly likely that women tended to report to the interviewer what they perceived as the correct health-related behaviour. Achievement of 73% participation in cervical screening amongst disadvantaged groups in England has needed major publicity, multiple written and verbal invitations and reminders, highly accessible services, and incentive payments for providers linked to uptake rates. Weiss also stresses the importance of ascertaining all cases of advanced disease during a precisely specified time period. In the South African study there is no description of how date of diagnosis was defined, of whether cases that became advanced during the 6 months they were captured, and of whether fatal cases were included.
These two factors alone are sufficient to raise serious doubts about the validity of this study, but there is a third concern. The difficulty of ruling out healthy screenee bias has been repeatedly commented on.1,7,8 The odds ratio in a case-control comparison will be distorted if there is a relationship between an individuals risk of an adverse outcome, and their likelihood of participating in screening. The authors attempt to remove any potential bias by matching for decade of age, urban/rural residence, race, education, parity, age at first sexual activity, use of contraceptives, and cigarette smoking. This may reduce the bias but it cannot rule it out. This healthy-screenee effect bedevils all studies that incorporate comparisons between those who self-select to participate and those who self-select not to.
In England and Wales, despite nationwide provision of cervical screening from the 1960s, it was not until the late 1980s following re-launch of the programme that a measurable and substantial fall in incidence and in cohort-specific mortality occurred. The relaunch introduced strict quality standards, lowering of thresholds for classifying and treating abnormalities, and meticulous follow-up and failsafe to ensure all identified abnormals received treatment.9,10 This adds further weight to the conclusion that the difference between screening history recollected amongst cases and controls in the South African study is the result of weaknesses in the method and not because limited screening is uniquely effective in the Cape.
Two other factors cannot go unmentioned. First, as the UK National Screening Committee emphasizes, all screening programmes do harm, and some do good as well as harm. Poor quality screening brings harmful outcomesanxiety for abnormals, iatrogenic harm from invasive investigations and treatments, and anger for those who develop disease despite screeningwith little or no benefit. Second, screening is a relatively expensive form of preventive medicine because of the large numbers of tests and treatments needed. Data from Bristol, UK, show that 1000 women need to be screened for 35 years to prevent a death from cervical cancer.11 This needs substantial resources. In South Africa, with higher incidence, the number needed to screen to prevent a death may be less, but the positive outcome will only be achieved in a properly resourced and quality assured programme.
![]() |
Notes |
---|
![]() |
References |
---|
![]() ![]() |
---|
2 Weiss NS. Application of the case-control method in the evaluation of screening. Epidemiol Rev 1994;16:10208.[ISI][Medline]
3 Hoffman M, Cooper D, Carrara H et al. Limited Pap screening associated with reduced risk of cervical cancer in South Africa.Int J Epidemiol 2003;32:57377.
4 Blanks RG, Moss SM, McGahan CE, Quinn MJ, Babb PJ. Effect of NHS breast screening programme on mortality from breast cancer in England and Wales, 19908: comparison of observed with predicted mortality.BMJ2000;321:66569.
5 Forrest P (Chairman of Working Group). Breast Cancer Screening; Report to the Health Ministers of England, Wales, Scotland and Northern Ireland. London: HMSO, 1986.
6 MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess 2000;4:1154.[Medline]
7 Moss SM. Case-control studies of screening. Int J Epidemiol 1991; 20:16.[Abstract]
8 Friedman DR, Dubin N. Case-control evaluation of breast cancer screening efficacy. Am J Epidemiol 1991;133:97484.[Abstract]
9 Performance of the NHS Cervical Screening Programme in England. National Audit Office. London, HMSO, 1998.
10 Quinn M, Babb P, Brock A, Kirby L, Jones J. Cancer Trends in England and Wales. London: Office for National Statistics, 2001.
11 Raffle AE, Alden B, Quinn M, Babb PJ, Brett M. Outcomes of screening to prevent cancer: analysis of cumulative incidence of cervical abnormality and modelling of cases and deaths prevented. BMJ 2003;326:90104.