Usefulness of a dispensary-based case-control study for assessing morbidity impact of a treated net programme

S Abdullaa, JRM Armstrong Schellenberga,,b, O Mukasaa and C Lengelerb

a Ifakara Health Research and Development Centre (IHRDC), PO Box 53, Ifakara, Tanzania.
b Swiss Tropical Institute, PO Box, 4002 Basel, Switzerland.

Dr Salim Abdulla, IHRDC Dar-es Salaam Office, PO Box 78373, Dar es Salaam, Tanzania. E-mail: ihrdc{at}twiga.com

Abstract

Background Case-control studies have been proposed as an appropriate tool for health impact evaluation of insecticide-treated nets (ITN) programmes.

Methods A dispensary-based case-control study was carried out in one village in Tanzania. Each case of fever and parasitaemia in a child under 5 years was paired with one community and one dispensary control without fever and parasitaemia. Cases and controls were compared with regard to ITN ownership and other factors assessed by a questionnaire. A cross-sectional survey of factors associated with parasitaemia, including ITN use, was carried out during the study. Dispensary attendance rates of the study children were calculated using passive case detection data.

Results Cases and dispensary controls had higher dispensary attendance rates compared to community controls and children with nets attended more for most of the illness events. A comparison of cases and community controls showed a strong and statistically significant association between untreated net use and being a case (odds ratio [OR] = 2.1, 95% CI : 1.3–3.4). For those with ITN there was a smaller and weaker association between risk of being a case and ITN use (OR =1.4, 95% CI : 0.9–2.2). Comparison of cases and dispensary controls showed no association between untreated or treated nets and the risk of being a case (for treated nets OR = 0.9, 95% CI : 0.5–1.4 and for untreated nets OR = 1.2, 95% CI : 0.7–2.0). These results are contrary to those from the cross-sectional assessment, where children with ITN had a lower prevalence of parasitaemia than those with no nets (OR = 0.5, 95% CI : 0.3–0.9), and also contrary to other assessments of the health impact of ITN in this population.

Conclusion The positive association between mild malaria and net ownership is counter-intuitive and best explained by attendance bias, since children with nets attended more frequently for all curative and preventive services at the dispensary than those without nets. Dispensary-based case-control studies may not be appropriate for assessing impact of treated nets on clinical malaria, while cross-sectional surveys might represent an attractive alternative.

Intervention programmes for community health problems in developing countries have usually been implemented after their efficacy was demonstrated in randomized controlled field trials. Recently more emphasis has been put in assessing the performance of such interventions under programme conditions before wide-scale use.1 These evaluations aim to test whether the benefits of the interventions observed under trial conditions are retained in a programme setting. Positive results will encourage allocation of resources to sustain and expand the programmes to cover whole populations in endemic countries.2

Experience in the impact evaluation of health intervention programmes is limited.2 These Phase IV assessments are complicated by the absence of appropriate control populations and biased access to interventions and health services. Also, there is usually a limited opportunity for establishing an elaborate evaluation system that may be required to accurately assess exposure and outcome events in the target population.1,3 Therefore, inexpensive, simple and non-intrusive evaluation tools are required.

Case-control studies have been proposed as the most convenient and appropriate tools in the evaluation of insecticide-treated nets (ITN) programmes.2 This was further advocated after the first ITN programme evaluation in the Gambia.4 Case-control studies are attractive because they can be performed relatively cheaply and quickly after the initiation of the intervention.1 But they also face well-described problems of bias and confounding, as with all observational studies.5

The use of ‘passive case detection’ through clinics rather than through active surveillance simplifies a case-control study6 and may improve classification of disease status.7 However, this increases the difficulty of selecting appropriate controls.

Alternatively, cross-sectional surveys can be done. These assess prevalence and lack the time sequence between exposure and disease events if a single survey is carried out.7 However, repeated cross-sectional studies have been used to assess impact of ITN programmes.8–11

Here we describe a case-control study relying on passive case detection in an area of Tanzania where other studies have reported a positive impact of an ITN social marketing programme on child health and survival.11,12 The programme utilized marketing techniques to promote, distribute and sell pre-treated bed nets and insecticide kits for re-treatment.13

Methods

Study site and population
The study was conducted in Idete village (08°5’S ; 36°30’E), Kilombero District, Southeast Tanzania. The village is one of 18 in the Kilombero Valley within a demographic surveillance system (DSS).13 The area has intense year-round malaria transmission.14 Prompt diagnosis and treatment of clinical cases is the main control strategy and resistance to chloroquine, the first line antimalarial at the time of the study, was high.15 Children under 5 years of age living in all the hamlets (vitongoji) of Idete village were included in the study.

Case-control study
All children in the DSS database who were under the age of 5 on 1 February 1998 were assigned a study number and given a special card bearing their name, date of birth, the area of residence and DSS identifiers. This list was updated every 6 months to include newborns and children who had migrated into the village. A passive case detection system (PCD) was operated at the only local dispensary between February 1998 and August 1999 and all children under 5 years who attended the dispensary for any complaints were eligible. A standard form was filled out and a blood sample was taken on all children who had a history of fever in the previous 48 hours, or a presumptive diagnosis of malaria. Haemoglobin (Hb) was measured using the Hemocue® (HemoCue AB, Ängelholm, Sweden) system and thick and thin films were prepared for microscopy. Records were also kept for all children who were not sick and had attended for preventive reasons, such as growth monitoring. Children with a temperature of more than 37.4°C and any parasitaemia were classified as cases.

For each case, one age-matched dispensary control was chosen among the children who had come for growth monitoring or attended for being sick for a cause other than malaria, within two weeks of the attendance of the case. An additional community control was chosen from the DSS database matched for age and area of residence (hamlet). If a suitable control could not be found then a child from an adjacent hamlet was chosen. Both cases and controls were then visited at home, individual verbal consent was asked from the guardian and a questionnaire applied. A blood sample was taken from controls who had not given a blood sample at the dispensary in the two weeks prior to the day of the interview. Controls with temperature more than 37.4°C and any parasitaemia were excluded in the analysis. Cases and controls were not eligible for recruitment again as either cases or controls for a period of one month.

Cross-sectional survey
In June–August 1998, all children under 5 years of age on 1 February 1998 were visited at home and a questionnaire applied to assess their use of ITN and other risk factors for malaria infection and anaemia (using a similar approach to the case-control study). A blood sample (for malaria parasites and Hb estimation) was taken for children who had not given a blood sample at the dispensary in the two weeks prior to the day of the interview. The position of all houses with children under 5 and the dispensary were also determined using a portable Global Positioning System device (Garmin International, Kansas City, USA). The geo-referenced points were then used to calculate the distance from each child's house to the dispensary.

Dispensary attendance rates
The PCD data were linked to the DSS data to estimate dispensary attendance rates for different illnesses among study children. Attendance for malaria as well as for growth monitoring and other non-malaria related illnesses (including injuries, burns, fungal infestations, conjunctivitis, abscesses, etc.) was done. Mosquito net ownership was ascribed from the cross-sectional data.

Analysis
In the case-control assessment, cases of mild malaria were compared with controls primarily with regard to their exposure to ITN. Both unmatched and matched pairs multivariate analysis using logistic regression was carried out using Stata version 6 (Stata Corporation, Texas, USA). The risk or confounding factors considered included the number of people in the room where the child slept, sex of the child, age category, time to the nearest shop, access to a clean water source, tribe, if the mother or guardian would advise a neighbour to send a child sick with fever to a formal health facility, religious affiliation, nutritional status, vaccination status, antimalarial and antipyretic use, mother/guardian's literacy and the condition of the net in terms of the number of holes and its use in the previous month. Family income (quantiles of total family income), which was assessed by asking the average monthly income from various activities of the family, was also considered. Significance testing was done using the likelihood ratio test. For the cross-sectional assessment, the parasitaemia and anaemia were compared between users and non-users of ITN, after controlling for confounding using logistic regression.

Results

Case-control study
Idete village had a total of 881 children under 5 years in the DSS database on 1 February 1998. A total of 3389 visits of children under 5 were recorded at the dispensary between February 1998 and August 1999. A total of 587 mild malaria cases were identified which were individually matched with 424 dispensary and 555 community controls. For some cases we could not identify a suitable control from the database and not all the identified cases and controls could be traced at their homes because of either travelling or moving to farm houses that could not be located. Those available for interview were 461 cases (78.5%), 333 (78.5%) dispensary controls and 423 (76.2%) community controls. Seven community controls and eight dispensary controls had fever and parasitaemia and were excluded in the analysis. Forty cases, 45 community and 17 dispensary controls were excluded due to missing information on actual distance to the dispensary.

Interviewed children had an average age of 26 months (95% CI : 24–27) for cases, 26 months (95% CI : 25–27) for community and 25 months (95% CI : 23–26) for dispensary controls. The mean Hb was 9.2 g/dl (95% CI : 9.0–9.5) for cases, 10.2 (95% CI : 10.0–10.4) for community controls and 10.1 (95% CI : 9.9–10.4) for dispensary controls. The Hb difference between cases and either type of control reached statistical significance (Wilcoxon rank-sum tests: P < 0.001).

Important risk factors for becoming a case compared to community controls included having a net and living far away from the dispensary. In the unmatched analysis, those with untreated nets were about twice as likely to be cases as those without nets (OR = 2.1, 95% CI : 1.3–3.4). Those with ITN also appeared to be more likely to be cases (OR = 1.4, 95% CI : 0.9–2.2) but the relationship was not statistically significant (Table 1aGo). When comparison was made using dispensary controls, there was no apparent effect of untreated nets (OR = 1.2, 95% CI : 0.7–2.0) (Table 1bGo) and only a small effect of treated nets, which did not reach statistical significance (OR = 0.9, 95% CI : 0.5–1.4). Similar results were observed with matched analysis (data not shown).


View this table:
[in this window]
[in a new window]
 
Table 1a Risk factors for being a case of malaria at the Idete dispensary (unmatched analysis using community controls)
 

View this table:
[in this window]
[in a new window]
 
Table 1b Risk factors for being a case of malaria at the Idete dispensary (unmatched analysis using dispensary controls)
 
Distance to the dispensary was a significant predictor of being a case compared to both types of controls. Those living more than 3 km away were twice as likely to be cases (community controls [OR = 1.9, 95% CI : 1.1–3.1] and dispensary controls [OR = 2.6, 95% CI : 1.5–4.7]). There were no other risk factors associated with being a case.

Dispensary attendance
Cases and dispensary controls had dispensary attendance rates of 12.5 per 1000 person-days while the rate was only 10.7 per 1000 for community controls. Children with nets attended more for most of the illness events (Table 2Go). Distance from the dispensary was a limiting factor for attendance; observed by the decrease in overall attendance with increasing distance to the dispensary ({chi}2 for trend P < 0.001). Ownership of ITN was an indicator for higher utilization of the dispensary for those living between 0.5 and 3 km but not for those living very close or very far from the dispensary (Table 2Go).


View this table:
[in this window]
[in a new window]
 
Table 2 Attendance rates for different reasons at Idete dispensary for children with and without treated nets
 
Cross-sectional study
In total 652 children were interviewed in the cross-sectional assessment. The analysis included 629 (96.5%) children for whom we had information on both the net and Hb status. The characteristics of the children included in the analysis of the cross-sectional survey showed that those without nets had a higher prevalence of parasitaemia, anaemia and splenomegaly (Table 3Go). Comparison of those with ITN and with no nets in a multivariate analysis showed that those with ITN had a lower prevalence of any parasitaemia (OR = 0.5, 95% CI : 0.3–0.9). Age of the child and distance to the dispensary were other important factors (Table 4Go). Positive effects of treated nets were also shown for prevalence of anaemia.


View this table:
[in this window]
[in a new window]
 
Table 3 Characteristics of the children in Idete village from the cross-sectional assessment
 

View this table:
[in this window]
[in a new window]
 
Table 4 Risk factors for parasitaemia in the cross-sectional survey
 
Discussion

We found a positive association between risk of clinical malaria and ownership of mosquito nets using a case-control design. This finding is counter-intuitive and contradicts the results of the cross-sectional assessment in the same village and a repeated cross-sectional assessment conducted in the surrounding villages, which suggested an impact of the ITN in reducing the prevalence of anaemia and parasitaemia by 60%.11 An estimated 27% reduction in childhood deaths associated with treated nets has also been shown in the same area.12 The discrepancy is likely to be a consequence of attendance bias in the Idete case-control study. Children with nets attended more frequently for all the services at the dispensary, which may be explained by the higher awareness of health issues or as a consequence of the dispensary being one of the ITN sales outlets. Hence, the exposure of interest pre-determined to some extent recruitment, resulting in analysis and estimation of impact being done on an already selected group of individuals. Those with nets would be included as cases if they became sick. But those without nets were less likely to attend the health facility so although they were eligible to be community controls they were not likely to be seen as cases if they became sick with malaria. Thus, the community controls were not comparable with cases and both cases and dispensary controls were not comparable with the general population in the area. The lack of a statistically significant association between ITN ownership and being a case when considering dispensary controls may be due to a lack of power or to the fact that the exposure of interest also determined the likelihood of being a dispensary control. Attendance bias is not easily measured nor can it be corrected for in the analysis.16

Case-control studies for the evaluation of an intervention, usually compare the risk of being diagnosed with the disease among the adopters and non-adopters of the intervention; similar to assessments using either longitudinal or cross-sectional studies.7 For all these approaches, the adopters of the intervention are likely to differ from non-adopters in subtle but important ways that cannot be assessed or controlled for in the analysis. The comparison of adopters versus non-adopters answers a different type of question from intervention versus control, with the former measuring the health benefit of those who adopt rather than health impact per se.7 Therefore the results of these two comparisons cannot be directly compared. In an ITN programme, a randomly allocated contemporaneous control group2 would no longer be ethically acceptable (given their proven impact on mortality) or feasible, as this involves withholding nets from some groups.

The comparisons of adopters versus non-adopters in our studies illustrate a particular difficulty of assessing ITN programme impact where ITN use is very high. It may then be impossible to make this comparison because of the small numbers of non-adopters, who will represent a highly selected group. Furthermore, the likelihood that there is a ‘mass effect’ of the ITN17 may give some protection to non-adopters and this reduces differences in the malaria disease burden between adopters and non-adopters.

The logistical difficulty of obtaining enough cases and controls adds to the complexity of such studies. In our study the recruitment extended over 18 months and had a high rate of non-interviewed cases and controls. In a previous study in the Gambia a difference in the timing and recruitment pattern of the community and health facility controls was thought to be the source of discrepancy in the study results.4 Together, these observations call into question the usefulness of health facility based case-control studies for evaluating the morbidity impact of ITN programmes. An alternative study design would have been to use active case detection in the community. All children in the village could be visited at home every week, temperature taken from all and blood slides from those with fever. The cases would then be those with fever and parasitaemia and the controls would be a random sample of those without fever. However, this is logistically more complex and expensive, and hence usually unrealistic in the frame of a programme evaluation.

The lack of a simple test to check for insecticide content of the nets18 meant that exposure to ITN could not be accurately ascertained as suggested by work in The Gambia.4,19 Reported ownership of ITN may have led to some misclassification of exposure in our studies and reduced the power.

The finding that distance to the dispensary is an important predictor of attendance is in line with findings elsewhere.20 However, it is unclear if the observation of more cases in those living far from the dispensary indicates a true occurrence of more disease in the periphery of the village, or if it is just a reflection of patterns of dispensary attendance and self-medication for malaria episodes in the village. Furthermore, the OR need to be interpreted with caution as cases and controls were matched on area of residence.

Every health programme implementing interventions shown to be effective in randomized controlled trials aims to demonstrate that the desired health impact is also achieved under large-scale implementation—and if not, then why. However, given that (1) programme staff are unlikely to be familiar with epidemiological studies and (2) most programmes wish to invest more of their resources in implementation rather than monitoring, a compromise with regard to the optimal study design will have to be made. The use of simple cross-sectional surveys seems to be an attractive option, if suitable health outcomes can be identified and relevant exposure measures are feasible.


KEY MESSAGES

  • Intervention programme evaluations are being advocated for policy decisions.
  • Case-control studies are increasingly used for health impact assessment of intervention programmes.
  • The impact of insecticide-treated mosquito nets in reducing malaria morbidity could not be demonstrated using a facility-based case-control study.
  • Health facility based case-control studies may not be suitable for health impact assessment of malaria control programmes using insecticide-treated nets, while cross-sectional surveys represent an attractive alternative.

 

Acknowledgments

We would like to thank the children and guardians who participated in the study, Jensen Charles, Eric Mahundu and Patrick Rangimoto for assisting in the implementation of the studies. We also thank the Director and staff of the Ifakara Health Research and Development Centre (IHRDC) for facilitating the conduct of the study, Dr F Lwila (District Medical Officer) and the Idete dispensary staff. We are also very grateful to Dr Tom Smith for giving us valuable comments on earlier drafts of this manuscript. Ethical clearance was obtained from IHRDC and the Tanzania Commission of Science and Technology (COSTECH). Financial support was provided by the Swiss Agency for Development and Co-operation and the Government of Tanzania. CL is in receipt of the PROSPER grant 32–41632.94 from the Swiss National Science Foundation.

References

1 Habicht JP, Victora CG, Vaughan JP. Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. Int J Epidemiol 1999;28:10–18.[Abstract]

2 Lengeler C, Snow RW. From efficacy to effectiveness: insecticide-treated bednets in Africa. Bull World Health Organ 1996;74:325–32.[ISI][Medline]

3 Mohr LB. Impact Analysis for Program Evaluation. Thousands Oaks, CA: Sage Publications, 2000.

4 D'Alessandro U, Olaleye B, Langerock P et al. The Gambian National Impregnated Bed Net Programme: evaluation of effectiveness by means of case-control studies. Trans R Soc Trop Med Hyg 1997;91:638–42.[CrossRef][ISI][Medline]

5 Schlesselman JJ. Case-Control Studies: Design, Conduct, Analysis. New York: Oxford University Press, 1982.

6 Rowland M, Hewitt S, Durrani N, Saleh P, Bouma M, Sondorp E. Sustainability of pyrethroid-impregnated bednets for malaria control in Afghan communities. Bull World Health Organ 1997;75:23–29.[ISI][Medline]

7 Kirkwood BR, Cousens SN, Victora CG, de Zoysa I. Issues in the design and interpretation of studies to evaluate the impact of community-based interventions. Trop Med Int Health 1997;2:1022–29.[CrossRef][ISI][Medline]

8 Barutwanayo M, Coosemans M, Delacollette C, Bisore S, Mpitabakana P, Seruzingo D. Campaign against malaria vectors in the framework of a rural development project in Burundi. Ann Soc Belg Med Trop 1991;71(Suppl.1):113–25.[ISI][Medline]

9 D'Alessandro U, Olaleye BO, McGuire W et al. Mortality and morbidity from malaria in Gambian children after introduction of an impregnated bednet programme. Lancet 1995;345:479–83.[CrossRef][ISI][Medline]

10 Van Bortel W, Delacollette C, Barutwanayo M, Coosemans M. Deltamethrin-impregnated bednets as an operational tool for malaria control in a hyper-endemic region of Burundi: impact on vector population and malaria morbidity. Trop Med Int Health 1996;1:824–35.[ISI][Medline]

11 Abdulla S, Armstrong Schellenberg JRM, Nathan R et al. Impact of an insecticide treated net programme on the prevalence of parasitaemia and anaemia in children under two years of age in the Kilombero Valley, Tanzania. Br Med J 2001;322:270–73.[Abstract/Free Full Text]

12 Armstrong Schellenberg JRM, Abdulla S, Nathan R et al. Effect of large-scale social marketing of insecticide-treated nets on child survival in rural Tanzania.Lancet 2001;357:1241–47.[CrossRef][ISI][Medline]

13 Armstrong Schellenberg JRM, Abdulla S, Minja H et al. KINET: a social marketing programme of treated nets and net treatment for malaria control in Tanzania, with evaluation of child health and long-term survival. Trans R Soc Trop Med Hyg 1999;93:225–31.[ISI][Medline]

14 Charlwood JD, Smith T, Lyimo E et al. Incidence of Plasmodium falciparum infection in infants in relation to exposure to sporozoite-infected Anophelines. Am J Trop Med Hyg 1998;59:243–51.[Abstract/Free Full Text]

15 Mshinda H, Font F, Hirt R, Mashaka M, Ascaso C, Menendez C. A comparative study of the efficacies of chloroquine and a pyrimethamine-dapsone combination in clearing Plasmodium falciparum parasitaemia in school children in Tanzania. Trop Med Int Health 1996;1:797–801.[ISI][Medline]

16 Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32:51–63.[ISI][Medline]

17 Howard SC, Omumbo J, Nevill C, Some ES, Donnelly CA, Snow RW. Evidence for a mass community effect of insecticide-treated bednets on the incidence of malaria on the Kenyan coast. Trans R Soc Trop Med Hyg 2000;94:357–60.[CrossRef][ISI][Medline]

18 Drakeley CJ, Schellenberg JA, Abdulla S, Lengeler C. Lack of specificity of Beilstein test in detecting pyrethroid insecticide on coloured mosquito nets. Trop Med Int Health 1999;4:639–40.[CrossRef][ISI][Medline]

19 Muller O, Quinones M, Cham K, Aikins M, Greenwood B. Detecting permethrin on treated bed nets. Lancet 1994;344:1699–700.

20 Stock R. Distance and the utilization of health facilities in rural Nigeria. Soc Sci Med 1983;17:563–70.[CrossRef][ISI][Medline]