1 World Health Organization, Roll Back Malaria Dept., Avenue Appia 20, CH 1211Geneva 27, Switzerland
2 ORC Macro, 11785 Beltsville Drive, Calverton, MD 20705 USA
3 World Health Organization, StopTB Department, Avenue Appia 20, CH 1211Geneva 27, Switzerland
4 World Health Organization, Roll Back Malaria Department, Avenue Appia 20, CH 1211Geneva 27, Switzerland
5 KEMRI Wellcome Trust Collaborative Programme, 00100 GPO, P.O. Box 43640, Nairobi, Kenya and Centre for Tropical Medicine, University of Oxford, John Radcliffe Hospital, Oxford, OX3 9DU, UK
Correspondence: Dr EL Korenromp, World Health Organization, Roll Back Malaria Department, Avenue Appia 20, CH 1211Geneva 27, Switzerland. E-mail: korenrompe{at}who.int..
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods and Results Mortality estimates from 41 DHS conducted in African countries between 1986 and 2002, for the interval of 04 years preceding each survey (with a mean time lag of 2.5 years), were reviewed. The median relative error on national mortality rates was 4.4%. In multivariate regression, the relative error decreased with increasing sample size, increasing fertility rates, and increasing mortality rates. The error increased with the magnitude of the survey design effect, which resulted from cluster sampling. With levels of precision observed in previous surveys, reductions in all-cause under-5 mortality rates between two subsequent surveys of 15% or more would be detectable. The detection of smaller mortality reductions would require increases in sample size, from a current median of 7060 to over 20 000 women. Across the actual surveys conducted between 1986 and 2002, varying mortality trends were apparent at a national scale, but only around half of these were statistically significant.
Conclusions The interpretation of changes in under-5 mortality rates between subsequent surveys needs to take into account statistical significance. DHS birth history surveys with their present sampling design would be able to statistically confirm under-5 mortality reductions in African countries if true reductions were 15% or larger, and are highly relevant to tracking progress towards existing international child health targets.
Accepted 23 February 2004
All-cause under-5 mortality is a key health outcome in developing countries, and the reversal in Africa during the 1990s of the mortality decline apparent since the 1960s is the subject of much concern.17 The reversal has been attributed mainly to the human immunodeficiency virus (HIV) epidemic, although HIV-related mortality alone cannot fully explain this trend.6,7
Various major health programmes and initiatives focus on under-5 mortality. Most UN member states have agreed to the UN Millennium Development Goal (MDG) of reducing the under-5 mortality rate by two-thirds between 1990 and 2015.8 Under-5 mortality is included among the indicators proposed by the Global Fund against AIDS, TB, and Malaria.9 With malaria causing, or contributing to, over 20% of deaths in African children,1012 the all-cause under-5 mortality rate is also increasingly regarded as an important indicator of the impact of malaria control.13
In Africa, where civil registration is in most countries notoriously inadequate,14 a main source of statistics on all-cause under-5 mortality is the Demographic and Health Surveys (DHS), which estimate infant and childhood mortality rates through nation-wide sample surveys of women aged 1549 years.15 DHS were initiated in the mid-1980s in order to monitor key population, health and nutrition outcomes, notably on reproductive health, in populations where such information was lacking from routine health systems and from vital registration. The surveys have subsequently been expanded to cover a wide range of topics related to population and health programmes. They are carried out approximately every 5 years in an increasing number of developing countries.
We assessed what magnitude of change in under-5 mortality rates in African countries would be statistically detectable through DHS surveys, and what sample size would be required to detect expected or targeted levels of mortality reductions. The results are discussed in the light of the urgent need to provide robust measures of public health impacts in accordance with new international targets and increased investment in disease control.
![]() |
Demographic and Health Surveys |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Sampling usually occurs in proportion to the size of the population living in each cluster, with an equal number of households being chosen in each cluster within a sampling domain, so that the sample design is self-weighting. However, certain clusters are sometimes oversampled, in order to allow a sufficiently large sample to yield reliable subnational estimates. To produce outcomes for the urban and rural parts of countries separately, urban areas may be oversampled in countries where these areas constitute only a small part of the total population, e.g. in Cameroon in 1991 and 1998 and in Uganda in 1988. In some surveys, e.g. Malawi 2000 and Uganda 2000/2001, certain areas were oversampled because of programmatic demands. Typically, the sample size is between 15 and 30 households for urban clusters, and between 30 and 40 households for rural clusters.
Survey outcomes are affected by two types of sampling errors: (1) the statistical error due to the limited sample size, and (2) the design effect, which represents the factor by which the cluster-based sampling compounds this error. A design effect of 1.2 means that the total error is 1.2 times higher than it would have been if a simple random sample, without clustering, had been chosen. The design effect depends on the number of households per cluster and on the extent to which the outcome of interest (e.g. child mortality) varies within and between clusters. Therefore, for a given survey design, the design effect differs between outcomes and between countries. DHS surveys provide details of the survey design as well as the design effects for a list of key outcomes in appendices of their final reports.
![]() |
Mortality data from DHS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
For the purpose of country-level mortality monitoring we focus on national-level outcomes. Figure 1 shows African countries where national estimates of under-5 mortality rates are available from one, two, or more surveys. Table 1 lists the standard errors on mortality, for selected surveys in sub-Saharan Africa. The median error is 6.45; at a median mortality rate of 151 per 1000 births, this translates to a relative error of 4.4% (range: 2.5%, 7.6%). The median design effect for mortality is 1.30 (range: 1.08, 2.07), at an overall median number of households per cluster (pooled over urban and rural strata) of 25.7. These surveys sampled a median of 7060 women (range 3200 to 15 367); the median total fertility rate (estimated cumulative lifetime births per woman at current fertility rates) across the surveys was 5.8 (range: 4.0, 7.4).
|
|
![]() |
Mortality trends in actual DHS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
![]() |
Scenarios of mortality reductions |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
To predict the extent of statistical error under hypothetical scenarios of changing mortality and what sample size would be required to detect such trends, we described the errors in all African surveys available as of September 2003 as a function of the baseline mortality rate, design effect, and the number of women interviewed, as published in final reports of these surveys. In addition, the total fertility rate was included as an approximation of the number of births that interviewed women had in the preceding 5 years, and for which under-5 mortality was evaluated. A weighted least squares multivariate model was fitted in SPSS version 10.0.7 (SPSS Inc., 19891999). Each survey was given an equal weight in the multivariate analyses. The four determinants were included as continuous variables; a logistic transformation was applied on 5q0 and the effects of the survey sample size and of the total fertility rate were linearized by using the reciprocal of their square roots.
The model obtained a good fit: across the 41 surveys, the four transformed variables together explained 94% of the variation in statistical error (Appendix). The error increased slightly with the baseline mortality rate (P < 0.001, Figure 3a). This increase was less than proportional, so that the relative error, in contrast, decreased with the baseline mortality rate (not shown). The error decreased with the number of women interviewed (P < 0.001, Figure 3b) and with the total fertility rate (P = 0.001, Figure 3c). Although both these variables are associated with the number of births for which survival is evaluated, the influence of fertility level was less marked because the surveys differed less in fertility levels than in sample size. Large design effects increased the error (P < 0.001, Figure 3d). The effects of all variables were independent from one another.
|
|
The model allowed us to predict the standard error for hypothetical surveys of given sample size and baseline mortality rate. For this evaluation we assumed a fixed design effect of 1.30, the median across all actual surveys (Table 1). The statistical significance of mortality trends over subsequent hypothetical surveys with a predicted standard error were evaluated by assuming mortality rates to be normally distributed. Detection of a mortality rate reduction of 20% would require a sample size of 6700 in case of a baseline mortality rate of 70 per 1000 and a sample size of 1450 women at a baseline mortality rate of 250 per 1000. Not surprisingly, larger sample sizes would be required with lower baseline mortality rates, which is graphically illustrated in Figure 4. Figure 4 also shows that most of the recently completed DHS in sub-Saharan Africa, with sample sizes of between 4000 and 8000 (Table 1) would be able to pick up a 1520% mortality reduction. To statistically detect a mortality rate reduction of only 10% would, however, require a sample size of between 6500 and 14 800 (Figure 4), approaching the maximum among actual recent surveys (Table 1).
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The international community has established new child survival targets over the next two decades and signed commitments to achieve these goals.8 The required two-thirds reduction in the under-5 mortality rate by 2015 translates to a reduction by 40% every 5 years from 2004 onwards. Encouragingly, all national DHS surveys are sufficiently powerful to detect this change. For Africa, a considerable part of the expected mortality reduction is to come about by improved malaria control, under the Roll Back Malaria strategy20 and continued financial support from the Global Fund against AIDS, TB, and Malaria.21 Based on evidence from African malaria intervention trials,2224 a 15% reduction in all-cause under-5 mortality is a realistic (minimum) expectation for the impact of improved malaria control in countries with increased distribution of insecticide-treated mosquito nets and increased coverage of malaria cases with prompt treatment with effective antimalarial drugs. Our analysis thus supports the proposal that the all-cause under-5 mortality rate as measured by national surveys should be a key indicator of the epidemiological impact of Roll Back Malaria.13
Limitations
A number of limitations must be recognized to the presented evaluations of mortality time trends and the statistical power of surveys. Survey-based estimates of under-5 mortality for the period 04 years preceding the survey refer to a midpoint of 2.5 years before the survey. The detection of the mortality impact of a health programme will therefore be delayed by an average of 2.5 years; in other words, only 2.5 years after an effective programme had been implemented could a mortality impact be reliably attributed to expanding coverage of interventions. Similarly, survey data from a current year could serve as the baseline measurement for the evaluation of a programme that started 2.5 years ago.
At subnational levels, the detection of under-5 mortality time trends would be less powerful than we have described for the national level (Figure 4). To compensate for the smaller sample size, DHS typically provide mortality outcomes for regions, provinces, and urban or rural areas for a 10-year interval preceding the survey rather than the 5-year interval used for national estimates. These reports at sub-national levels are often used to highlight spatial inequities, but again often without any consideration of the statistical significance of eventual differences. Even so, the relative errors on subnational rate estimates are large: a median of 7.1% of the mortality rate for urban areas, 3.9% for rural areas, and 78% for provinces and regions (not shown). To statistically detect changes between surveys for subnational areas would therefore require sample sizes of at least 35% larger than those shown in Figure 4. More robust techniques or sampling is required to provide reliable estimation of subnational mortality trends. These might include demographic surveillance systems (DSS) which monitor mortality prospectively over time in defined populations,11,25 although these systems often suffer from a lack of representativeness of the populations covered.25
Similarly, the errors of mortality rates for specific age groups among the under-5s are larger that those presented here for all under-5s pooled. For example, for the most recent 04 year interval, the median relative standard error was 5.6% for infant mortality rates across surveys reported in Table 1 and 6.6% for child (14 years) mortality rates. The magnitude of these sampling errors dictates that 1322% larger sample sizes would be required than for total under-5 mortality rates.
Our analysis did not address non-sampling errors in the birth history surveys. It is nevertheless likely that non-sampling errors on under-5 mortality estimates are substantial in some surveys17,26,27and these cannot be reduced by increasing sample size. Although interviewers in DHS are extensively trained to probe for all births and deaths, and to elicit accurate information about dates of birth and ages at death, omission of deaths and misreporting of ages and dates of birth are causes of concern. Mothers may not report all of their births, particularly for children born long before the survey and who died at a very young age. Interviewers may fail to record all deaths in order to avoid asking potentially uncomfortable questions about the pregnancy, delivery, and feeding practices for children who have died. Age inflation may be a problem in some surveys, particularly in countries where mothers do not know the exact age of their children. In those countries, some interviewers might increase the ages of young children to avoid a long set of questions on maternal and child health, as well as height and weight measurements, that are restricted to young children. However, the impact of age displacement on estimates of childhood mortality has been found to be small or negligible.17 Overall, non-sampling errors may substantially distort mortality rate estimates in some surveys, although if these errors remain similar across surveys, the effect on time trends will be small. A potential bias that will affect time trends is the missing of children whose mothers have died because birth history data are limited to the biological children of living women. Since mortality may be higher among orphaned children,2831 birth histories may underestimate overall mortality. This is of particular concern in countries in Southern and Eastern Africa with high or rising HIV-related mortality among young women.
Explanation and interpretation of trends
The interpretation of statistically detectable mortality changes in terms of specific diseases or health interventions was beyond the scope of this analysis. The most universal current determinant of changing child mortality in Africa is the increase in HIV-related deaths;7 in evaluations of the impact of major child health programmes such as the Integrated Management of Childhood Illnesses (IMCI),32 Roll Back Malaria, or diarrhoeal management programmes, increasing HIV mortality would confound (deflate) the apparent impact. The proportion of under-5 mortality attributable to HIV can be estimated from HIV prevalences in antenatal clinic attenders and subtracted from all-cause mortality. For the year 1999, this estimation has been undertaken for all countries in sub-Saharan Africa.7 HIV infection caused an estimated 7.7% of under-5 deaths, as compared with 2% in 1990. Where HIV prevalence estimates are available for other years, the changing burden of HIV-related under-5 mortality could similarly be estimated and subtracted from corresponding child mortality surveys to obtain the trend in non-HIV mortality. However, uncertainties inherent to the estimation of HIV-related under-5 mortality rates would add to the error in survey-based all-cause mortality estimates, decreasing the statistical power to evaluate trends in non-HIV mortality.
The DHS data represent one of the sources for the under-5 mortality estimation by UNICEF that is agreed upon as one of the benchmarks of the Millennium Development Goals.33 The latter estimation takes into account, besides DHS, estimates of mortality rates from censuses, vital registration, and non-birth history surveys, such as the Multiple Indicator Cluster Surveys.34 This data synthesis assigns different weights to the different data and estimation methods based on their respective quality and validity, as judged by the analysts. Error estimates are typically not available for the indirect mortality estimates, but sampling errors in the non-birth history surveys are likely to be at least as large as those in DHS surveys, considering several additional biases inherent to the indirect method. Non-sampling errors can be substantial in all data sources, and they might introduce spurious time trends if certain sources with specific biases contribute proportionally more in certain time periods. Although the syntheses of all available data might reduce the sampling error in the resulting overall time trend, it is our belief therefore that the overall error in such combined estimates is on the same order of magnitude as we presented here for DHS specifically.
Implications for future surveys
An obvious approach to improving the statistical precision to detect under-5 mortality transitions would be to increase sample size. This is apparent among more recent DHS surveys in sub-Saharan Africa: 8 DHS conducted in 2000 or later had an average sample size of 9904 women, compared with 6225 over the 51 surveys conducted before 2000 (Figure 5). As is implicit in our scenario analysis (Figure 4) from the use of a fixed design effect, increasing the sample size largely takes the form of more clusters, rather than more households per cluster. Increasing the number of clusters sampled is statistically and logistically preferable to increasing the number of households per cluster, in which case the gain in statistical power would be less than shown in Figure 4, owing to an increased design effect.
|
An alternative suggestion has been to increase the frequency of surveys (e.g. every 3 years) while retaining the current number of clusters. More frequent surveys would help the monitoring of outcomes that can change rapidly over time and that are measurable very precisely, such as the coverage with mosquito nets among under-5s in response to a malaria control programme.35 For mortality rates, however, increasing survey frequency is probably not the most efficient way to increase the statistical power. Mortality reductions that occurred between pairs of subsequent surveys would be smaller, because the group of under-5s contributing to subsequent 5-year intervals would start overlapping. As a consequence, an even larger sample size would be required (see Figure 4, in which it is of note that the sample size requirements hold true irrespective of the interval between the surveys).
With increasing investment in child survival initiatives in Africa there is an increasing need to ensure that targets set by the international community are monitored. Statistical precision, largely dependent upon sample size, has often been ignored in the presentation of under-5 mortality trends. Birth history surveys will continue to be the benchmark to monitor under-5 mortality rates. In their current design, DHS surveys in Africa should provide enough power to detect mortality rate reductions at national levels of 15% or larger within time scales defined by international partners, and they are thus highly relevant to tracking progress towards existing international child health targets. The changing dynamic of mortality needs to be constantly reassessed in terms of the ability of these surveys to detect temporal changes: increasing HIV prevalence will blunt impacts achieved through other disease-specific interventions and substantial declines in mortality rates would merit increased cluster sample size over time.
KEY MESSAGES
|
![]() |
Acknowledgments |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Ewbank DC. A strategy for evaluating the mortality impact of child survival programmes in Africa: Combatting Childhood Communicable Diseases Project. Int J Epidemiol 1993; 22(Suppl.1):S814.[Abstract]
3 Ahmad OB, Lopez AD, Inoue M. The decline in child mortality: a reappraisal. Bull World Health Organ 2000; 78:117591.[ISI][Medline]
4 Rutstein SO. Factors associated with trends in infant and child mortality in developing countries during the 1990s. Bull World Health Organ 2000; 78:125670.[ISI][Medline]
5 Timaeus IM. Impact of the HIV epidemic on mortality in sub-Saharan Africa: evidence from national surveys and censuses. AIDS 1998;(Suppl.1):S15S27.
6 Adetunji J. Trends in under-5 mortality rates and the HIV/AIDS epidemic. Bull World Health Organ 2000; 78:120006.[ISI][Medline]
7 Walker N, Schwartlander B, Bryce J. Meeting international goals in child survival and HIV/AIDS. Lancet 2002; 360:28489.[CrossRef][ISI][Medline]
8 Millennium Assembly of the United Nations. Millennium Development Goals. 2000. http://www.developmentgoals.org/Child_Mortality.htm
9 The Global Fund against AIDS, TB and malaria. http://www.globalfundatm.org.
10 World Health Organization. World Health Report 2002. Reducing Risks, Promoting Healthy Life. Geneva: World Health Organization. no. WHA 540.1, 2002.
11 Korenromp EL, Williams BG, Gouws E, Dye C, Snow RW. Measurement of trends in childhood malaria mortality in Africa: an assessment of progress toward targets based on verbal autopsy. Lancet Infect Dis 2003; 3:34958.[CrossRef][ISI][Medline]
12 Black RE, Morris SS, Bryce J. Where and why are 10 million children dying every year? Lancet 2003; 361:222634.[CrossRef][ISI][Medline]
13 Roll Back Malaria. Framework for Monitoring Progress and Evaluating Outcomes and Impact. Geneva: World Health Organization. no. WHO/CDS/RBM/2000.25, 2000.
14 Mung'ala VO, Snow RW. Death registration on the Kenyan Coast. East Afr Med J 1994; 71:74750.[ISI][Medline]
15 ORC Macro, MEASURE DHS+. Demographic and Health Surveys (DHS). ORC Macro, Calverton, MD, USA. http://www.measuredhs.com
16 Hill AG, David PH. Monitoring changes in child mortality: new methods for use in developing countries. Health Policy Plann 1988; 3:21426.
17 Bicego G, Ahmad OB. Infant and child mortality. Demographic and Health Surveys Comparative Studies No. 20. 1996: August. Macro International Inc., Calverton MD. Demographic and Health Surveys Comparative studies no.
18 Altman DG, Machin D, Bryant TN, Gardner M (eds). Statistics with Confidence. 2nd Edn. London: British Medical Journal University Publications, 2000.
19 Boerma JT, Sommerfelt AE. Demographic and health surveys (DHS): contributions and limitations. World Health Stat Q 1993; 46:22226.[Medline]
20 Roll Back Malaria. The African summit on Roll Back Malaria. 2000: April 25, 2000. World Health Organization, Abuja. no. WHO/CDS/RBM/2000.17.
21 Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS. How many child deaths can we prevent this year? Lancet 2003; 362:6571.[CrossRef][ISI][Medline]
22 Lengeler C. Insecticide-treated bednets and curtains for preventing malaria. Cochrane Database Syst Rev 2000; CD000363.
23 Phillips-Howard PA, Nahlen BL, Kolczak MS et al. Efficacy of permethrin-treated bednets in the prevention of mortality in young children in an area of high perennial malaria transmission in western Kenya. Am J Trop Med Hyg 2003; 68:2339.
24 Kidane G, Morrow RH. Teaching mothers to provide home treatment of malaria in Tigray, Ethiopia: a randomised trial. Lancet 2000; 356:55055.[CrossRef][ISI][Medline]
25 Byass P, Berhane Y, Emmelin A et al. The role of demographic surveillance systems (DSS) in assessing the health of communities: an example from rural Ethiopia. Public Health 2002; 116:14550.[ISI][Medline]
26 Rutstein SO. Fertility levels, trends, and differentials 19951999. 2002: December. ORC Macro, Calverton MD. DHS Comparative Reports no. 3.
27 Curtis S. Assessment of the quality of data used for direct estimation of infant and child mortality in DHS-II surveys. 1995: March. Macro International Inc., Calverton MD. Occasional Papers no. 3.
28 Ryder RW, Nsuami M, Nsa W et al. Mortality in HIV-1-seropositive women, their spouses and their newly born children during 36 months of follow-up in Kinshasa, Zaire. AIDS 1994; 8:66772.[ISI][Medline]
29 McDermott JM, Slutsker L, Steketee RW, Wirima JJ, Breman JG, Heymann DL. Prospective assessment of mortality among a cohort of pregnant women in rural Malawi. Am J Trop Med Hyg 1996; 55:6670.[ISI][Medline]
30 Ng'weshemi J, Urassa M, Isingo R et al. HIV impact on mother and child mortality in rural Tanzania. J Acquir Immune Defic Syndr 2003; 33:393404.[Medline]
31 Crampin AC, Floyd S, Glynn JR et al. The long-term impact of HIV and orphanhood on the mortality and physical well-being of children in rural Malawi. AIDS 2003; 17:38997.[CrossRef][ISI][Medline]
32 World Health Organization Division of Child Health and Development. Integrated management of childhood illness: conclusions. Bull World Health Organ 1997;75(Suppl.1):11928.[ISI][Medline]
33 Hill K, Pande R, Mahy M, Jones G. Trends in child mortality in the developing world: 1960 to 1996. New York: UNICEF, 1999.
34 UNICEF. Multiple Indicator Cluster Surveys (MICS-2)/End of Decade Assessment. 2001. http://www.childinfo.org/MICS2/
35 Korenromp EL, Miller J, Cibulskis RE, Cham MK, Alnwick D, Dye C. Monitoring mosquito net coverage for malaria control in Africa: possession versus use by children under 5 years. Trop Med Int Health 2003; 8:693703.[CrossRef][ISI][Medline]