1 Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
2 Division of Clinical Care Research, Department of Medicine, Tufts University School of Medicine, Boston, MA.
Received for publication April 3, 2002; accepted for publication May 8, 2003.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Africa; cluster analysis; data collection; developing countries; planning techniques; random allocation; randomized controlled trial
Abbreviations: Abbreviations: CONSORT, Consolidated Standards of Reporting Trials; CRCT, cluster randomized controlled trial; ICC, intracluster correlation coefficient.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In this article, we report the results of a methodological evaluation of CRCTs performed in sub-Saharan Africa. Our aim was to assess the extent to which the prerequisite design and analysis aspects of cluster randomization were taken into account and reported properly in the trial publications. This information may be important for gaining insight into improving the conduct and reporting of future CRCTs in developing countries.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Study reports that obviously reflected secondary publication of a main study report were also excluded. However, whenever secondary publications reported additional useful information about the trial design or analysis, this information was recorded and was used to give respective credit to the trial.
Identification of trials
We used a comprehensive database of randomized controlled trials in sub-Saharan Africa (10). This database is based on MEDLINE, the Cochrane Controlled Trials Register, and the African Published Trials Register of the South African Cochrane Center, which includes hand searching of major African journals. MEDLINE and the Cochrane Controlled Trials Register searches were updated until November 2001. A number of terms reflecting CRCTs were used in conjunction with "Africa" and "sub-Saharan Africa" and with specific geographic names. Details on the search strategy are available on request.
Data extraction
We reviewed each potentially eligible article to determine whether it satisfied the selected criteria. From each eligible article, we extracted the following information: author, journal, year of publication, country or countries of recruitment, disease(s) or condition(s) targeted, number of trial arms and type of intervention, and methodological criteria (as described below). One author extracted data on all items. The other assessed all questionable items. In case of disagreement, consensus was reached after discussion.
Methodological criteria
To formulate the specific items of the methodological evaluation of the included trials, we referred to the Consolidated Standards of Reporting Trials (CONSORT) statement checklist (11, 12), taking into account the published suggestions on extending the CONSORT statement to CRCTs (13). The standard CONSORT checklist of items that should be included in the trial report was thus modified selectively to take into account the main specific methodological issues referring to the design, conduct, analysis, and reporting of CRCTs.
Thus, for each article, we recorded whether 1) the study was identified as a CRCT in the title; 2) the rationale was given for choosing the cluster design and, if so, what the rationale was; 3) the exclusion and inclusion criteria were stated for individuals, clusters, or both; 4) the planned intervention was aimed at individuals, clusters, or both; 5) the primary outcome(s) was stated clearly; 6) the sample size, number of clusters, and cluster size were reported; 7) the sample size calculations took clustering into account; 8) the intracluster correlation coefficient (ICC) was calculated and recorded; 9) the design effect was estimated and reported; 10) the unit of randomization was described; 11) pairing and/or stratification were used; 12) the within-cluster recruitment procedure was stated to be cross-sectional or longitudinal; 13) within-cluster sampling was used; 14) the method of masking was stated; 15) allocation schedule control (location of code) was described; 16) a participants flow diagram was provided; 17) the level of analysis was stated; 18) clustering was taken into account in calculating confidence intervals or p values; 19) results in absolute numbers were provided in sufficient detail; 20) prognostic variables by treatment group and any attempt to adjust for them were described; and 21) protocol deviations were reported. The ICC is defined as the ratio of the between-cluster component of variance to the total variance (sum of between-cluster and within-cluster variances). The design effect is estimated by using the formula 1 + (m 1) x ICC, where m is the mean sample size in a cluster unit, and it signifies how many more individuals are required for a cluster design versus a trial randomizing individuals in order to have the same power.
Whenever all of the qualitative criteria listed above were met by less than 20 percent of the trial reports, we used Fishers exact tests to examine whether the situation was better in more recent trials (published in 1996 or later) than in earlier trials. We also used Spearmans correlation coefficients to evaluate whether sample size measures correlated with the year of publication.
Analyses were conducted by using SPSS software (SPSS Inc, Chicago, Illinois). All p values reported in this article are two-tailed.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The number of CRCTs published per year increased over time. The earliest sub-Saharan CRCT identified was published in 1973, but 40 (78 percent) trials were published after 1990, and 25 (49 percent) were published in 1996 or later. The trials were conducted in 20 sub-Saharan African countries, including the Gambia (n = 9 trials), South Africa (n = 8), Tanzania (n = 6), Kenya (n = 5), Ghana (n = 3), Zimbabwe (n = 3), Zaire (n = 2), Uganda (n = 2), and Ethiopia (n = 2); another 11 countries contributed one trial each. Only one CRCT had been conducted in multiple countries. A large number of these trials were published in The Lancet (n = 11); other trials were published in Tropical Medicine & International Health (n = 6), Transactions of the Royal Society of Tropical Medicine and Hygiene (n = 5), Bulletin of the World Health Organization (n = 4), AIDS (n = 3), The American Journal of Tropical Medicine and Hygiene (n = 2), International Journal of Epidemiology (n = 2), Journal of the Dental Association of South Africa (n = 2), Social Science & Medicine (n = 2), and 14 other journals (one trial each).
Common subjects included malaria (18 trials (35 percent)) as well as sexually transmitted diseases, acquired immunodeficiency syndrome, and reproductive health (seven trials (14 percent)). Another 16 CRCTs focused on other infectious and parasitic diseases (diarrheal diseases (n = 4), trachoma (n = 4), hepatitis B (n = 2), intestinal helminthes (n = 2), childhood-cluster diseases and immunization (n = 2), trichiasis (n = 1), and otitis media (n = 1)). Oral conditions and proper drug use were the focus of four and three trials, respectively, while nutrition, epilepsy, and antenatal care accounted for one trial each.
Most trials (n = 37 (73 percent)) had two arms, but nine trials had three arms and five had three to six arms. Thirty-four CRCTs (67 percent) focused predominantly on prevention and 12 on treatment, while the type of intervention in the remaining five studies was either a combination of the two (therapeutic interventions combined with health education activities) or focused on drug management (including stock management and rational drug use). Among preventive intervention trials, 13 focused on the use of insecticide-treated bed nets for preventing malaria.
As shown in table 1, only one trial was identified as a CRCT in the title, whereas several trials were identified as "community trials" or "community-based trials" without stating their specific cluster design. Most of the trials offered no justification for randomizing clusters rather than individuals. The rationale for choosing the cluster design was given in 11 studies; five studies reported logistics and/or administrative reasons, four used this design to avoid intercontamination of the randomized groups, and two more ascribed their choice of the design to some specific characteristics of the type of intervention. Most of the trials stated the inclusion/exclusion criteria at the level of the individual, whereas the planned intervention was aimed at the individual in almost half of the studies reviewed.
|
Table 2 reports on the assignment and masking features of the 51 eligible trials. All trials described the unit of randomization, many of them actually in sufficient detail. Various pairing and stratification methods and recruitment procedures were used. Although the types of clusters were quite diverse, they can be broadly classified into one of three main categories: villages and residential areas, schools or classes, and health care settings (mostly primary health clinics). Double blinding was uncommon, and allocation control was rarely described (table 2).
|
|
Some improvement was documented in several parameters over time. Specifically, clustering had been accounted for in sample size calculations in 10 of 26 trials published in 1996 or later but in none of the earlier reports (p = 0.001); clustering was accounted for in the analysis in 13 of these 26 trials versus 6 of 25 earlier ones (p = 0.083). The only trials that reported an ICC and/or design effects were also recent. Less impressive, nonsignificant improvements were also seen in the reporting of rationale, description of allocation control, and presentation of absolute numbers in sufficient detail (not shown). A nonsignificant correlation was found between year of publication and sample size (r = 0.17, p = 0.23) or number of clusters (r = 0.12, p = 0.44).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The results of our study are quite similar to those of previous empirical evaluations that targeted relatively smaller sets of CRCTs in other medical domains, and they may even suggest a lower rate of use of appropriate statistical methods in CRCTs than estimated previously. In their review of 16 nontherapeutic intervention trials, Donner et al. (7) found that only three studies (19 percent) accounted for between-cluster variability in sample size or power calculations; however, eight of the 16 (50 percent) trials took into account the effect of clustering in the analysis. Simpson et al. (6) recorded a similar picture in an appraisal of 21 primary prevention trials. Finally, in a review of 24 computer-based clinical decision support systems intervention trials, Chuang et al. (8) found that only one study (4 percent) took clustering into account in calculating sample size, although 14 of the 24 trials (58 percent) used adequate statistical methods for analysis. With the exception of four trials in the Chuang et al. report, all other trials in these three empirical evaluations were published before 1996. Half of the trials in our evaluation were published in 19962001, and, despite some definite improvement in appreciating the implications of the cluster design in these trials compared with earlier ones, deficiencies were still common.
We also tried to record several other facets of the quality of reporting of CRCTs in sub-Saharan Africa, and several problems were detected. Inadequate reporting may be associated with either clear overestimation of the effects of interventions (14, 15) or unpredictable bias in the effect size (16). Moreover, the report of a trial is a proxy for the true trial quality (17, 18), although sometimes the actual quality of a trials design, conduct, and analysis may not be adequately reflected in the study report (19). For example, it is possible that trial investigators may give considerable thought to the rationale for using a cluster design, but this rationale may not be presented properly in the final report. Adopting a standardized checklist may facilitate adequate reporting of CRCTs. The CONSORT statement checklist has been adopted by most of the main medical journals and by many research teams. The modified CONSORT checklist that we used is a relatively simple tool that could assist researchers when reporting a CRCT.
Several methodological articles have been published addressing the specific issues of the design, conduct, and analysis of CRCTs (17, 18, 2022). However, publications presenting essential, valuable information such as the estimates of ICCs and design effects are still limited (23, 24). It is therefore recommended that authors include their ICC estimates in the main trial publications to help future investigators plan CRCTs.
The lack of appropriate statistical implementation of cluster designs in the past might have been in part also due to lack of readily available software. Currently, however, analyses can be conducted easily by using the PROC MIXED procedure of the Statistical Analysis System (25). Specialized software such as ACLUSTER (26) has also become available, focusing on estimating intracluster correlation coefficients, calculating sample size for cluster designs, and analytical methods for binary, continuous, and time-to-event outcomes.
Despite the relative inefficiency of cluster randomization compared with individual randomization (in terms of statistical power), future investigators should not be discouraged from using this design whenever indicated. In many interventions, there is no alternative to cluster randomization. Ethical and political concerns, the need to minimize the potential for intercontamination of the randomized groups, administrative problems, and a limited budget can force researchers to abandon the possibility of using a design that randomizes individuals. In developing countries such as in sub-Saharan Africa, where there is a large burden of disease and research resources are limited (10), CRCTs are likely to be very useful for addressing a variety of important medical and public health-related questions. Careful design, conduct, and analysis as well as proper reporting of CRCTs could improve the quality of medical research and contribute toward more effective health care in developing countries.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|