a UK Cochrane Centre, NHS Research & Development Programme, Oxford OX2 7LG, UK.
Steve McDonald, Australasian Cochrane Centre, Monash Institute of Health Services Research, Monash Medical Centre, Locked Bag 29, Clayton, VIC 3168, Australia. E-mail: steve.mcdonald{at}med.monash.edu.au
Abstract
Background Randomized trials are important for controlling selection biases, and where sufficient numbers of participants are involved, have the potential to yield reliable estimates of treatment effects.
Methods We investigated trends in the number and size of randomized trials reported in general health care journals from 1948 to 1997. From the handsearching of 18 general health care journals we collected data on the number of reports of randomized trials in each journal per year, and the number of participants in each trial.
Results A total of 5503 reports of trials were identified in 18 general health care journals. More than a third appeared in the British Medical Journal. The peak period for trial reports was the mid 1980s, with more in 1986 than any other year (242). By the mid 1990s the number per year had declined by a third. Trials with fewer than 100 participants accounted for most of the reports (69%). In spite of the overall decline in the number of trial reports, those involving 100 participants or more continued to increase throughout the period studied.
Conclusions The continued increase in the number of larger trials reported is encouraging, especially if it represents an increase in the size of trials more generally. Further research is needed to determine whether the trends over time identified here are reflective more of trends in the actual conduct of, rather than simply the reporting, of randomized trials.
Keywords Randomized controlled trials, periodicals
Accepted 3 October 2001
In 1948 the British Medical Journal (BMJ) published the landmark trial of streptomycin for pulmonary tuberculosis.1 Although not the earliest reported randomized trial,2,3 it is credited with setting the standard for the conduct and reporting of modern randomized trials, and marks the earliest year from which members of the Cochrane Collaboration routinely search for trials.4
Efforts by the Cochrane Collaboration to identify randomized trials have resulted in over 310 000 references being made available in The Cochrane Controlled Trials Register.5,6 Cochrane centres have contributed to this activity by handsearching general health care journals, a procedure in which the complete contents of a journal are read to identify reports of trials. This study presents an analysis of the number and size of randomized trials reported in a sample of these journals.
Methods
To ensure a range of countries and languages was represented, each Cochrane centre involved in the study selected one general health care journal from one or more of the countries for which it acts as the reference centre. Where possible, this was the journal of the national medical association (or equivalent).
Journals were handsearched for the 50 years from 1948 to 1997 for all reports of trials in which participants were randomly (or quasi-randomly) assigned to alternative forms of care. Trial reports published as letters, conference abstracts and news items were excluded from this study. We analysed data on the annual number of trial reports identified in each journal, and the number of participants in each trial.
Results
Data for 18 journals were supplied by 10 Cochrane centres. Fifteen were searched for the entire period from 1948 to 1997. Seven journals were published in English, three in German, and one each in Danish, Dutch, Finnish, French, Italian, Norwegian, Spanish and Swedish (Table 1).
|
|
The number of reports of large trials per year increased from a mean of 9 in the mid 1950s to 44 in the mid 1980s. In contrast to the pattern for small trials, this increased into the 1990s (to 62). The increase in the 1990s was primarily a feature of two journals (Annals of Internal Medicine and the BMJ) which account for 76% of all large trials identified between 19901997. The number of large trials appearing in the other journals was either declining or the trend was uncertain. The proportion of trials defined as large increased over time. During the 1950s and 1960s a third of the trials were large. This proportion fell to a quarter by the mid 1980s but then increased, so that just over half the trials had 100 or more participants by the mid 1990s.
Discussion
Our study found that, in spite of a decline in the number of trial reports appearing in general health care journals over the last 1015 years, there was an increase throughout the 50-year period in the number of trials involving 100 or more participants.
The downturn in trial reports seen in these journals is unlikely to reflect a wider trend to conduct fewer randomized trials. A search of MEDLINE, for example, using the publication type term Randomized-Controlled-Trial reveals a large year-on-year increase since 1966 in the number of studies retrieved. In addition, a survey of 2000 controlled trials in schizophrenia,8 found largely in psychiatric journals by hand- and electronic-searching, revealed a steady increase in the annual number of trials over 50 years. More plausible explanations are the growth in the number of specialist journals, and the changing nature of general journals. A publication analysis of trials conducted by the UK Medical Research Council shows a trend towards reporting in specialist journals (I Chalmers, personal communication). For the period 19481967, 98% (59/60) of MRC trials were reported in the BMJ or the Lancet, but this was only 20% (32/160) for 19781997. Competition from specialist journals, particularly those with an international readership, is likely to be most keenly felt by the general journals with smaller circulations.
Decreases in the number of trial reports in general journals may indicate changing editorial policies and author preferences. The emergence of systematic reviews may have affected the number of trials being published in general journals, and the trend towards larger trials may suggest that editors are becoming more selective in the trials they publish. Authors themselves may be by-passing general journals, especially if the trial is small and of only limited interest, or deciding that their research will receive greater exposure in a leading specialist journal.
The trend towards an increasing number and proportion of large trials is encouraging, although this is only apparent in the Annals of Internal Medicine and the BMJ. The number of trials involving over 1000 participants has also increased in these two journals. This might suggest that editors are becoming more selective in the trials they decide to publish, but also, hopefully, indicates a realization by researchers and funding bodies of the need to conduct trials that are of a sufficient size to detect moderate but important treatment effects.
Our study had several potential limitations. First, our sample size was small and omitted other major general journals such as JAMA and the Lancet. Second, we acknowledge that 100 is an arbitrary marker of trial size and that other factors besides absolute sample size numbers affect power calculations. Third, we did not attempt to correct for potential confounding resulting from changes in the number of pages and/or number of issues published per year in these journals. Finally, a small number of trials might have been counted in the analyses more than once, for example, where multiple reports of the same trial appeared in two or more journals.
Conclusion
Our findings have revealed a decline in the number of trial reports in general health care journals, but also a tendency for the trials that do appear to be larger. Further research is needed to determine whether these findings are reflective more of trends in the conduct of trials or in their reporting.
KEY MESSAGES
|
Acknowledgments
Financial support: The handsearch of the European journals included in this study formed part of a European Union project under the BIOMED-1 Program [contract number BMH1-CT94-1289] which was co-ordinated at the UK Cochrane Centre. The authors thank Daniel Galandi, Kirsten Lone Jensen, Joan Marti and Jane Nguyen for contributing to the data extraction, Jim Halsey for running the statistical analyses, Iain Chalmers for permission to use unpublished material, and the many handsearchers who between them read through a combined total of over 800 years of published papers. MW is supported by a Daphne Jackson Fellowship.
Notes
b Current address: Australasian Cochrane Centre, Monash Institute of Health Services Research, Melbourne, VIC 3168, Australia.
Group participants: Gerd Antes (German Cochrane Centre), Xavier Bonfill (Iberoamerican Cochrane Centre), Frank Davidoff (Annals of Internal Medicine), Peter Gotzsche (Nordic Cochrane Centre), Margaret Haugh (French Cochrane Centre), Marjan Loep (Dutch Cochrane Centre), Philippa Middleton (Australasian Cochrane Centre), Elizabeth Pienaar (South African Cochrane Centre) and Vanna Pistotti (Italian Cochrane Centre).
References
1 Medical Research Council Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment for pulmonary tuberculosis. Br Med J 1948;ii:76982.
2 Royal College of Physicians of Edinburgh. Controlled trials from history. (www.rcpe.ac.uk/controlled_trials) Accessed 30 September 2001.
3
Doll R. Controlled trials: the 1948 watershed. Br Med J 1998;317: 121720.
4 Bero L, Rennie D. The Cochrane Collaboration. Preparing, maintaining, and disseminating systematic reviews of the effects of health care. JAMA 1995;274: 196264.[CrossRef][ISI][Medline]
5 Lefebvre C, Clarke MJ. Identifying randomised trials. In: Egger M, Davey Smith G, Altman DG (eds). Systematic Reviews in Health Care: Meta-analysis in Context. 2nd Edn. London: Br Med J Publishing, 2001.
6 The Cochrane Controlled Trials Register. In: The Cochrane Library. Oxford: Update Software, 2001. Updated quarterly.
7 Medical Research Council Poliomyelitis Vaccines Committee. The assessment of the British vaccine against poliomyelitis. Br Med J 1957;i:127177.
8
Thornley B, Adams C. Content and quality of 2000 controlled trials in schizophrenia over 50 years. Br Med J 1998;317: 118184.