Underreporting of clinical trials in journals and registries is undermining the progress of medical science and breaching an ethical obligation to patients, advocates of evidence-based medicine warn.
Despite the time, effort, and cost involved in performing trials, and the implicit contract that assures participants they are helping advance knowledge, the results of many trials are never published in peer-reviewed journals. In addition, clinical trial registriespotentially a comprehensive source of information about ongoing and completed trialsare incomplete and inconsistent, particularly for industry-sponsored trials.
Critics have pointed fingers at drug companies protection of commercial interests, journal editors bias in favor of studies with positive or newsworthy results, and investigators lack of time and interest. Whatever the cause, some experts agree that simple measures could help. Research ethics committees (RECs), they say, should require registration at the outset and publication at the conclusionand sponsors should require publication as part of the agreements with the investigators they fund.
In a study appearing in the March 22 Lancet, researchers at the Institut dInvestigacions Biomèdiques August Pi i Sunyer in Barcelona, Spain, tracked all 158 trials approved by their institutions REC in 1997. Three years later, 123 of the clinical trials were completed and only 38 of them (31%) had been published or were in press at peer-reviewed journals. "We believe that public dissemination of clinical research results is an important ethical requirement, and that RECs are in a privileged position, along with institutions, research funders, editors, and consumers, to ensure it," the authors wrote. "It is important that we begin to consider appropriate resource allocation that will permit RECs to do their work and ensure proper dissemination of results of clinical trials."
Sir Iain Chalmers, who wrote a commentary to accompany the Lancet article, is editor of The James Lind Library and one of the founders of the international Cochrane Collaboration, both of which have been established to promote informed health-care decision making. "Through their acquiescence in underreporting of research," Chalmers said in an interview, ethics committees are "guilty of gross dereliction of their duty to protect the interests of patients and the public."
Howard Mann, M.D., associate professor of radiology at the University of Utah, Salt Lake City, agreed that underreporting is a long-ignored problem. A member of his own institutions ethics committee, Mann proposed, in the Aug. 3, 2002 Lancet, a set of responsibilities for ethics committees in disseminating trial results.
"This imperative is not sufficiently addressed in national regulations or international codes of conduct such as the Declaration of Helsinki," Mann said in an interview. One exception is Canada, where the Natural Sciences and Engineering Research Council guidelines state that "research journalists, journal editors, members of editorial peer review boards, sponsors and regulators should address [underreporting of clinical trials] as an issue of scientific and ethical urgency."
"Every funding organization should ensure that the results of every trial it has fundedwhether completed or abandonedare publicly disseminated," said Mann. "Im not aware of any funding organization that has such a policy."
Kay Dickersin, Ph.D., director of the Center for Clinical Trials and Evidence-based Healthcare and the U.S. Cochrane Center at Brown University, Providence, R.I., said the Department of Defense breast cancer program stipulates in its contracts that investigators must publish, adding that "it should be part of any funding agreement."
Publication Bias
Publication bias appears to be a pervasive problem and it can seriously skew the outcome of systematic reviews. Some researchers see journal editors and peer reviewers as the main culprits, believing they favor studies with "positive" results suggesting efficacy of a new treatment. But others who have studied the issue say the larger problem is that many "negative" studies are never even submitted for publication.
In a study published in the June 5, 2002, Journal of the American Medical Association, Dickersin and colleagues looked for evidence of bias in accepting articles for publication in that journal, examining outcomes for manuscripts submitted between 1996 and 1999. The authors found no statistically significant difference between publication of positive or negative prospective interventional studies, although the results suggest a "tendency" toward positive publication bias, Dickersin said.
"The data show that investigators are the bulk of the problem," she said. In a meta-analysis published in the Dec. 31, 1993, Annals of the New York Academy of Sciences, Dickersin and Yuan I. Min followed up on 997 clinical trials to examine publication patterns. They found that investigators were about three times as likely to submit "significant" study results for publication compared with nonsignificant ones. In an analysis of controlled trials only, the difference was sixfold. "In every case, failure to publish was investigator-based, and not due to editorial decisions," the authors wrote.
Reasons given for not submitting a paper include lack of time, uninteresting results, and beliefs that the submission would be rejected by journals.
"Some investigators report that publication wasnt an aim for our study," Dickersin said. "That raises ethical issues: You cant do research and say publication is only an option. The research is as good as not done if you dont get the results out there."
Chalmers said many journal editors have become more aware of the need to avoid publication bias, and several have announced policies with this aim. And as he predicted in 1990, electronic publishing has removed the obstacle of limited journal space. With unlimited space online, Mann concluded, "There is no excuse for not making the results of all trials publicly accessible."
Trial Registries
Clinical trial registries are a key resource not only for physicians and patients seeking new treatment options, but also for researchers undertaking systematic reviews. Even if a trials results are unpublished, registration would ensure that reviewers know about it and could seek results from the investigators. But the hundreds of registries maintained by governments, hospitals, industry associations, and others are inconsistent in their coverage, and many industry trials are not included in any public registry.
In a study published in the Sept. 7, 2002, British Medical Journal, Eric Manheimer and Diana Anderson, researchers at Dickersins center at Brown, used industry sources to identify drugs in phase II or III trials for colon or prostate cancer. Then they searched public registries including ClinicalTrials.gov, CancerNet.gov (now Cancer.gov), and CenterWatch.com to see if the trials were listed.
They found substantial gaps in the public registries. ClinicalTrials.gov, the most comprehensive, contained listings for only seven of the 12 prostate cancer drugs and 10 of the 20 colon cancer drugs. Trials of three prostate cancer and three colon cancer drugs were listed on only one registry each, while three prostate cancer drugs and eight colon cancer drugs were not listed in any registry.
Mann noted that ethics boards are the logical choice to enforce registration. "To the extent that local RECs are overburdened, one can appreciate a reluctance to assume any additional oversight responsibilities," he said. However, "registration of all clinical trials should be mandated by legislative and/or enforceable administrative action. In the meantime, RECs should mandate this as a condition of approval."
In the United States, the Food and Drug Administration Modernization Act of 1997 requires that all trials testing efficacy of a treatment for serious or life-threatening conditions be included in a publicly accessible database. ClinicalTrials.gov was created to serve this purpose, and trial sponsors are supposed to submit informationincluding eligibility criteria, trial sites, and contact detailsto the database within 21 days after the trial begins.
In Europe, the situation varies by country, Chalmers said. Since the 1980s, Spain has required all trials to be registered, but the registry is not publicly accessible. Research governance directives in the United Kingdom require all clinical trials done within the National Health Service to be registered publicly.
"Quite good progress has been made in registering all non-commercially funded trials within the NHS, but very little progress in terms of commercially funded studies," Chalmers said. "In phase I and II trials, one can understand that drug companies may well want to keep confidential the details of how their new molecules are behaving. But by the time you get to phase III trials, everyone in the field knows about it anyway ... its not as if its a secret at that stage."
There are signs of progress toward openness on both sides of the Atlantic. Since 1997, U.K.-based GlaxoWellcome (now GlaxoSmithKline) has had a policy of providing public information on all its trials, although Chalmers noted that "the extent to which theyve kept up with that undertaking is open to question ... they actually find it quite difficult to know what trials theyre doing." And in April 2003, the Association of the British Pharmaceutical Industry announced the launch of a voluntary trials registry.
In March 2002, the U.S. FDA issued a "Guidance for Industry," explaining how drug companies are supposed to register trials to comply with the 1997 law. Dickersin said, however, that the Modernization Act is flawed in that the FDA lacks the authority to enforce the law and no funds were appropriated to develop, maintain, and ensure compliance with the law"two pretty major flaws."
Terry Toigo, director of FDAs Office of Special Health Issues, said the agency is analyzing data from a study to determine the extent of industry compliance. Letters were sent to sponsors of all commercial, non-NCI cancer protocols submitted to the agencys Center for Drug Evaluation and Research from January through September 2002, informing them of the guidance document and how to submit trial information. Of 127 industry-sponsored trials that met criteria for listing in ClinicalTrials.gov, only 61 (47%) of protocols had been submitted to the database.
In an interview, Toigo said the data will help senior FDA officials assess the need for further efforts to faciliate or perhaps to compel participation in ClinicalTrials.gov.
At TrialsCentral.org, Anderson and Dickersin maintain a "registry of registries." Because it is cumbersome, requiring users to search each registry separately, she described its most important use as being "a demonstration project to show how sad and ridiculous this situation is, rather than being of great use to patients."
"There should not be all this overlapping effort, with several hundred registries of trials out there," she said. "We need a single comprehensive registry, at least for the U.S., if not for the world." She cautioned that the logistical problems of creating a comprehensive registryavoiding duplication, determining which trials are actually testing efficacyare formidable.
"Those problems sound trivial, but they arent," she said. "Its just something were all going to have to figure out a way to work on together.... Its a good bet that if we required ethics committees to send all their trials to a central place, we could have a comprehensive registry."
This article has been cited by other articles in HighWire Press-hosted journals:
![]() |
||||
|
Oxford University Press Privacy Policy and Legal Statement |