a Instituto do Radium, Campinas, São Paulo, Brazil.
b Present affiliation: H Lee Moffitt Cancer Center, Tampa, FL, USA.
c Universidade Federal de São Paulo/Escola Paulista de Medicina, São Paulo, Brazil.
Otavio AC Clark, H Lee Moffitt Cancer Center, BMT PROGRAM, 12902 Magnolia Drive, Tampa, FL 336129497, USA. E-mail: clark{at}evidencias.com
Abstract
Background An unbiased systematic review (SR) should analyse as many articles as possible in order to provide the best evidence available. However, many SR use only databases with high English-language content as sources for articles. Literatura Latino Americana e do Caribe em Ciências da Saúde (LILACS) indexes 670 journals from the Latin American and Caribbean health literature but is seldom used in these SR. Our objective is to evaluate if LILACS should be used as a routine source of articles for SR.
Methods First we identified SR published in 1997 in five medical journals with a high impact factor. Then we searched LILACS for articles that could match the inclusion criteria of these SR. We also checked if the authors had already identified these articles located in LILACS.
Results In all, 64 SR were identified. Two had already searched LILACS and were excluded. In 39 of 62 (63%) SR a LILACS search identified articles that matched the inclusion criteria. In 5 (8%) our search was inconclusive and in 18 (29%) no articles were found in LILACS. Therefore, in 71% (44/72) of cases, a LILACS search could have been useful to the authors. This proportion remains the same if we consider only the 37 SR that performed a meta-analysis. In only one case had the article identified in LILACS already been located elsewhere by the authors' strategy.
Conclusion LILACS is an under-explored and unique source of articles whose use can improve the quality of systematic reviews. This database should be used as a routine source to identify studies for systematic reviews.
Keywords Bibliographic databases, information systems, bias, reviews, epidemiologic research design, meta-analysis, online systems
Accepted 2 October 2001
An unbiased systematic review (SR) should include all relevant articles in order to offer the best evidence available.1 However, some SR use a narrow search strategy for articles, limiting their search to English literature or to databases like MEDLINE and EMBASE2 that index mostly journals from the developed countries.
In 1995, Gibbs asserted that there is a lost science in the developing countries, largely ignored in developed countries because of (1) the lack of an organized way to assess its scientific production and (2) the low impact factor of the rare indexed periodicals edited in these countries.3
Latin American and Caribbean medical literature is a part of this lost science, but there is an organized database in health science and biological literature, with a potential role in the identification of articles for SR.
Literatura Latino Americana e do Caribe em Ciências da Saúde (LILACS) is a database produced by Biblioteca Regional de Medicina (BIREME) and the Pan-American Health Organization (PAHO) at the Latin American and Caribbean Health Sciences Information Center in São Paulo, Brazil, since 1982.4 In the year of 2000, 670 journals were indexed. LILACS is available in CD-ROM and on the internet, free of charge.4 There are interfaces in three languages (Portuguese, Spanish and English). Most of the journals indexed are published in Portuguese or Spanish. All titles and some abstracts are translated into English.
Despite its broad reach, the LILACS database is seldom used as a source of articles in SR. In 1998, only 3 of 50 collaborative review groups in the Cochrane Collaboration used this database.
If LILACS contains articles that could be included in SR it should be used as a routine source of articles for SR.
Methods
We performed a search in MEDLINE, according to the strategy described by McKibbon,5 to locate the SR published in 1997 in five medical journals with high impact factors: Annals of Internal Medicine (Annals), British Medical Journal (BMJ), Journal of the American Medical Association (JAMA), The Lancet (LANCET) and The New England Journal of Medicine (NEJM).
All records retrieved were assessed in full text. We included studies that used any systematic way of searching literature, with explicit criteria of selection of articles. Those SR that had used LILACS were excluded.
The following data were extracted: first author, journal, databases searched, languages included, inclusion criteria and if a meta-analysis was performed.
For each included SR, we searched LILACS (30th edition on CD-ROM, January,1998) to locate articles that matched the SR's inclusion criteria. The LILACS search was classified as positive, if we found at least one article that could fit the inclusion criteria; as negative if we located no such articles, and as inconclusive if we found at least one article that may or may not have been included, depending on additional information. For LILACS searches that were positive or inconclusive, we checked the references listed in the SR to verify if the identified articles had already been located. Two of the authors extracted data blinded to the results of the other. Discrepancies were resolved by reaching a consensus.
Results
We located 64 SR (the complete list is available from the authors). Two had performed a LILACS search and were excluded (Figure 1).
|
Of the SR, 37% (23/62) limited their search to English-language articles; in 32% (20/62) we could not determine if language restrictions were used. Only 29% (18/62) explicitly had no language restrictions. Meta-analysis was performed in 60% (37/62) of the SR.
MEDLINE was searched in 95% (59/62) of the SR and in 76% (47/62) it was the only database searched. EMBASE was searched in 16% (10/62) and other databases in 14% (9/62). Of the SR, 5% (3/62) did not mention the databases searched.
The LILACS search was positive in 63% (39/62) of them, inconclusive in 8% (5/62) and negative in 29% (18/62) (Figure 1).
Among those 44 SR with a positive or inconclusive LILACS search, only one had located the article identified in LILACS by other methods. Therefore, in 70% of the cases (43/62) a LILACS search was effective in identifying new articles suitable for inclusion and not located by authors. This percentage remains the same if we consider only the 37 SR that performed meta-analysis. In this subgroup, a LILACS search was positive in 60% (22/37), inconclusive in 11% (4/37) and negative in 30% (11/37).
Discussion
Systematic reviews must locate all suitable articles to offer the best evidence to the reader.1 Unfortunately some SR use a narrow search strategy making them subject to bias.2 As we have shown, a search in the lost science can offer useful articles for SR.
In our study, only 2 of 64 SR (3%) had searched LILACS, but in 70% (44/62) of them a search in this database could have identified articles potentially relevant to the review.
The Brazilian Cochrane Center6 and BIREME4 have been working in an effort to facilitate access to and searching of LILACS. A highly sensitive search strategy for clinical trials adapted to cover three languages was developed7 and free access to LILACS via the internet is available.4 A program to re-tag randomized studies is also ongoing.
Concerns regarding the quality of publications in Portuguese or Spanish can arise, but a previous study found no significant qualitative differences among trials published in English or non-English languages,2 Spanish included.
This study demonstrates that LILACS could have added information to 70% of SR surveyed and that these articles could not be identified by other search methods.
In conclusion, LILACS is an under-explored and unique source of articles, its use can improve the quality of SR. This database should be used as a routine source of studies in the preparation of SR.
KEY MESSAGES
|
Acknowledgments
Source of support: Universidade Federal de São Paulo, São Paulo, SP-Brazil. The authors also thank Dr Benjamin Djulbegovic for the critical review of the article.
References
1
Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997;126: 37680.
2 Moher D, Fortin P, Jadad AR et al. Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet 1996;347: 36366.[ISI][Medline]
3 Gibbs W. Lost Science in the Third World. Scientific American 1995;273: 7683.[Medline]
4 Available from: URL: www.bireme.br [Accessed in July 2001].
5 McKibbon KA, Walker-Dilks CF, Wilczynski NL, Haynes RB. Beyond ACP Journal Club: how to harness MEDLINE for review articles. ACP J Club 1996;124: A1213.
6 Available from: URL: www.centrocochranedobrasil.org.br [Accessed in July 2001].
7 Castro AA, Clark OA, Atallah AN. Optimal search strategy for clinical trials in the Latin American and Caribbean Health Science Literature database (LILACS database): update. São Paulo Med J 1999;117: 13839.