Building a Pandemic Retrieval Test Collection

William Hersh, M.D.
Last updated: May 11, 2020


Latest Updates

The overall goals of this challenge evaluation are to:
Researchers, clinicians, and policy makers involved with the response to COVID-19 are constantly searching for reliable information on the virus and its impact. This presents a unique opportunity for the information retrieval (IR) and text processing communities to contribute to the response to this pandemic, as well as to study methods for quickly standing up such systems for similar future events.

The Allen Institute for AI (AI2) and collaborators have made availabile an open dataset, the COVID-19 Open Research Dataset (CORD-19). This collection of biomedical literature articles now contains over 50,000 articles and is updated weekly. AI2 has also created a search page for CORD-19, which has its own search engine and links to several others.
We are carrying out an IR challenge evaluation for search engines that find relevant COVID-related articles within this collection. This challenge will provide:
The challenge may in the future expand to more detailed tasks such as information-filtering, question-answering, fact-checking, and argument mining. The initial ad-hoc retrieval task will follow the "Cranfield" evaluation procedures that are used in the Text Retrieval Conference (TREC) and related challenge evaluations.

Communication about the challenge is being done through a Google Group. We are also disseminating information via Twitter, using the hashtag, #COVIDSearch.

Mentions about the project (and OHSU participation) include:
Participants in this project include:
Last updated: May 11 2020 by William Hersh