Shared Task: Biomedical Translation Task

Task description

This task aims to evaluate systems on the translation of documents from the biomedical domain. The test data will consist of biomedical abstracts and it will address the following language pairs:


ATTENTION!! We ask the participants to not download the Medline database by themselves in order to retrieve training data. Submissions that are derived from a model that was trained on the whole PubMed will be not considered in the evaluation.

Participants can rely on training (and development) data from various sources, for instance:

Participants are also free to use out-of-domain data.


Evaluation will be carried out both automatically and manually. Automatic evaluation will make use of standard machine translation metrics.

Native speakers of each of the languages will manually check the quality of the translation for a small sample of the submissions. If necessary, we also expect participants to support us in the manual evaluation (accordingly to the number of submissions).

We plan to release test sets for the following language pairs and sources:

Scientific abstracts:

For the test set of Medline abstracts, the format will be plain text files. The format will be the following:

(coming soon)

Submission Requirements

(coming soon)

Important dates

Release of test dataJune/July, 2024
Results submission deadlineJune/July, 2024
Paper submission deadlineTBA around 12th August (follows EMNLP)
Paper notificationTBA (follows EMNLP)
Camera-ready version dueTBA around 23rd September (follows EMNLP)
Conference EMNLP12-13 November, 2024

All deadlines are in AoE (Anywhere on Earth).


Rachel Bawden (University of Edinburgh, UK)
Giorgio Maria Di Nunzio (University of Padua, Italy)
Cristian Grozea (Fraunhofer Institute, Germany)
Antonio Jimeno Yepes (University of Melbourne, Australia)
Aurélie Névéol (Université Paris Saclay, CNRS, LISN, France)
Mariana Neves (German Federal Institute for Risk Assessment, Germany)
Roland Roller (DFKI, Germany)
Philippe Thomas (DFKI, Germany)
Federica Vezzani (University of Padua, Italy)
Maika Vicente Navarro, Maika Spanish Translator, Melbourne, Australia
Dina Wiemann (Novartis, Switzerland)
Lana Yeganova (NCBI/NLM/NIH, USA)

Please contact us in the mail Please join our discussion forum.