EMNLP 2024

NINTH CONFERENCE ON
MACHINE TRANSLATION (WMT24)

November 12-13, 2024
Miami, Florida, USA
 
[HOME]
TRANSLATION TASKS: [GENERAL MT (NEWS)] [LOW-RESOURCE LANGUAGES OF SPAIN] [INDIC MT] [CHAT TASK] [BIOMEDICAL] [MULTIINDIC22MT TASK] [ENGLISH-TO-LOWRES MULTIMODAL MT TASK] [NON-REPETITIVE] [PATENT] [LITERARY]
EVALUATION TASKS: [METRICS TASK] [MT TEST SUITES] [QUALITY ESTIMATION]
OTHER TASKS: [OPEN LANGUAGE DATA INITIATIVE]

OVERVIEW

This shared task focuses on automatic methods for estimating the quality of neural machine translation output at run-time, without relying on reference translations. It will cover estimation at sentence and word levels, as well as critical error detection. This year, the critical error detection task will also include a subtask where participants will be encouraged to provide explanations for the detected errors, as well as, suggest corrections.

Languages covered

  • English to German (MQM)

  • English to Spanish (MQM)

  • English to Hindi (MQM, DA, Post-edits)

  • English to Gujarati (DA)

  • English to Telugu (DA)

  • English to Tamil (DA, Post-edits)

This year no training and validation datasets will be released. Instead, the participants will use the datasets from the previous year’s shared task available at wmt-qe-task.github.io/

Tasks organised

Task 1: Sentence-level Quality Estimation

Task 2: Fine grained error detection

Task 3: Context-aware Evaluation and Correction

DEADLINES (tentative)

Test set ready to download

July 2024

Submission deadline for QE task

July 2024

Paper submission deadline to WMT

TBA (follows EMNLP)

WMT Notification of acceptance

TBA (follows EMNLP)

WMT Camera-ready deadline

TBA (follows EMNLP)

Conference

12-13 November, 2024

All deadlines are in AoE (Anywhere on Earth). Dates are specified with respect to EMNLP 2024.

ORGANIZERS

  • Pushpak Bhattacharyya, Indian Institute of Technology Bombay, India

  • Frédéric Blain, Tilburg University, Netherlands

  • Rajen Chatterjee, Apple Inc, US

  • Sourabh Deoghare, Indian Institute of Technology Bombay, India

  • Catarina Farinha, Unbabel, Portugal

  • Markus Freitag, Google, US

  • Nuno M Guerreiro, Instituto de Telecomunicações, Unbabel, Instituto Superior Técnico, Centrale Supélec – Université Paris-Saclay

  • Diptesh Kanojia, University of Surrey, UK

  • Andre Martins: Instituto de Telecomunicações, Instituto Superior Técnico, Universidade de Lisboa (Lisbon ELLIS Unit), Unbabel

  • Matteo Negri, FBK, Italy

  • Constantin Orǎsan, University of Surrey, UK

  • Ricardo Rei, Unbabel, Portugal

  • José GC de Souza, Unbabel

  • Marco Turchi, Zoom, US

  • Chrysoula Zerva: Instituto de Telecomunicações, Instituto Superior Técnico, Universidade de Lisboa (Lisbon ELLIS Unit)

STAY IN TOUCH