Papers

Table of Contents
Full proceedings volume (PDF) / Bibliography (BibTeX)


pdf bib Front matter pages
Research Papers
pdf bib Document-Level Language Models for Machine Translation
Frithjof Petrick, Christian Herold, Pavel Petrushkov, Shahram Khadivi and Hermann Ney 		pp. 373‑389
pdf bib ChatGPT MT: Competitive for High- (but Not Low-) Resource Languages
Nathaniel Robinson, Perez Ogayo, David R. Mortensen and Graham Neubig 		pp. 390‑405
pdf bib Large Language Models Effectively Leverage Document-level Context for Literary Translation, but Critical Errors Persist
Marzena Karpinska and Mohit Iyyer 		pp. 406‑438
pdf bib Identifying Context-Dependent Translations for Evaluation Set Production
Rachel Wicks and Matt Post 		pp. 439‑465
pdf bib Machine Translation with Large Language Models: Prompting, Few-shot Learning, and Fine-tuning with QLoRA
Xuan Zhang, Navid Rajabi, Kevin Duh and Philipp Koehn 		pp. 466‑479
pdf bib Towards Effective Disambiguation for Machine Translation with Large Language Models
Vivek Iyer, Pinzhen Chen and Alexandra Birch 		pp. 480‑493
pdf bib A Closer Look at Transformer Attention for Multilingual Translation
Jingyi Zhang, Hongfei Xu, Kehai Chen and Gerard de Melo		 pp. 494‑504
pdf bib Bridging the Gap between Position-Based and Content-Based Self-Attention for Neural Machine Translation
Felix Schmidt and Mattia Di Gangi 		pp. 505‑519
pdf bib Visual Prediction Improves Zero-Shot Cross-Modal Machine Translation
Tosho Hirasawa, Emanuele Bugliarello, Desmond Elliott and Mamoru Komachi 		pp. 520‑533
pdf bib The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 Languages
Benjamin Muller, Belen Alastruey, Prangthip Hansanti, Elahe Kalbassi, Christophe Ropers, Eric Smith, Adina Williams, Luke Zettlemoyer, Pierre Andrews and Marta R. Costa-jussà 		pp. 534‑548
pdf bib Towards Better Evaluation for Formality-Controlled English-Japanese Machine Translation
Edison Marrese-Taylor, Pin Chen Wang and Yutaka Matsuo 		pp. 549‑558
pdf bib There’s No Data like Better Data: Using QE Metrics for MT Data Filtering
Jan-Thorsten Peter, David Vilar, Daniel Deutsch, Mara Finkelstein, Juraj Juraska and Markus Freitag 		pp. 559‑575
pdf bib Trained MT Metrics Learn to Cope with Machine-translated References
Jannis Vamvas, Tobias Domhan, Sony Trenous, Rico Sennrich and Eva Hasler 		pp. 981‑993
pdf bib Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph Level
Daniel Deutsch, Juraj Juraska, Mara Finkelstein and Markus Freitag 		pp. 994‑1011
pdf bib Automating Behavioral Testing in Machine Translation
Javier Ferrando, Matthias Sperber, Hendra Setiawan, Dominic Telaar and Saša Hasan 		pp. 1012‑1028
pdf bib One Wide Feedforward Is All You Need
Telmo Pires, António Vilarinho Lopes, Yannick Assogba and Hendra Setiawan 		pp. 1029‑1042
pdf bib A Benchmark for Evaluating Machine Translation Metrics on Dialects without Standard Orthography
Noëmi Aepli, Chantal Amrhein, Florian Schottmann and Rico Sennrich 		pp. 1043‑1063
pdf bib The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation
Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, Parker Riley, André Martins, Graham Neubig, Ankush Garg, Jonathan Clark, Markus Freitag and Orhan Firat 		pp. 1064‑1081
Shared Task Overview Papers
pdf bib Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet
Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Makoto Nagata, Toshiaki Nakazawa, Martin Popel, Maja Popović and Mariya Shmatova 		pp. 1‑42
pdf bib Findings of the WMT 2023 Biomedical Translation Shared Task: Evaluation of ChatGPT 3.5 as a Comparison System
Mariana Neves, Antonio Jimeno Yepes, Aurélie Névéol, Rachel Bawden, Giorgio Maria Di Nunzio, Roland Roller, Philippe Thomas, Federica Vezzani, Maika Vicente Navarro, Lana Yeganova, Dina Wiemann and Cristian Grozea 		pp. 43‑54
pdf bib Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMs
Longyue Wang, Zhaopeng Tu, Yan Gu, Siyou Liu, Dian Yu, Qingsong Ma, Chenyang Lyu, Liting Zhou, Chao-Hong Liu, Yufeng Ma, Weiyu Chen, Yvette Graham, Bonnie Webber, Philipp Koehn, Andy Way, Yulin Yuan and Shuming Shi 		pp. 55‑67
pdf bib Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23)
Mathias Müller, Malihe Alikhani, Eleftherios Avramidis, Richard Bowden, Annelies Braffort, Necati Cihan Camgöz, Sarah Ebling, Cristina España-Bonet, Anne Göhring, Roman Grundkiewicz, Mert Inan, Zifan Jiang, Oscar Koller, Amit Moryossef, Annette Rios, Dimitar Shterionov, Sandra Sidler-Miserez, Katja Tissi and Davy Van Landuyt 		pp. 68‑94
pdf bib Findings of the WMT 2023 Shared Task on Parallel Data Curation
Steve Sloto, Brian Thompson, Huda Khayrallah, Tobias Domhan, Thamme Gowda and Philipp Koehn 		pp. 95‑102
pdf bib Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent
Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frederic Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie and George Foster 		pp. 576‑626
pdf bib Findings of the WMT 2023 Shared Task on Quality Estimation
Frederic Blain, Chrysoula Zerva, Ricardo Rei, Nuno M. Guerreiro, Diptesh Kanojia, José G. C. de Souza, Beatriz Silva, Tânia Vaz, Yan Jingxuan, Fatemeh Azadi, Constantin Orasan and André Martins 		pp. 627‑651
pdf bib Findings of the Word-Level AutoCompletion Shared Task in WMT 2023
Lemao Liu, Francisco Casacuberta, George Foster, Guoping Huang, Philipp Koehn, Geza Kovacs, Shuming Shi, Taro Watanabe and Chengqing Zong 		pp. 652‑660
pdf bib Findings of the WMT 2023 Shared Task on Machine Translation with Terminologies
Kirill Semenov, Vilém Zouhar, Tom Kocmi, Dongdong Zhang, Wangchunshu Zhou and Yuchen Eleanor Jiang 		pp. 661‑669
pdf bib Findings of the WMT 2023 Shared Task on Automatic Post-Editing
Pushpak Bhattacharyya, Rajen Chatterjee, Markus Freitag, Diptesh Kanojia, Matteo Negri and Marco Turchi 		pp. 670‑679
pdf bib Findings of the WMT 2023 Shared Task on Low-Resource Indic Language Translation
Santanu Pal, Partha Pakray, Sahinur Rahman Laskar, Lenin Laitonjam, Vanlalmuansangi Khenglawt, Sunita Warjri, Pankaj Kundan Dadure and Sandeep Kumar Dash 		pp. 680‑692
General Translation Task
pdf bib Samsung R&D Institute Philippines at WMT 2023
Jan Christian Blaise Cruz 		pp. 103‑109
pdf bib NAIST-NICT WMT’23 General MT Task Submission
Hiroyuki Deguchi, Kenji Imamura, Yuto Nishida, Yusuke Sakai, Justin Vasselli and Taro Watanabe 		pp. 110‑118
pdf bib CUNI at WMT23 General Translation Task: MT and a Genetic Algorithm
Josef Jon, Martin Popel and Ondřej Bojar 		pp. 119‑127
pdf bib SKIM at WMT 2023 General Translation Task
Keito Kudo, Takumi Ito, Makoto Morishita and Jun Suzuki 		pp. 128‑136
pdf bib KYB General Machine Translation Systems for WMT23
Ben LI, Yoko Matsuzaki and Shivam Kalkar 		pp. 137‑142
pdf bib Yishu: Yishu at WMT2023 Translation Task
Luo Min, yixin tan and Qiulin Chen 		pp. 143‑149
pdf bib PROMT Systems for WMT23 Shared General Translation Task
Alexander Molchanov and Vladislav Kovalenko 		pp. 150‑154
pdf bib AIST AIRC Submissions to the WMT23 Shared Task
Matiss Rikters and Makoto Miwa 		pp. 155‑161
pdf bib MUNI-NLP Submission for Czech-Ukrainian Translation Task at WMT23
Pavel Rychly and Yuliia Teslia 		pp. 162‑165
pdf bib Exploring Prompt Engineering with GPT Language Models for Document-Level Machine Translation: Insights and Findings
Yangjian Wu and Gang Hu 		pp. 166‑169
pdf bib Treating General MT Shared Task as a Multi-Domain Adaptation Problem: HW-TSC’s Submission to the WMT23 General MT Shared Task
Zhanglin Wu, Daimeng Wei, Zongyao Li, Zhengzhe YU, Shaojun Li, Xiaoyu Chen, Hengchao Shang, Jiaxin GUO, Yuhao Xie, Lizhi Lei, Hao Yang and Yanfei Jiang 		pp. 170‑174
pdf bib UvA-MT’s Participation in the WMT 2023 General Translation Shared Task
Di Wu, Shaomu Tan, David Stap, Ali Araabi and Christof Monz 		pp. 175‑180
pdf bib Achieving State-of-the-Art Multilingual Translation Model with Minimal Data and Parameters
Hui Zeng 		pp. 181‑186
pdf bib IOL Research Machine Translation Systems for WMT23 General Machine Translation Shared Task
Wenbo Zhang 		pp. 187‑191
pdf bib GTCOM and DLUT’s Neural Machine Translation Systems for WMT23
Hao Zong 		pp. 192‑197
Test Suites
pdf bib RoCS-MT: Robustness Challenge Set for Machine Translation
Rachel Bawden and Benoît Sagot 		pp. 198‑216
pdf bib Multifaceted Challenge Set for Evaluating Machine Translation Performance
Xiaoyu Chen, Daimeng Wei, Zhanglin Wu, Ting Zhu, Hengchao Shang, Zongyao Li, Jiaxin GUO, Ning Xie, Lizhi Lei, Hao Yang and Yanfei Jiang 		pp. 217‑223
pdf bib Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT?
Shushen Manakhimova, Eleftherios Avramidis, Vivien Macketanz, Ekaterina Lapshinova-Koltunski, Sergei Bagdasarov and Sebastian Möller 		pp. 224‑245
pdf bib IIIT HYD’s Submission for WMT23 Test-suite Task
Ananya Mukherjee and Manish Shrivastava 		pp. 246‑251
pdf bib Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INES
Beatrice Savoldi, Marco Gaido, Matteo Negri and Luisa Bentivogli 		pp. 252‑262
Biomedical Translation Task
pdf bib Biomedical Parallel Sentence Retrieval Using Large Language Models
Sheema Firdous and Sadaf Abdul Rauf 		pp. 263‑270
pdf bib The Path to Continuous Domain Adaptation Improvements by HW-TSC for the WMT23 Biomedical Translation Shared Task
Zhanglin Wu, Daimeng Wei, Zongyao Li, Zhengzhe YU, Shaojun Li, Xiaoyu Chen, Hengchao Shang, Jiaxin GUO, Yuhao Xie, Lizhi Lei, Hao Yang and Yanfei Jiang 		pp. 271‑274
pdf bib Investigating Techniques for a Deeper Understanding of Neural Machine Translation (NMT) Systems through Data Filtering and Fine-tuning Strategies
Lichao Zhu, Maria Zimina, Maud Bénard, Behnoosh Namdar, Nicolas Ballier, Guillaume Wisniewski and Jean-Baptiste Yunès 		pp. 275‑281
Literary Translation Task
pdf bib MAX-ISI System at WMT23 Discourse-Level Literary Translation Task
Li An, Linghao Jin and Xuezhe Ma 		pp. 282‑286
pdf bib The MAKE-NMTVIZ System Description for the WMT23 Literary Task
Fabien Lopez, Gabriela González, Damien Hansen, Mariam Nakhle, Behnoosh Namdarzadeh, Nicolas Ballier, Marco Dinarelli, Emmanuelle Esperança-Rodier, Sui He, Sadaf Mohseni, Caroline Rossi, Didier Schwab, Jun Yang, Jean-Baptiste Yunès and Lichao Zhu 		pp. 287‑295
pdf bib DUTNLP System for the WMT2023 Discourse-Level Literary Translation
Anqi Zhao, Kaiyu Huang, Hao Yu and Degen Huang 		pp. 296‑301
pdf bib HW-TSC’s Submissions to the WMT23 Discourse-Level Literary Translation Shared Task
Yuhao Xie, Zongyao Li, Zhanglin Wu, Daimeng Wei, Xiaoyu Chen, Zhiqiang Rao, Shaojun Li, Hengchao Shang, Jiaxin GUO, Lizhi Lei, Hao Yang and Yanfei Jiang 		pp. 302‑306
pdf bib TJUNLP:System Description for the WMT23 Literary Task in Chinese to English Translation Direction
Shaolin Zhu and Deyi Xiong 		pp. 307‑311
African Languages Translation Task
pdf bib Machine Translation for Nko: Tools, Corpora, and Baseline Results
Moussa Doumbouya, Baba Mamadi Diané, Solo Farabado Cissé, Djibrila Diané, Abdoulaye Sow, Séré Moussa Doumbouya, Daouda Bangoura, Fodé Moriba Bayo, Ibrahima Sory 2. Conde, Kalo Mory Diané, Chris Piech and Christopher Manning 		pp. 312‑341
Sign Language Translation Task
pdf bib TTIC’s Submission to WMT-SLT 23
Marcelo Sandoval-Castaneda, Yanhong Li, Bowen Shi, Diane Brentari, Karen Livescu and Gregory Shakhnarovich 		pp. 342‑348
pdf bib KnowComp Submission for WMT23 Sign Language Translation Task
Baixuan Xu, Haochen Shi, Tianshi Zheng, Qing Zong, Weiqi Wang, Zhaowei Wang and Yangqiu Song 		pp. 349‑356
Parallel Data Curation Task
pdf bib A Fast Method to Filter Noisy Parallel Data WMT2023 Shared Task on Parallel Data Curation
Nguyen-Hoang Minh-Cong, Nguyen Van Vinh and Nguyen Le-Minh 		pp. 357‑363
pdf bib A Sentence Alignment Approach to Document Alignment and Multi-faceted Filtering for Curating Parallel Sentence Pairs from Web-crawled Data
Steinthor Steingrimsson 		pp. 364‑372
Metrics Task
pdf bib ACES: Translation Accuracy Challenge Sets at WMT 2023
Chantal Amrhein, Nikita Moghe and Liane Guillou 		pp. 693‑710
pdf bib Challenging the State-of-the-art Machine Translation Metrics from a Linguistic Perspective
Eleftherios Avramidis, Shushen Manakhimova, Vivien Macketanz and Sebastian Möller 		pp. 711‑727
pdf bib Tokengram_F, a Fast and Accurate Token-based chrF++ Derivative
Sören DREANO, Derek Molloy and Noel Murphy 		pp. 728‑735
pdf bib Embed_Llama: Using LLM Embeddings for the Metrics Shared Task
Sören DREANO, Derek Molloy and Noel Murphy 		pp. 736‑743
pdf bib eBLEU: Unexpectedly Good Machine Translation Evaluation Using Simple Word Embeddings
Muhammad ElNokrashy and Tom Kocmi 		pp. 744‑748
pdf bib Cometoid: Distilling Strong Reference-based Machine Translation Metrics into Even Stronger Quality Estimation Metrics
Thamme Gowda, Tom Kocmi and Marcin Junczys-Dowmunt 		pp. 749‑753
pdf bib MetricX-23: The Google Submission to the WMT 2023 Metrics Shared Task
Juraj Juraska, Mara Finkelstein, Daniel Deutsch, Aditya Siddhant, Mehdi Mirzazadeh and Markus Freitag 		pp. 754‑765
pdf bib GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4
Tom Kocmi and Christian Federmann 		pp. 766‑773
pdf bib Metric Score Landscape Challenge (MSLC23): Understanding Metrics’ Performance on a Wider Landscape of Translation Quality
Chi-kiu Lo, Samuel Larkin and Rebecca Knowles 		pp. 774‑797
pdf bib MEE4 and XLsim : IIIT HYD’s Submissions’ for WMT23 Metrics Shared Task
Ananya Mukherjee and Manish Shrivastava 		pp. 798‑803
pdf bib Quality Estimation Using Minimum Bayes Risk
Subhajit Naskar, Daniel Deutsch and Markus Freitag 		pp. 804‑809
pdf bib Evaluating Metrics for Document-context Evaluation in Machine Translation
Vikas Raunak, Tom Kocmi and Matt Post 		pp. 810‑812
pdf bib Semantically-Informed Regressive Encoder Score
Vasiliy Viskov, George Kokush, Daniil Larionov, Steffen Eger and Alexander Panchenko 		pp. 813‑819
pdf bib Empowering a Metric with LLM-assisted Named Entity Annotation: HW-TSC’s Submission to the WMT23 Metrics Shared Task
Zhanglin Wu, Yilun Liu, Min Zhang, Xiaofeng Zhao, Junhao Zhu, Ming Zhu, Xiaosong Qiao, Jingfei Zhang, Ma Miaomiao, Zhao Yanqing, Song Peng, shimin tao, Hao Yang and Yanfei Jiang 		pp. 820‑826
Quality Estimation Task
pdf bib Unify Word-level and Span-level Tasks: NJUNLP’s Participation for the WMT2023 Quality Estimation Shared Task
Xiang Geng, Zhejian Lai, Yu Zhang, shimin tao, Hao Yang, Jiajun CHEN and Shujian Huang 		pp. 827‑832
pdf bib HW-TSC 2023 Submission for the Quality Estimation Shared Task
Yuang Li, Chang Su, Ming Zhu, Mengyao Piao, Xinglin Lyu, Min Zhang and Hao Yang 		pp. 833‑838
pdf bib Scaling up CometKiwi: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task
Ricardo Rei, Nuno M. Guerreiro, José Pombal, Daan van Stigt, Marcos Treviso, Luisa Coheur, José G. C. de Souza and André Martins 		pp. 839‑846
pdf bib SurreyAI 2023 Submission for the Quality Estimation Shared Task
Archchana Sindhujan, Diptesh Kanojia, Constantin Orasan and Tharindu Ranasinghe 		pp. 847‑853
pdf bib MMT’s Submission for the WMT 2023 Quality Estimation Shared Task
Yulong Wu, Viktor Schlegel, Daniel Beck and Riza Batista-Navarro 		pp. 854‑860
pdf bib IOL Research’s Submission for WMT 2023 Quality Estimation Shared Task
ZEYU YAN 		pp. 861‑869
Word-Level Autocompletion Task
pdf bib SJTU-MTLAB’s Submission to the WMT23 Word-Level Auto Completion Task
Xingyu Chen and Rui Wang 		pp. 870‑874
pdf bib PRHLT’s Submission to WLAC 2023
Angel Navarro, Miguel Domingo and Francisco Casacuberta 		pp. 875‑879
pdf bib KnowComp Submission for WMT23 Word-Level AutoCompletion Task
Yi Wu, Haochen Shi, Weiqi Wang and Yangqiu Song 		pp. 880‑887
Terminology Translation Task
pdf bib Terminology-Aware Translation with Constrained Decoding and Large Language Model Prompting
Nikolay Bogoychev and Pinzhen Chen 		pp. 888‑894
pdf bib Lingua Custodia’s Participation at the WMT 2023 Terminology Shared Task
Jingshu Liu, Mariam Nakhlé, Gaëtan Caillout and Raheel Qadar 		pp. 895‑899
pdf bib Domain Terminology Integration into Machine Translation: Leveraging Large Language Models
Yasmin Moslem, Gianfranco Romani, Mahdi Molaei, John D. Kelleher, Rejwanul Haque and Andy Way 		pp. 900‑909
pdf bib OPUS-CAT Terminology Systems for the WMT23 Terminology Shared Task
Tommi Nieminen 		pp. 910‑916
pdf bib VARCO-MT: NCSOFT’s WMT’23 Terminology Shared Task Submission
Geon Woo Park, Junghwa Lee, Meiying Ren, Allison Shindell and Yeonsoo Lee 		pp. 917‑923
Automatic Post-Editing Task
pdf bib HW-TSC’s Participation in the WMT 2023 Automatic Post Editing Shared Task
Jiawei Yu, Min Zhang, Zhao Yanqing, Xiaofeng Zhao, Yuang Li, Su Chang, Yinglu Li, Ma Miaomiao, shimin tao and Hao Yang 		pp. 924‑928
Indic Languages Translation Task
pdf bib Neural Machine Translation for English - Manipuri and English - Assamese
Goutam Agrawal, Rituraj Das, Anupam Biswas and Dalton Meitei Thounaojam 		pp. 929‑932
pdf bib GUIT-NLP’s Submission to Shared Task: Low Resource Indic Language Translation
Mazida Ahmed, Kuwali Talukdar, Parvez Boruah, Prof. Shikhar Kumar Sarma and Kishore Kashyap 		pp. 933‑938
pdf bib NICT-AI4B’s Submission to the Indic MT Shared Task in WMT 2023
Raj Dabre, Jay Gala and Pranjal Chitale 		pp. 939‑947
pdf bib Machine Translation Advancements for Low-Resource Indian Languages in WMT23: CFILT-IITB’s Effort for Bridging the Gap
Pranav Gaikwad, Meet Doshi, Sourabh Deoghare and Pushpak Bhattacharyya 		pp. 948‑951
pdf bib Low-Resource Machine Translation Systems for Indic Languages
Ivana Kvapilíková and Ondřej Bojar 		pp. 952‑956
pdf bib MUNI-NLP Systems for Low-resource Indic Machine Translation
Edoardo Signoroni and Pavel Rychly 		pp. 957‑964
pdf bib NITS-CNLP Low-Resource Neural Machine Translation Systems of English-Manipuri Language Pair
Kshetrimayum Boynao Singh, Avichandra Singh Ningthoujam, Loitongbam Sanayai Meetei, Sivaji Bandyopadhyay and Thoudam Doren Singh 		pp. 965‑969
pdf bib IACS-LRILT: Machine Translation for Low-Resource Indic Languages
Dhairya Suman, Atanu Mandal, Santanu Pal and Sudip Naskar 		pp. 970‑975
pdf bib IOL Research Machine Translation Systems for WMT23 Low-Resource Indic Language Translation Shared Task
Wenbo Zhang 		pp. 976‑980

Last modified on November 13, 2023, 1:10 p.m.