Syntax Based Metrics

Syntax based metrics measure how well automatic translations should match the syntactic relationships in human reference translations.

Syntax Based Metrics is the main subject of 40 publications. 5 are discussed here.

Topics in Evaluation

Publications

Metrics may account for syntactic information

Liu, Ding and Gildea, Daniel (2005): Syntactic Features for Evaluation of Machine Translation, Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

(Liu and Gildea, 2005). A more syntactic approach to machine translation may consider the matching of dependency structure of reference and system output

Owczarzak, Karolina and van Genabith, Josef and Way, Andy (2007): Dependency-Based Automatic Evaluation for Machine Translation, Proceedings of SSST, NAACL-HLT 2007 / AMTA Workshop on Syntax and Structure in Statistical Translation

(Owczarzak et al., 2007), possibly including dependency labels

Owczarzak, Karolina and van Genabith, Josef and Way, Andy (2007): Labelled Dependencies in Machine Translation Evaluation, Proceedings of the Second Workshop on Statistical Machine Translation

(Owczarzak et al., 2007). A wide range of linguistic features for evaluation is explored by

Giménez, Jesús and Màrquez, Lluís (2007): Linguistic Features for Automatic Evaluation of Heterogenous MT Systems, Proceedings of the Second Workshop on Statistical Machine Translation

Giménez and Màrquez (2007). A method based on latent semantic analysis has be proposed for machine translation evaluation

Florence Reeder (2006): Measuring MT Adequacy Using Latent Semantic Analysis, 5th Conference of the Association for Machine Translation in the Americas (AMTA)

(Reeder, 2006).

Benchmarks

Discussion

New Publications

McCaffery, Martin and Nederhof, Mark-Jan (2016): DTED: Evaluation of Machine Translation Structure Using Dependency Parsing and Tree Edit Distance, Proceedings of the First Conference on Machine Translation
add
@InProceedings{mccaffery-nederhof:2016:WMT,
author = {McCaffery, Martin and Nederhof, Mark-Jan},
title = {DTED: Evaluation of Machine Translation Structure Using Dependency Parsing and Tree Edit Distance},
booktitle = {Proceedings of the First Conference on Machine Translation},
month = {August},
address = {Berlin, Germany},
publisher = {Association for Computational Linguistics},
pages = {491--498},
url = {http://www.aclweb.org/anthology/W/W16/W16-2340},
year = 2016
}
McCaffery and Nederhof (2016)
Isozaki, Hideki and Kouchi, Natsume (2015): Dependency Analysis of Scrambled References for Better Evaluation of Japanese Translation, Proceedings of the Tenth Workshop on Statistical Machine Translation
add
@InProceedings{isozaki-kouchi:2015:WMT,
author = {Isozaki, Hideki and Kouchi, Natsume},
title = {Dependency Analysis of Scrambled References for Better Evaluation of {Japanese} Translation},
booktitle = {Proceedings of the Tenth Workshop on Statistical Machine Translation},
month = {September},
address = {Lisbon, Portugal},
publisher = {Association for Computational Linguistics},
pages = {450--456},
url = {http://aclweb.org/anthology/W15-3058},
year = 2015
}
Isozaki and Kouchi (2015)
Li, Liangyou and Yu, Hui and Liu, Qun (2015): MT Tuning on RED: A Dependency-Based Evaluation Metric, Proceedings of the Tenth Workshop on Statistical Machine Translation
add
@InProceedings{li-yu-liu:2015:WMT,
author = {Li, Liangyou and Yu, Hui and Liu, Qun},
title = {MT Tuning on RED: A Dependency-Based Evaluation Metric},
booktitle = {Proceedings of the Tenth Workshop on Statistical Machine Translation},
month = {September},
address = {Lisbon, Portugal},
publisher = {Association for Computational Linguistics},
pages = {428--433},
url = {http://aclweb.org/anthology/W15-3055},
year = 2015
}
Li et al. (2015)
Comelles, Elisabet and Atserias, Jordi (2015): VERTa: a Linguistically-motivated Metric at the WMT15 Metrics Task, Proceedings of the Tenth Workshop on Statistical Machine Translation
add
@InProceedings{comelles-atserias:2015:WMT,
author = {Comelles, Elisabet and Atserias, Jordi},
title = {VERTa: a Linguistically-motivated Metric at the WMT15 Metrics Task},
booktitle = {Proceedings of the Tenth Workshop on Statistical Machine Translation},
month = {September},
address = {Lisbon, Portugal},
publisher = {Association for Computational Linguistics},
pages = {366--372},
url = {http://aclweb.org/anthology/W15-3045},
year = 2015
}
Comelles and Atserias (2015)
Barančíková, Petra and Rosa, Rudolf (2015): Targeted Paraphrasing on Deep Syntactic Layer for MT Evaluation, Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015)
add
@InProceedings{baranvcikova-rosa:2015:Depling,
author = {Baran\v{c}\'{\i}kov\'{a}, Petra and Rosa, Rudolf},
title = {Targeted Paraphrasing on Deep Syntactic Layer for {MT} Evaluation},
booktitle = {Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015)},
month = {August},
address = {Uppsala, Sweden},
publisher = {Uppsala University, Uppsala, Sweden},
pages = {20--27},
url = {http://www.aclweb.org/anthology/W15-2104},
year = 2015
}
Barančíková and Rosa (2015)
Lo, Chi-kiu and Addanki, Karteek and Saers, Markus and Wu, Dekai (2013): Improving machine translation by training against an automatic semantic frame based evaluation metric, Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
add
@InProceedings{lo-EtAl:2013:Short,
author = {Lo, Chi-kiu and Addanki, Karteek and Saers, Markus and Wu, Dekai},
title = {Improving machine translation by training against an automatic semantic frame based evaluation metric},
booktitle = {Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
month = {August},
address = {Sofia, Bulgaria},
publisher = {Association for Computational Linguistics},
pages = {375--381},
url = {http://www.aclweb.org/anthology/P13-2067},
year = 2013
}
Lo et al. (2013)
Chuchunkov, Alexander and Tarelkin, Alexander and Galinskaya, Irina (2014): Applying HMEANT to English-Russian Translations, Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation
add
@InProceedings{chuchunkov-tarelkin-galinskaya:2014:SSST-8,
author = {Chuchunkov, Alexander and Tarelkin, Alexander and Galinskaya, Irina},
title = {Applying HMEANT to English-Russian Translations},
booktitle = {Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation},
month = {October},
address = {Doha, Qatar},
publisher = {Association for Computational Linguistics},
pages = {43--50},
url = {http://www.aclweb.org/anthology/W14-4005},
year = 2014
}
Chuchunkov et al. (2014)
Wu, Dekai and Lo, Chi-kiu and Beloucif, Meriem and Saers, Markus (2014): Better Semantic Frame Based MT Evaluation via Inversion Transduction Grammars, Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation
add
@InProceedings{wu-EtAl:2014:SSST-8,
author = {Wu, Dekai and Lo, Chi-kiu and Beloucif, Meriem and Saers, Markus},
title = {Better Semantic Frame Based {MT} Evaluation via Inversion Transduction Grammars},
booktitle = {Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation},
month = {October},
address = {Doha, Qatar},
publisher = {Association for Computational Linguistics},
pages = {22--33},
url = {http://www.aclweb.org/anthology/W14-4003},
year = 2014
}
Wu et al. (2014)
Isozaki, Hideki and Kouchi, Natsume and Hirao, Tsutomu (2014): Dependency-based Automatic Enumeration of Semantically Equivalent Word Orders for Evaluating Japanese Translations, Proceedings of the Ninth Workshop on Statistical Machine Translation
add
@InProceedings{isozaki-kouchi-hirao:2014:W14-33,
author = {Isozaki, Hideki and Kouchi, Natsume and Hirao, Tsutomu},
title = {Dependency-based Automatic Enumeration of Semantically Equivalent Word Orders for Evaluating {Japanese} Translations},
booktitle = {Proceedings of the Ninth Workshop on Statistical Machine Translation},
month = {June},
address = {Baltimore, Maryland, USA},
publisher = {Association for Computational Linguistics},
pages = {287--292},
url = {http://www.aclweb.org/anthology/W14-3335},
year = 2014
}
Isozaki et al. (2014)
Meriem Beloucif and Chi Lo-kiu and Dekai Wu (2014): Improving MEANT Based Semantically Tuned SMT, Proceedings of the International Workshop on Spoken Language Translation (IWSLT)
add
@inproceedings{Beloucif:iwslt:2014,
author = {Meriem Beloucif and Chi Lo-kiu and Dekai Wu},
title = {Improving {MEANT} Based Semantically Tuned {SMT}},
pages = {34--41},
booktitle = {Proceedings of the International Workshop on Spoken Language Translation (IWSLT)},
year = 2014
}
Beloucif et al. (2014)
Joty, Shafiq and Guzmán, Francisco and Màrquez, Lluís and Nakov, Preslav (2014): DiscoTK: Using Discourse Structure for Machine Translation Evaluation, Proceedings of the Ninth Workshop on Statistical Machine Translation
add
@InProceedings{joty-EtAl:2014:W14-33,
author = {Joty, Shafiq and Guzm\'{a}n, Francisco and M\`{a}rquez, Llu\'{i}s and Nakov, Preslav},
title = {DiscoTK: Using Discourse Structure for Machine Translation Evaluation},
booktitle = {Proceedings of the Ninth Workshop on Statistical Machine Translation},
month = {June},
address = {Baltimore, Maryland, USA},
publisher = {Association for Computational Linguistics},
pages = {402--408},
url = {http://www.aclweb.org/anthology/W14-3352},
year = 2014
}
Joty et al. (2014)
Echizen'ya, Hiroshi and Araki, Kenji and Hovy, Eduard (2014): Application of Prize based on Sentence Length in Chunk-based Automatic Evaluation of Machine Translation, Proceedings of the Ninth Workshop on Statistical Machine Translation
add
@InProceedings{echizenya-araki-hovy:2014:W14-33,
author = {Echizen'ya, Hiroshi and Araki, Kenji and Hovy, Eduard},
title = {Application of Prize based on Sentence Length in Chunk-based Automatic Evaluation of Machine Translation},
booktitle = {Proceedings of the Ninth Workshop on Statistical Machine Translation},
month = {June},
address = {Baltimore, Maryland, USA},
publisher = {Association for Computational Linguistics},
pages = {381--386},
url = {http://www.aclweb.org/anthology/W14-3349},
year = 2014
}
Echizen'ya et al. (2014)
Guzmán, Francisco and Joty, Shafiq and Màrquez, Lluís and Nakov, Preslav (2014): Using Discourse Structure Improves Machine Translation Evaluation, Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
add
@InProceedings{guzman-EtAl:2014:P14-1,
author = {Guzm\'{a}n, Francisco and Joty, Shafiq and M\`{a}rquez, Llu\'{i}s and Nakov, Preslav},
title = {Using Discourse Structure Improves Machine Translation Evaluation},
booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
month = {June},
address = {Baltimore, Maryland},
publisher = {Association for Computational Linguistics},
pages = {687--698},
url = {http://www.aclweb.org/anthology/P14-1065},
year = 2014
}
Guzmán et al. (2014)
Lo, Chi-kiu and Beloucif, Meriem and Saers, Markus and Wu, Dekai (2014): XMEANT: Better semantic MT evaluation without reference translations, Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
add
@InProceedings{lo-EtAl:2014:P14-2,
author = {Lo, Chi-kiu and Beloucif, Meriem and Saers, Markus and Wu, Dekai},
title = {XMEANT: Better semantic {MT} evaluation without reference translations},
booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
month = {June},
address = {Baltimore, Maryland},
publisher = {Association for Computational Linguistics},
pages = {765--771},
url = {http://www.aclweb.org/anthology/P14-2124},
year = 2014
}
Lo et al. (2014)
Yu, Hui and Wu, Xiaofeng and Xie, Jun and Jiang, Wenbin and Liu, Qun and Lin, Shouxun (2014): RED: A Reference Dependency Based MT Evaluation Metric, Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers
add
@InProceedings{yu-EtAl:2014:Coling3,
author = {Yu, Hui and Wu, Xiaofeng and Xie, Jun and Jiang, Wenbin and Liu, Qun and Lin, Shouxun},
title = {RED: A Reference Dependency Based {MT} Evaluation Metric},
booktitle = {Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers},
month = {August},
address = {Dublin, Ireland},
publisher = {Dublin City University and Association for Computational Linguistics},
pages = {2042--2051},
url = {http://www.aclweb.org/anthology/C14-1193},
year = 2014
}
Yu et al. (2014)
Lo, Chi-kiu and Wu, Dekai (2013): MEANT at WMT 2013: A Tunable, Accurate yet Inexpensive Semantic Frame Based MT Evaluation Metric, Proceedings of the Eighth Workshop on Statistical Machine Translation
add
@InProceedings{lo-wu:2013:WMT,
author = {Lo, Chi-kiu and Wu, Dekai},
title = {{MEANT} at {WMT} 2013: A Tunable, Accurate yet Inexpensive Semantic Frame Based {MT} Evaluation Metric},
booktitle = {Proceedings of the Eighth Workshop on Statistical Machine Translation},
month = {August},
address = {Sofia, Bulgaria},
publisher = {Association for Computational Linguistics},
pages = {422--428},
url = {http://www.aclweb.org/anthology/W13-2254},
year = 2013
}
Lo and Wu (2013)
Jesús Giménez and Lluís Márquez (2010): Linguistic measures for automatic machine translation evaluation, Machine Translation
add
@article{MTJ:2010:Gimenez,
author = {Jes{\'u}s Gim{\'e}nez and Llu{\'i}s M{\'a}rquez},
title = {Linguistic measures for automatic machine translation evaluation},
pages = {209-240},
journal = {Machine Translation},
volume = {24},
number = {3-4},
month = {December},
year = 2010
}
Giménez and Márquez (2010)
Bojar, Ondrej and Wu, Dekai (2012): Towards a Predicate-Argument Evaluation for MT, Proceedings of the Sixth Workshop on Syntax, Semantics and Structure in Statistical Translation
add
@InProceedings{bojar-wu:2012:SSST-6,
author = {Bojar, Ondrej and Wu, Dekai},
title = {Towards a Predicate-Argument Evaluation for MT},
booktitle = {Proceedings of the Sixth Workshop on Syntax, Semantics and Structure in Statistical Translation},
month = {July},
address = {Jeju, Republic of Korea},
publisher = {Association for Computational Linguistics},
pages = {30--38},
url = {http://www.aclweb.org/anthology/W12-4204},
year = 2012
}
Bojar and Wu (2012)
Giménez, Jesús and Màrquez, Lluís (2009): On the Robustness of Syntactic and Semantic Features for Automatic MT Evaluation, Proceedings of the Fourth Workshop on Statistical Machine Translation
add
@InProceedings{gimenez-marquez:2009:WMT-09,
author = {Gim{\'e}nez, Jes{\'u}s and M{\`a}rquez, Llu{\'i}s},
title = {On the Robustness of Syntactic and Semantic Features for Automatic {MT} Evaluation},
booktitle = {Proceedings of the Fourth Workshop on Statistical Machine Translation},
month = {March},
address = {Athens, Greece},
publisher = {Association for Computational Linguistics},
pages = {250--258},
url = {http://www.aclweb.org/anthology/W/W09/W09-0440},
year = 2009
}
Giménez and Màrquez (2009)
Pado, Sebastian and Galley, Michel and Jurafsky, Daniel and Manning, Christopher D. (2009): Machine Translation Evaluation with Textual Entailment Features, Proceedings of the Fourth Workshop on Statistical Machine Translation
add
@InProceedings{pado-EtAl:2009:WMT-09,
author = {Pado, Sebastian and Galley, Michel and Jurafsky, Daniel and Manning, Christopher D.},
title = {Machine Translation Evaluation with Textual Entailment Features},
booktitle = {Proceedings of the Fourth Workshop on Statistical Machine Translation},
month = {March},
address = {Athens, Greece},
publisher = {Association for Computational Linguistics},
pages = {37--41},
url = {http://www.aclweb.org/anthology/W/W09/W09-0404},
year = 2009
}
Pado et al. (2009)
Popovic, Maja and Ney, Hermann (2009): Syntax-Oriented Evaluation Measures for Machine Translation Output, Proceedings of the Fourth Workshop on Statistical Machine Translation
add
@InProceedings{popovic-ney:2009:WMT-09,
author = {Popovic, Maja and Ney, Hermann},
title = {Syntax-Oriented Evaluation Measures for Machine Translation Output},
booktitle = {Proceedings of the Fourth Workshop on Statistical Machine Translation},
month = {March},
address = {Athens, Greece},
publisher = {Association for Computational Linguistics},
pages = {29--32},
url = {http://www.aclweb.org/anthology/W/W09/W09-0402},
year = 2009
}
Popovic and Ney (2009)
Amigó, Enrique and Giménez, Jesús and Gonzalo, Julio and Verdejo, Felisa (2009): The Contribution of Linguistic Features to Automatic Machine Translation Evaluation, Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP
add
@InProceedings{amigo-EtAl:2009:ACLIJCNLP,
author = {Amig\'{o}, Enrique and Gim\'{e}nez, Jes\'{u}s and Gonzalo, Julio and Verdejo, Felisa},
title = {The Contribution of Linguistic Features to Automatic Machine Translation Evaluation},
booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
month = {August},
address = {Suntec, Singapore},
publisher = {Association for Computational Linguistics},
pages = {306--314},
url = {http://www.aclweb.org/anthology/P/P09/P09-1035},
year = 2009
}
Amigó et al. (2009)
Pado, Sebastian and Galley, Michel and Jurafsky, Dan and Manning, Christopher D. (2009): Robust Machine Translation Evaluation with Entailment Features, Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP
add
@InProceedings{pado-EtAl:2009:ACLIJCNLP,
author = {Pado, Sebastian and Galley, Michel and Jurafsky, Dan and Manning, Christopher D.},
title = {Robust Machine Translation Evaluation with Entailment Features},
booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
month = {August},
address = {Suntec, Singapore},
publisher = {Association for Computational Linguistics},
pages = {297--305},
url = {http://www.aclweb.org/anthology/P/P09/P09-1034},
year = 2009
}
Pado et al. (2009)
Wang, Bo and Zhao, Tiejun and Yang, Muyun and Li, Sheng (2009): References Extension for the Automatic Evaluation of MT by Syntactic Hybridization, Proceedings of the Third Workshop on Syntax and Structure in Statistical Translation (SSST-3) at NAACL HLT 2009
add
@InProceedings{wang-EtAl:2009:SSST,
author = {Wang, Bo and Zhao, Tiejun and Yang, Muyun and Li, Sheng},
title = {References Extension for the Automatic Evaluation of {MT} by Syntactic Hybridization},
booktitle = {Proceedings of the Third Workshop on Syntax and Structure in Statistical Translation (SSST-3) at NAACL HLT 2009},
month = {June},
address = {Boulder, Colorado},
publisher = {Association for Computational Linguistics},
pages = {37--44},
url = {http://www.aclweb.org/anthology/W09-2305},
year = 2009
}
Wang et al. (2009)
Echizen-ya, Hiroshi and Araki, Kenji (2010): Automatic Evaluation Method for Machine Translation Using Noun-Phrase Chunking, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
add
@InProceedings{echizenya-araki:2010:ACL,
author = {Echizen-ya, Hiroshi and Araki, Kenji},
title = {Automatic Evaluation Method for Machine Translation Using Noun-Phrase Chunking},
booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
month = {July},
address = {Uppsala, Sweden},
publisher = {Association for Computational Linguistics},
pages = {108--117},
url = {http://www.aclweb.org/anthology/P10-1012},
year = 2010
}
Echizen-ya and Araki (2010)
Xiong, Deyi and Zhang, Min and Li, Haizhou (2010): Error Detection for Statistical Machine Translation Using Linguistic Features, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
add
@InProceedings{xiong-zhang-li:2010:ACL,
author = {Xiong, Deyi and Zhang, Min and Li, Haizhou},
title = {Error Detection for Statistical Machine Translation Using Linguistic Features},
booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
month = {July},
address = {Uppsala, Sweden},
publisher = {Association for Computational Linguistics},
pages = {604--611},
url = {http://www.aclweb.org/anthology/P10-1062},
year = 2010
}
Xiong et al. (2010)
Lo, Chi-kiu and Wu, Dekai (2011): Structured vs. Flat Semantic Role Representations for Machine Translation Evaluation, Proceedings of Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation
add
@InProceedings{lo-wu:2011:SSST,
author = {Lo, Chi-kiu and Wu, Dekai},
title = {Structured vs. Flat Semantic Role Representations for Machine Translation Evaluation},
booktitle = {Proceedings of Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation},
month = {June},
address = {Portland, Oregon, USA},
publisher = {Association for Computational Linguistics},
pages = {10--20},
url = {http://www.aclweb.org/anthology/W11-1002},
year = 2011
}
Lo and Wu (2011)
Pighin, Daniele and Màrquez, Lluís (2011): Automatic Projection of Semantic Structures: an Application to Pairwise Translation Ranking, Proceedings of Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation
add
@InProceedings{pighin-marquez:2011:SSST,
author = {Pighin, Daniele and M\`{a}rquez, Llu\'{i}s},
title = {Automatic Projection of Semantic Structures: an Application to Pairwise Translation Ranking},
booktitle = {Proceedings of Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation},
month = {June},
address = {Portland, Oregon, USA},
publisher = {Association for Computational Linguistics},
pages = {1--9},
url = {http://www.aclweb.org/anthology/W11-1001},
year = 2011
}
Pighin and Màrquez (2011)
He, Yifan and Du, Jinhua and Way, Andy and van Genabith, Josef (2010): The DCU Dependency-Based Metric in WMT-MetricsMATR 2010, Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
add
@InProceedings{he-EtAl:2010:WMT,
author = {He, Yifan and Du, Jinhua and Way, Andy and van Genabith, Josef},
title = {The DCU Dependency-Based Metric in WMT-MetricsMATR 2010},
booktitle = {Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR},
month = {July},
address = {Uppsala, Sweden},
publisher = {Association for Computational Linguistics},
pages = {349--353},
url = {http://www.aclweb.org/anthology/W10-1753},
year = 2010
}
He et al. (2010)
Sebastian Padó and Daniel Cer and Michel Galley and Dan Jurafsky and Christopher D. Manning (2009): Measuring machine translation quality as semantic equivalence: A metric based on entailment features, Machine Translation
add
@article{MTJ:2009:Pado,
author = {Sebastian Pad\'{o} and Daniel Cer and Michel Galley and Dan Jurafsky and Christopher D. Manning},
title = {Measuring machine translation quality as semantic equivalence: A metric based on entailment features},
url = {http://www.researchgate.net/publication/220419015\_Measuring\_machine\_translation\_quality\_as\_semantic\_equivalence\_A\_metric\_based\_on\_entailment\_features/file/d912f50bcabcb3781f.pdf},
googlescholar = {6607510070651316706},
pages = {181--193},
journal = {Machine Translation},
volume = {23},
number = {2--3},
month = {September},
year = 2009
}
Padó et al. (2009)
Lo, Chi-kiu and Wu, Dekai (2010): Semantic vs. Syntactic vs. N-gram Structure for Machine Translation Evaluation, Proceedings of the 4th Workshop on Syntax and Structure in Statistical Translation
add
@InProceedings{lo-wu:2010:SSST,
author = {Lo, Chi-kiu and Wu, Dekai},
title = {Semantic vs. Syntactic vs. N-gram Structure for Machine Translation Evaluation},
booktitle = {Proceedings of the 4th Workshop on Syntax and Structure in Statistical Translation},
month = {August},
address = {Beijing, China},
publisher = {Coling 2010 Organizing Committee},
pages = {52--60},
url = {http://www.aclweb.org/anthology/W10-3807},
year = 2010
}
Lo and Wu (2010)
Banchs, Rafael E. and Li, Haizhou (2011): AM-FM: A Semantic Framework for Translation Quality Assessment, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Techologies
add
@InProceedings{banchs-li:2011:ACL-HLT2011,
author = {Banchs, Rafael E. and Li, Haizhou},
title = {AM-FM: A Semantic Framework for Translation Quality Assessment},
booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Techologies},
month = {June},
address = {Portland, Oregon, USA},
publisher = {Association for Computational Linguistics},
pages = {153--158},
url = {http://www.aclweb.org/anthology/P11-2027},
year = 2011
}
Banchs and Li (2011)
Lo, Chi-kiu and Wu, Dekai (2011): MEANT: An inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility based on semantic roles, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Techologies
add
@InProceedings{lo-wu:2011:ACL-HLT2011,
author = {Lo, Chi-kiu and Wu, Dekai},
title = {MEANT: An inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility based on semantic roles},
booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Techologies},
month = {June},
address = {Portland, Oregon, USA},
publisher = {Association for Computational Linguistics},
pages = {220--229},
url = {http://www.aclweb.org/anthology/P11-1023},
year = 2011
}
Lo and Wu (2011)
Macháček, Matouš and Bojar, Ondřej (2011): Approximating a Deep-Syntactic Metric for MT Evaluation and Tuning, Proceedings of the Sixth Workshop on Statistical Machine Translation
add
@InProceedings{machavcek-bojar:2011:WMT,
author = {Mach\'{a}\v{c}ek, Matou\v{s} and Bojar, Ondřej},
title = {Approximating a Deep-Syntactic Metric for {MT} Evaluation and Tuning},
booktitle = {Proceedings of the Sixth Workshop on Statistical Machine Translation},
month = {July},
address = {Edinburgh, Scotland},
publisher = {Association for Computational Linguistics},
pages = {92--98},
url = {http://www.aclweb.org/anthology/W11-2108},
year = 2011
}
Macháček and Bojar (2011)
Karolina Owczarzak and Josef van Genabith and Andy Way (2007): Evaluating machine translation with LFG dependencies, Machine Translation
add
@article{MTJ:2007:Owczarzak,
author = {Karolina Owczarzak and Josef van Genabith and Andy Way},
title = {Evaluating machine translation with {LFG} dependencies},
pages = {95--119},
journal = {Machine Translation},
volume = {21},
number = {2},
month = {June},
year = 2007
}
Owczarzak et al. (2007)

MT Research Survey Wiki

A Comprehensive Survey of Neural and Statistical Machine Translation Research Publications

Search Descriptions

Syntax Based Metrics

Publications

Benchmarks

Discussion

Related Topics

New Publications