Abstract
Pronouns are a long-standing challenge in machine translation. We present a study of the performance of a range of rule-based, statistical and neural MT systems on pronoun translation based on an extensive manual evaluation using the PROTEST test suite, which enables a fine-grained analysis of different pronoun types and sheds light on the difficulties of the task. We find that the rule-based approaches in our corpus perform poorly as a result of oversimplification, whereas SMT and early NMT systems exhibit significant shortcomings due to a lack of awareness of the functional and referential properties of pronouns. A recent Transformer-based NMT system with cross-sentence context shows very promising results on non-anaphoric pronouns and intra-sentential anaphora, but there is still considerable room for improvement in examples with cross-sentence dependencies.
Abstract (translated)
代词是机器翻译中长期存在的挑战。我们基于使用PROTEST测试套件的广泛手动评估，对代词翻译的一系列基于规则的，统计和神经MT系统的性能进行了研究，该测试套件能够对不同的代词类型进行细粒度分析并揭示任务的困难。我们发现，由于过度简化，我们语料库中基于规则的方法表现不佳，而SMT和早期NMT系统由于缺乏对代词的功能和参考属性的认识而表现出显着的缺点。最近一个基于变形金刚的NMT系统具有跨句子上下文，显示了非照应代词和句内回指的非常有希望的结果，但在具有跨句子依赖性的例子中仍有相当大的改进空间。
URL
https://arxiv.org/abs/1808.10196
