Abstract
Existing approaches to neural machine translation (NMT) generate the target language sequence token by token from left to right. However, this kind of unidirectional decoding framework cannot make full use of the target-side future contexts which can be produced in a right-to-left decoding direction, and thus suffers from the issue of unbalanced outputs. In this paper, we introduce a synchronous bidirectional neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time. Specifically, we first propose a new algorithm that enables synchronous bidirectional decoding in a single model. Then, we present an interactive decoding model in which left-to-right (right-to-left) generation does not only depend on its previously generated outputs, but also relies on future contexts predicted by right-to-left (left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on large-scale NIST Chinese-English, WMT14 English-German, and WMT18 Russian-English translation tasks. Experimental results demonstrate that our model achieves significant improvements over the strong Transformer model by 3.92, 1.49 and 1.04 BLEU points respectively, and obtains the state-of-the-art performance on Chinese-English and English-German translation tasks.
Abstract (translated)
现有的神经机器翻译方法（NMT）通过令牌从左到右生成目标语言序列令牌。然而，这种单向解码框架不能充分利用目标端未来的上下文，而这些上下文可以在从右到左的解码方向上产生，因此存在输出不平衡的问题。在本文中，我们介绍了一个同步双向神经机器翻译（SB-NMT），预测其输出使用左到右和右到左的解码同时和互动，以同时利用历史和未来的信息。具体地说，我们首先提出了一种新的算法，使同步双向解码在一个单一的模型。然后，我们提出了一种交互式解码模型，其中，从左到右（从右到左）的生成不仅依赖于其先前生成的输出，还依赖于由右到左（从左到右）解码预测的未来上下文。我们对所提出的SB-NMT模型进行了大规模的NIST中英文、WMT14英德语和WMT18俄语-英语翻译任务的广泛评估。实验结果表明，该模型比传统的强变压器模型分别提高了3.92、1.49和1.04个百分点，在中英、英德翻译任务中取得了最新的效果。
URL
https://arxiv.org/abs/1905.04847
