Abstract
Recently, neural machine translation has achieved remarkable progress by introducing well-designed deep neural networks into its encoder-decoder framework. From the optimization perspective, residual connections are adopted to improve learning performance for both encoder and decoder in most of these deep architectures, and advanced attention connections are applied as well. Inspired by the success of the DenseNet model in computer vision problems, in this paper, we propose a densely connected NMT architecture (DenseNMT) that is able to train more efficiently for NMT. The proposed DenseNMT not only allows dense connection in creating new features for both encoder and decoder, but also uses the dense attention structure to improve attention quality. Our experiments on multiple datasets show that DenseNMT structure is more competitive and efficient.
Abstract (translated)
最近，神经机器翻译通过将精心设计的深度神经网络引入其编码器 - 解码器框架中取得了显着的进步。从优化的角度来看，在大多数深度架构中，采用残差连接来提高编码器和解码器的学习性能，并且还应用了高级关注连接。受DenseNet模型在计算机视觉问题中的成功启发，在本文中，我们提出了一种密集连接的NMT架构（DenseNMT），能够更有效地训练NMT。所提出的DenseNMT不仅允许密码连接为编码器和解码器创建新特征，而且还使用密集的注意力结构来提高注意力质量。我们在多个数据集上的实验表明，DenseNMT结构更具竞争力和效率。
URL
https://arxiv.org/abs/1806.00722
