Abstract
Despite the progress made in sentence-level NMT, current systems still fall short at achieving fluent, good quality translation for a full document. Recent works in context-aware NMT consider only a few previous sentences as context and may not scale to entire documents. To this end, we propose a novel and scalable top-down approach to hierarchical attention for context-aware NMT which uses sparse attention to selectively focus on relevant sentences in the document context and then attends to key words in those sentences. We also propose single-level attention approaches based on sentence or word-level information in the context. The document-level context representation, produced from these attention modules, is integrated into the encoder or decoder of the Transformer model depending on whether we use monolingual or bilingual context. Our experiments and evaluation on English-German datasets in different document MT settings show that our selective attention approach not only significantly outperforms context-agnostic baselines but also surpasses context-aware baselines in most cases.
Abstract (translated)
尽管在句子级的非翻译方面取得了进展，但目前的翻译系统在实现完整文档的流畅、高质量翻译方面仍然存在不足。最近在上下文感知的作品中，NMT只考虑前面的几句话作为上下文，可能无法扩展到整个文档。为此，我们提出了一种新颖的、可扩展的、自上而下的面向上下文感知的NMT层次注意方法，该方法利用稀疏注意来选择性地关注文档上下文中的相关句子，然后关注这些句子中的关键词。我们还提出了基于上下文中句子或词级信息的单级注意方法。由这些注意模块生成的文档级上下文表示，根据我们使用的是单语上下文还是双语上下文，集成到Transformer模型的编码器或解码器中。我们在不同文档mt设置下对英德数据集的实验和评估表明，我们的选择性注意方法不仅显著优于上下文无关基线，而且在大多数情况下也超过上下文感知基线。
URL
https://arxiv.org/abs/1903.08788
