Abstract
The field of neural machine translation (NMT) has changed with the advent of large language models (LLMs). Much of the recent emphasis in natural language processing (NLP) has been on modeling machine translation and many other problems using a single pre-trained Transformer decoder, while encoder-decoder architectures, which were the standard in earlier NMT models, have received relatively less attention. In this paper, we explore translation models that are universal, efficient, and easy to optimize, by marrying the world of LLMs with the world of NMT. We apply LLMs to NMT encoding and leave the NMT decoder unchanged. We also develop methods for adapting LLMs to work better with the NMT decoder. Furthermore, we construct a new dataset involving multiple tasks to assess how well the machine translation system generalizes across various tasks. Evaluations on the WMT and our datasets show that results using our method match or surpass a range of baselines in terms of translation quality, but achieve $2.4 \sim 6.5 \times$ inference speedups and a $75\%$ reduction in the memory footprint of the KV cache. It also demonstrates strong generalization across a variety of translation-related tasks.
Abstract (translated)
神经机器翻译（NMT）领域随着大型语言模型（LLMs）的出现发生了变化。最近自然语言处理（NLP）领域的重点是使用单个预训练的Transformer解码器来建模机器翻译及其他许多问题，而早期NMT模型中使用的编码器-解码器架构则相对较少受到关注。在本文中，我们探索了一种将LLM的世界与NMT的世界结合的方法，以创建通用、高效且易于优化的翻译模型。我们将LLMs应用于NMT编码，并保持原有的NMT解码器不变。同时，我们也开发了适应方法来改进LLMs使其更好地配合NMT解码器工作。此外，我们构建了一个包含多个任务的新数据集，用于评估机器翻译系统在各种任务中的泛化能力。我们在WMT和我们的数据集上进行了测试，结果显示使用我们提出的方法所得到的翻译质量与一系列基线方法相比不相上下甚至更优，但实现了2.4至6.5倍的速度提升以及KV缓存内存占用减少了75%。此外，该方法还展示了在各种翻译相关任务上的强大泛化能力。
URL
https://arxiv.org/abs/2503.06594
