Abstract
Generally, the decoder-only large language models (LLMs) are adapted to context-aware neural machine translation (NMT) in a concatenating way, where LLMs take the concatenation of the source sentence (i.e., intra-sentence context) and the inter-sentence context as the input, and then to generate the target tokens sequentially. This adaptation strategy, i.e., concatenation mode, considers intra-sentence and inter-sentence contexts with the same priority, despite an apparent difference between the two kinds of contexts. In this paper, we propose an alternative adaptation approach, named Decoding-enhanced Multi-phase Prompt Tuning (DeMPT), to make LLMs discriminately model and utilize the inter- and intra-sentence context and more effectively adapt LLMs to context-aware NMT. First, DeMPT divides the context-aware NMT process into three separate phases. During each phase, different continuous prompts are introduced to make LLMs discriminately model various information. Second, DeMPT employs a heuristic way to further discriminately enhance the utilization of the source-side inter- and intra-sentence information at the final decoding phase. Experiments show that our approach significantly outperforms the concatenation method, and further improves the performance of LLMs in discourse modeling.
Abstract (translated)
通常，只有 decoder-only 大型语言模型（LLMs）是通过连接源句（即内部句子上下文）和间句子上下文来进行语境感知神经机器翻译（NMT），并在连接的同时生成目标词。这种适应策略，即连接模式，尽管两种上下文的表面差异似乎存在，但仍然将内部和间句子上下文赋予相同的优先级。在本文中，我们提出了另一种适应方法，名为解码增强多阶段提示调整（DeMPT），以使LLMs能够有选择性地建模和利用内部和间句子上下文，并更有效地将LLMs适应于语境感知 NMT。首先，DeMPT将语境感知 NMT 过程划分为三个单独的阶段。在每阶段，我们引入了不同的连续提示，使LLMs能够有选择性地建模各种信息。其次，DeMPT采用了一种启发式的方法，在解码阶段进一步增强了源 side 的间和内句子上下文的利用率。实验证明，我们的方法显著优于串联方法，并进一步提高了LLM在论述建模方面的性能。
URL
https://arxiv.org/abs/2402.15200
