Abstract
Large language model (LLM) shows promising performances in a variety of downstream tasks, such as machine translation (MT). However, using LLMs for translation suffers from high computational costs and significant latency. Based on our evaluation, in most cases, translations using LLMs are comparable to that generated by neural machine translation (NMT) systems. Only in particular scenarios, LLM and NMT models show respective advantages. As a result, integrating NMT and LLM for translation and using LLM only when necessary seems to be a sound solution. A scheduling policy that optimizes translation result while ensuring fast speed and as little LLM usage as possible is thereby required. We compare several scheduling policies and propose a novel and straightforward decider that leverages source sentence features. We conduct extensive experiments on multilingual test sets and the result shows that we can achieve optimal translation performance with minimal LLM usage, demonstrating effectiveness of our decider.
Abstract (translated)
大型语言模型（LLM）在多种下游任务中表现出色，例如机器翻译（MT）。然而，使用LLM进行翻译面临着高昂的计算成本和显著的延迟问题。根据我们的评估，在大多数情况下，基于LLM的翻译效果与神经机器翻译（NMT）系统生成的效果相当。只有在特定场景下，LLM和NMT模型各自展现出优势。 因此，将NMT和LLM结合起来用于翻译，并仅在必要时使用LLM似乎是一个合理的解决方案。为了优化翻译结果并确保快速的速度以及尽可能少地使用LLM，需要制定一个调度策略。我们对比了几个不同的调度策略，并提出了一种新的、简单的决策机制，该机制利用源语言句子的特征来决定是否调用LLM。 我们在多语种测试集上进行了广泛的实验，结果显示我们可以实现最优翻译性能的同时减少对LLM的使用，证明了我们的决策机制的有效性。
URL
https://arxiv.org/abs/2505.13554
