Abstract
Neural machine translation (NMT) systems typically employ maximum a posteriori (MAP) decoding to select the highest-scoring translation from the distribution mass. However, recent evidence highlights the inadequacy of MAP decoding, often resulting in low-quality or even pathological hypotheses -- the decoding objective is not aligned with real-world translation quality. This paper proposes calibrating hypothesis likelihoods with translation quality from a distribution view by directly optimizing their Pearson correlation -- thereby enhancing the effectiveness of translation decoding. With our method, translation on large language models (LLMs) improves substantially after limited training (2K instances per direction). This improvement is orthogonal to those achieved through supervised fine-tuning, leading to substantial gains across a broad range of metrics and human evaluations -- even when applied to top-performing translation-specialized LLMs fine-tuned on high-quality translation data, such as Tower, or when compared to recent preference optimization methods, like CPO. Moreover, the calibrated translation likelihood can directly serve as a strong proxy for translation quality, closely approximating or even surpassing some state-of-the-art translation quality estimation models, like CometKiwi. Lastly, our in-depth analysis demonstrates that calibration enhances the effectiveness of MAP decoding, thereby enabling greater efficiency in real-world deployment. The resulting state-of-the-art translation model, which covers 10 languages, along with the accompanying code and human evaluation data, has been released to the community: this https URL.
Abstract (translated)
神经机器翻译（NMT）系统通常采用最大后验概率（MAP）解码来从分布中选出最高分的译文。然而，近期的研究表明，MAP解码存在不足之处，常常导致低质量甚至病态的假设——其解码目标与现实世界的翻译质量不一致。本文提出了一种通过直接优化它们之间的皮尔逊相关性来校准假设可能性和翻译质量的方法，从而增强翻译解码的有效性。采用我们的方法，在有限训练（每方向2000个实例）后，大规模语言模型（LLMs）的翻译性能显著提升。这种改进与通过监督微调获得的进步相独立，并且在广泛的度量标准和人工评估中取得了实质性的增益——即使应用于经过高质量翻译数据精细调整的顶级专业翻译模型，如Tower或最近的偏好优化方法（CPO）。此外，校准后的翻译可能性可以直接作为翻译质量的强大代理，紧密接近甚至超越一些最先进的翻译质量估计模型，例如CometKiwi。最后，深入分析表明，校准可以增强MAP解码的有效性，在实际部署中实现更高的效率。我们发布的最先进的翻译模型涵盖了10种语言，并附有代码和人工评估数据，具体信息可访问以下链接：this https URL。
URL
https://arxiv.org/abs/2504.19044
