Abstract
Neural Machine Translation (NMT) models for low-resource languages suffer significant performance degradation under domain shift. We quantify this challenge using Dhao, an indigenous language of Eastern Indonesia with no digital footprint beyond the New Testament (NT). When applied to the unseen Old Testament (OT), a standard NMT model fine-tuned on the NT drops from an in-domain score of 36.17 chrF++ to 27.11 chrF++. To recover this loss, we introduce a hybrid framework where a fine-tuned NMT model generates an initial draft, which is then refined by a Large Language Model (LLM) using Retrieval-Augmented Generation (RAG). The final system achieves 35.21 chrF++ (+8.10 recovery), effectively matching the original in-domain quality. Our analysis reveals that this performance is driven primarily by the number of retrieved examples rather than the choice of retrieval algorithm. Qualitative analysis confirms the LLM acts as a robust "safety net," repairing severe failures in zero-shot domains.
Abstract (translated)
神经机器翻译（NMT）模型在资源匮乏的语言中遇到领域偏移时，性能会显著下降。我们使用达豪语来量化这一挑战——这是一种印度尼西亚东部的土著语言，在数字世界中的唯一足迹是《新约圣经》(NT)。当我们将其应用于未见过的《旧约圣经》(OT) 时，基于 NT 进行微调的标准 NMT 模型性能从内部领域的 36.17 chrF++ 下降到 27.11 chrF++。为了弥补这种损失，我们引入了一种混合框架：首先使用在 NT 上微调的 NMT 模型生成初步草稿，然后通过检索增强生成（RAG）的方法利用大型语言模型（LLM）对其进行精炼。最终系统达到了 35.21 chrF++（+8.10 的恢复），几乎与原始内部领域的质量相当。 我们的分析表明，这种性能的提升主要归因于检索到的例子的数量，而非所选择的检索算法。定性分析确认了 LLM 在零样本领域中充当了一个可靠的“安全网”，修复了严重的错误。
URL
https://arxiv.org/abs/2601.09982
