Abstract
The growing use of large language models (LLMs) has increased the need for automatic evaluation systems, particularly to address the challenge of information hallucination. Although existing faithfulness evaluation approaches have shown promise, they are predominantly English-focused and often require expensive human-labeled training data for fine-tuning specialized models. As LLMs see increased adoption in multilingual contexts, there is a need for accurate faithfulness evaluators that can operate across languages without extensive labeled data. This paper presents Self-Taught Evaluators for Multilingual Faithfulness, a framework that learns exclusively from synthetic multilingual summarization data while leveraging cross-lingual transfer learning. Through experiments comparing language-specific and mixed-language fine-tuning approaches, we demonstrate a consistent relationship between an LLM's general language capabilities and its performance in language-specific evaluation tasks. Our framework shows improvements over existing baselines, including state-of-the-art English evaluators and machine translation-based approaches.
Abstract (translated)
大规模语言模型（LLM）的广泛应用增加了对自动评估系统的需求，特别是为了解决信息虚构的问题。尽管现有的忠实性评价方法显示出了一定的效果，但它们主要集中在英语上，并且通常需要昂贵的人工标注数据来微调专门化的模型。随着LLMs在多语种环境中得到更广泛的应用，迫切需要能够在无需大量标注数据的情况下跨语言准确评估忠实性的工具。本文提出了“Self-Taught Evaluators for Multilingual Faithfulness”框架，该框架仅从合成的多语言摘要数据中学习，并利用跨语言迁移学习技术。通过比较特定语言和混合语言微调方法的实验，我们展示了LLM的一般语言能力与其在特定语言评估任务中的表现之间存在一致的关系。我们的框架在现有的基线之上取得了改进，包括最先进的英语评价器和基于机器翻译的方法。
URL
https://arxiv.org/abs/2507.20752
