Abstract
The automatic verbalization of structured knowledge is a key task for making knowledge graphs accessible to non-expert users and supporting retrieval-augmented generation systems. Although recent advances in Data-to-Text generation have improved multilingual coverage, little attention has been paid to potential biases in the verbalization of rare entities, frequently known as long-tail entities. In this work, we present the first systematic study of long-tail entities in Data-to-Text generation. We introduce TailNLG, a new multilingual benchmark in English, Italian, and Spanish, built from Wikidata and covering entities with varying levels of popularity. We evaluate three different families of large language models in zero-shot settings and compare their performance on rare versus common entities, as well as against the established WebNLG benchmark. Our results reveal a consistent bias against long-tail entities: embedding-based scores are lower, and model uncertainty is higher for rare entities. We further show that the impact of long-tail entities varies across models and languages, and that existing evaluation metrics do not consistently capture these differences, highlighting the need for more reliable evaluation frameworks.
Abstract (translated)
结构化知识的自动叙述化是让非专业用户访问知识图谱并支持检索增强生成系统的关键任务。尽管近期数据到文本生成技术的进步提升了多语言覆盖范围，但针对稀有实体（通常称为长尾实体）叙述化中潜在偏差的研究尚未受到足够关注。本研究首次系统性地探讨了数据到文本生成中的长尾实体问题。我们提出了TailNLG——一个基于维基数据构建的新的多语言基准测试集，涵盖英语、意大利语和西班牙语，并包含不同流行度级别的实体。我们在零样本设置下评估了三类大型语言模型，比较了它们在稀有与常见实体上的表现，并与成熟的WebNLG基准测试进行了对比。研究结果揭示了针对长尾实体的持续性偏差：稀有实体的基于嵌入的评分更低，模型不确定性更高。我们还发现长尾实体的影响因模型和语言而异，且现有评估指标未能一致捕捉这些差异，这凸显了建立更可靠评估框架的必要性。
URL
https://arxiv.org/abs/2603.27768
