Abstract
Language models are known to exhibit various forms of cultural bias in decision-making tasks, yet much less is known about their degree of cultural familiarity in open-ended text generation tasks. In this paper, we introduce the task of culturally-adapted art description generation, where models describe artworks for audiences from different cultural groups who vary in their familiarity with the cultural symbols and narratives embedded in the artwork. To evaluate cultural competence in this pragmatic generation task, we propose a framework based on culturally grounded question answering. We find that base models are only marginally adequate for this task, but, through a pragmatic speaker model, we can improve simulated listener comprehension by up to 8.2%. A human study further confirms that the model with higher pragmatic competence is rated as more helpful for comprehension by 8.0%.
Abstract (translated)
语言模型在决策任务中已知存在多种形式的文化偏见，但关于其在开放式文本生成任务中的文化熟悉度却鲜有研究。本文提出文化适应艺术描述生成任务，要求模型为具有不同文化背景的受众描述艺术品，这些受众对作品中嵌入的文化符号与叙事熟悉程度各异。为评估该语用生成任务中的文化能力，我们提出一个基于文化 grounded 问答的评估框架。研究发现基础模型仅勉强适用于该任务，但通过语用说话人模型可将模拟听者理解度提升最多8.2%。人类研究进一步证实，具备更高语用能力的模型在帮助理解方面获得评价提升8.0%。
URL
https://arxiv.org/abs/2604.02557
