Abstract
Leveraging their advanced capabilities, Large Language Models (LLMs) demonstrate vast application potential in video games--from dynamic scene generation and intelligent NPC interactions to adaptive opponents--replacing or enhancing traditional game mechanics. However, LLMs' trustworthiness in this application has not been sufficiently explored. In this paper, we reveal that the models' inherent social biases can directly damage game balance in real-world gaming environments. To this end, we present FairGamer, the first bias evaluation Benchmark for LLMs in video game scenarios, featuring six tasks and a novel metrics ${D_lstd}$. It covers three key scenarios in games where LLMs' social biases are particularly likely to manifest: Serving as Non-Player Characters, Interacting as Competitive Opponents, and Generating Game Scenes. FairGamer utilizes both reality-grounded and fully fictional game content, covering a variety of video game genres. Experiments reveal: (1) Decision biases directly cause game balance degradation, with Grok-3 (average ${D_lstd}$ score=0.431) exhibiting the most severe degradation; (2) LLMs demonstrate isomorphic social/cultural biases toward both real and virtual world content, suggesting their biases nature may stem from inherent model characteristics. These findings expose critical reliability gaps in LLMs' gaming applications. Our code and data are available at anonymous GitHub this https URL .
Abstract (translated)
利用其先进的能力，大型语言模型（LLMs）在视频游戏中的应用潜力巨大——从动态场景生成、智能NPC互动到适应性对手，这些都可能替代或增强传统的游戏机制。然而，人们对LLM在这种应用场景中的可信度尚未进行充分探讨。在这篇论文中，我们揭示了模型内在的社会偏见可以直接损害实际游戏环境中的平衡性。为此，我们介绍了FairGamer——首个用于评估视频游戏中LLMs社会偏见的基准测试工具，该工具包括六个任务和一种新的度量指标${D_lstd}$。它涵盖了三个在游戏场景中LLM的社会偏见特别容易表现出来的关键领域：作为非玩家角色、充当竞争对手以及生成游戏场景。FairGamer同时使用了基于现实和完全虚构的游戏内容，涵盖多种视频游戏类型。 实验结果表明： 1. 决策偏差直接导致游戏平衡的恶化，其中Grok-3（平均${D_lstd}$分数=0.431）表现最差。 2. LLM对于真实世界和虚拟世界的主题均展示出同构的社会/文化偏见，这表明它们的偏见可能源自模型本身的特性。这些发现揭示了LLMs在游戏应用中的关键可靠性差距。 我们的代码和数据可在匿名GitHub（此URL不可直接链接，请访问论文获取具体网址）上获得。
URL
https://arxiv.org/abs/2508.17825
