Abstract
Previous approaches to persona simulation large language models (LLMs) have typically relied on learning basic biographical information, or using limited role-play dialogue datasets to capture a character's responses. However, a holistic representation of an individual goes beyond surface-level facts or conversations to deeper thoughts and thinking. In this work, we introduce CharacterBot, a model designed to replicate both the linguistic patterns and distinctive thought processes of a character. Using Lu Xun, a renowned Chinese writer, as a case study, we propose four training tasks derived from his 17 essay collections. These include a pre-training task focused on mastering external linguistic structures and knowledge, as well as three fine-tuning tasks: multiple-choice question answering, generative question answering, and style transfer, each aligning the LLM with Lu Xun's internal ideation and writing style. To optimize learning across these tasks, we introduce a CharLoRA parameter updating mechanism, where a general linguistic style expert collaborates with other task-specific experts to better study both the language style and the understanding of deeper thoughts. We evaluate CharacterBot on three tasks for linguistic accuracy and opinion comprehension, demonstrating that it significantly outperforms the baselines on our adapted metrics. We hope that this work inspires future research on deep character persona simulation LLM.
Abstract (translated)
此前，用于人格模拟的大语言模型（LLM）的方法通常依赖于学习基本的传记信息或使用有限的角色扮演对话数据集来捕捉角色的响应。然而，一个人物的全面表示超越了表面事实或对话，深入到更深层次的思想和思维方式中。在本研究中，我们介绍了CharacterBot，这是一个旨在复制人物的语言模式及其独特思维过程的模型。以中国著名作家鲁迅为案例研究，我们提出了四项训练任务，这些任务源自他的17部散文集。这包括一个预训练任务，专注于掌握外部语言结构和知识，以及三个微调任务：多选题问答、生成式问题回答和风格转换，每个任务都旨在将LLM与鲁迅的内部思考方式和写作风格对齐。为了在这四项任务中优化学习过程，我们引入了CharLoRA参数更新机制，在这个机制中，一个通用的语言风格专家与其他特定任务的专家合作，以更好地研究语言风格以及更深层次思想的理解。我们在三个任务上评估CharacterBot的语言准确性和观点理解能力，并证明它在我们的自适应指标上显著优于基线模型。我们希望这项工作能激发未来关于深度人物角色模拟大语言模型的研究。
URL
https://arxiv.org/abs/2502.12988
