Abstract
Large Language Models (LLMs) have quickly become an invaluable assistant for a variety of tasks. However, their effectiveness is constrained by their ability to tailor responses to human preferences and behaviors via personalization. Prior work in LLM personalization has largely focused on style transfer or incorporating small factoids about the user, as knowledge injection remains an open challenge. In this paper, we explore injecting knowledge of prior conversations into LLMs to enable future work on less redundant, personalized conversations. We identify two real-world constraints: (1) conversations are sequential in time and must be treated as such during training, and (2) per-user personalization is only viable in parameter-efficient settings. To this aim, we propose PLUM, a pipeline performing data augmentation for up-sampling conversations as question-answer pairs, that are then used to finetune a low-rank adaptation adapter with a weighted cross entropy loss. Even in this first exploration of the problem, we perform competitively with baselines such as RAG, attaining an accuracy of 81.5% across 100 conversations.
Abstract (translated)
大型语言模型（LLMs）已迅速成为各种任务中不可或缺的助手。然而，它们的有效性受限于通过个性化调整响应以符合人类偏好和行为的能力。之前关于LLM个性化的研究主要集中在风格转换或融入用户的一些小事实点上，因为知识注入仍然是一个开放挑战。在本文中，我们探索将先前对话的知识注入到LLMs中，以便在未来实现更少冗余且个性化的对话。我们识别了两个现实世界的约束条件：（1）对话是按时间顺序进行的，在训练过程中必须如此处理；（2）面向用户的个性化只有在参数高效设置下才可行。为此，我们提出了PLUM管道，用于执行数据增强以将对话上采样为问题-答案对，然后利用加权交叉熵损失来微调低秩适应适配器。即使在这个问题的初步探索中，我们的表现也与RAG等基线相当，在100次对话中的准确率达到81.5%。
URL
https://arxiv.org/abs/2411.13405
