Abstract
While recent research increasingly showcases the remarkable capabilities of Large Language Models (LLMs), it's vital to confront their hidden pitfalls. Among these challenges, the issue of memorization stands out, posing significant ethical and legal risks. In this paper, we presents a Systematization of Knowledge (SoK) on the topic of memorization in LLMs. Memorization is the effect that a model tends to store and reproduce phrases or passages from the training data and has been shown to be the fundamental issue to various privacy and security attacks against LLMs. We begin by providing an overview of the literature on the memorization, exploring it across five key dimensions: intentionality, degree, retrievability, abstraction, and transparency. Next, we discuss the metrics and methods used to measure memorization, followed by an analysis of the factors that contribute to memorization phenomenon. We then examine how memorization manifests itself in specific model architectures and explore strategies for mitigating these effects. We conclude our overview by identifying potential research topics for the near future: to develop methods for balancing performance and privacy in LLMs, and the analysis of memorization in specific contexts, including conversational agents, retrieval-augmented generation, multilingual language models, and diffusion language models.
Abstract (translated)
虽然最近的研究越来越展示了大型语言模型（LLMs）的非凡能力，但面对其隐藏的陷阱至关重要。在这些挑战中，记忆问题突出，带来了重大的伦理和法律风险。在本文中，我们关于记忆在LLMs上的系统化知识（SoK）。记忆是模型倾向于存储和复制训练数据中的短语或段落的效应，已经被证明是各种对LLMs进行隐私和安全攻击的根本问题。我们首先对相关文献进行了回顾，探讨了记忆在五个关键维度上的影响：故意性、程度、可检索性、抽象性和透明度。接下来，我们讨论了用于衡量记忆的指标和方法，并分析了导致记忆现象的因素。然后我们研究了记忆在具体模型架构中的表现，并探讨了减轻这些影响的方法。最后，我们在概述中指出了未来可能的研究方向：为LLMs开发平衡性能和隐私的方法，以及分析特定情境（包括对话机器人、检索增强生成、多语言语言模型和扩散语言模型）下的记忆现象。
URL
https://arxiv.org/abs/2410.02650
