Abstract
The recent success of large language models (LLMs) trained on static, pre-collected, general datasets has sparked numerous research directions and applications. One such direction addresses the non-trivial challenge of integrating pre-trained LLMs into dynamic data distributions, task structures, and user preferences. Pre-trained LLMs, when tailored for specific needs, often experience significant performance degradation in previous knowledge domains -- a phenomenon known as "catastrophic forgetting". While extensively studied in the continual learning (CL) community, it presents new manifestations in the realm of LLMs. In this survey, we provide a comprehensive overview of the current research progress on LLMs within the context of CL. This survey is structured into four main sections: we first describe an overview of continually learning LLMs, consisting of two directions of continuity: vertical continuity (or vertical continual learning), i.e., continual adaptation from general to specific capabilities, and horizontal continuity (or horizontal continual learning), i.e., continual adaptation across time and domains (Section 3). We then summarize three stages of learning LLMs in the context of modern CL: Continual Pre-Training (CPT), Domain-Adaptive Pre-training (DAP), and Continual Fine-Tuning (CFT) (Section 4). Then we provide an overview of evaluation protocols for continual learning with LLMs, along with the current available data sources (Section 5). Finally, we discuss intriguing questions pertaining to continual learning for LLMs (Section 6). The full list of papers examined in this survey is available at this https URL.
Abstract (translated)
近年来，基于静态、预先收集的通用数据集训练的大语言模型（LLMs）的成功引发了大量的研究方向和应用。其中一种方向解决了将预训练的LLM集成到动态数据分布、任务结构和用户偏好中的非平凡挑战。经过专门调整以满足特定需求后，预训练的LLM在先前知识领域中的表现常常会显著下降，这种现象被称为“灾难性遗忘”。尽管在持续学习（CL）领域得到了广泛研究，但LLMs在LLM领域中呈现出了新的表现形式。在本次调查中，我们全面概述了LLM在CL背景下的当前研究进展。本次调查分为四个主要部分：我们首先描述了持续学习LLMs的概述，包括两个方向：垂直连续（或垂直持续学习），即从通用到特定能力的持续适应，以及水平连续（或水平持续学习），即跨越时间和领域的持续适应（第3节）。接着我们总结了在现代CL背景下学习LLM的三个阶段：持续预训练（CPT）、领域自适应预训练（DAP）和持续微调（CFT）（第4节）。然后我们概述了使用LLMs进行持续学习的评估协议以及当前可用的数据源（第5节）。最后，我们讨论了与LLM的持续学习相关的一些有趣问题（第6节）。本次调查中审查的论文清单可以在这个https:// URL中找到。
URL
https://arxiv.org/abs/2404.16789
