Abstract
This paper introduces \textbf{Q-tuning}, a novel approach for continual prompt tuning that enables the lifelong learning of a pre-trained language model. When learning a new task, Q-tuning trains a task-specific prompt by adding it to a prompt queue consisting of the prompts from older tasks. To better transfer the knowledge of old tasks, we design an adaptive knowledge aggregation technique that reweighs previous prompts in the queue with a learnable low-rank matrix. Once the prompt queue reaches its maximum capacity, we leverage a PCA-based eviction rule to reduce the queue's size, allowing the newly trained prompt to be added while preserving the primary knowledge of old tasks. In order to mitigate the accumulation of information loss caused by the eviction, we additionally propose a globally shared prefix prompt and a memory retention regularization based on information theory. Extensive experiments demonstrate that our approach outperforms the state-of-the-art methods substantially on continual prompt tuning benchmarks. Moreover, our approach enables lifelong learning on linearly growing task sequences while requiring constant complexity for training and inference.
Abstract (translated)
本文介绍了一种名为 \textbf{Q-tuning} 的新方法，用于持续 prompt 调整，从而实现预训练语言模型的终身学习。在学习新任务时，Q-tuning 通过将新任务加入一个由 older 任务提示组成的提示队列中，来训练一个任务特定的提示。为了更好地转移旧任务的知識，我們設計了一種自適應的知識聚合技術，通過可學習的低秩矩陣重新權重队列中的先前提示。一旦提示隊列達到其最大容量，我們利用基于主成分分析（PCA）的驱逐规则来减少队列的大小，从而在保留主要舊任务知识的同时，允许新训练的提示加入队列。为了减轻驱逐操作造成的信息损失的累积，我们还提出了一个全局共享前缀提示和基于信息理论的内存保留 regularization。大量实验证明，与最先进的 methods相比，我们的方法在持续 prompt 调整基准测试中显著表现出优势。此外，我们的方法在 linearly growing 任务序列上实现终身学习，同时需要训练和推理的常规模度为不变。
URL
https://arxiv.org/abs/2404.14607
