Abstract
Continual learning aims to accumulate knowledge over a data stream while mitigating catastrophic forgetting. In Non-exemplar Class Incremental Learning (NECIL), forgetting arises during incremental optimization because old classes are inaccessible, hindering the retention of prior knowledge. To solve this, previous methods struggle in achieving the stability-plasticity balance in the training stages. However, we note that the testing stage is rarely considered among them, but is promising to be a solution to forgetting. Therefore, we propose RoSE, which is a simple yet effective method that \textbf{R}est\textbf{o}res forgotten knowledge through test-time \textbf{S}emantic \textbf{E}volution. Specifically designed for minimizing forgetting, RoSE is a test-time semantic drift compensation framework that enables more accurate drift estimation in a self-supervised manner. Moreover, to avoid incomplete optimization during online testing, we derive an analytical solution as an alternative to gradient descent. We evaluate RoSE on CIFAR-100, TinyImageNet, and ImageNet100 datasets, under both cold-start and warm-start settings. Our method consistently outperforms most state-of-the-art (SOTA) methods across various scenarios, validating the potential and feasibility of test-time evolution in NECIL.
Abstract (translated)
持续学习旨在通过数据流积累知识，同时减轻灾难性遗忘。在非示例类增量学习（NECIL）中，由于旧类别不可访问，所以在增量优化过程中会引发遗忘，从而阻碍先前知识的保留。为了解决这个问题，以往的方法在训练阶段难以实现稳定性和可塑性的平衡。然而，我们注意到，在这些方法中很少考虑测试阶段，但该阶段有可能成为解决遗忘问题的一种解决方案。因此，我们提出了一种简单而有效的方法——RoSE（通过测试时间语义进化恢复遗忘知识）。具体而言，为最小化遗忘设计的RoSE是一个测试时语义漂移补偿框架，它以自监督的方式使更准确的漂移估计得以实现。此外，为了在在线测试过程中避免不完全优化，我们推导出了一种替代梯度下降的方法作为解析解。 我们在冷启动和温启动设置下对CIFAR-100、TinyImageNet 和 ImageNet100 数据集上评估了RoSE。我们的方法在各种场景中均超过了大多数最先进的（SOTA）方法，验证了测试时进化在NECIL中的潜力和可行性。
URL
https://arxiv.org/abs/2503.16793
