Abstract
To cope with uncertain changes of the external world, intelligent systems must continually learn from complex, evolving environments and respond in real time. This ability, collectively known as general continual learning (GCL), encapsulates practical challenges such as online datastreams and blurry task boundaries. Although leveraging pretrained models (PTMs) has greatly advanced conventional continual learning (CL), these methods remain limited in reconciling the diverse and temporally mixed information along a single pass, resulting in sub-optimal GCL performance. Inspired by meta-plasticity and reconstructive memory in neuroscience, we introduce here an innovative approach named Meta Post-Refinement (MePo) for PTMs-based GCL. This approach constructs pseudo task sequences from pretraining data and develops a bi-level meta-learning paradigm to refine the pretrained backbone, which serves as a prolonged pretraining phase but greatly facilitates rapid adaptation of representation learning to downstream GCL tasks. MePo further initializes a meta covariance matrix as the reference geometry of pretrained representation space, enabling GCL to exploit second-order statistics for robust output alignment. MePo serves as a plug-in strategy that achieves significant performance gains across a variety of GCL benchmarks and pretrained checkpoints in a rehearsal-free manner (e.g., 15.10\%, 13.36\%, and 12.56\% on CIFAR-100, ImageNet-R, and CUB-200 under Sup-21/1K). Our source code is available at \href{this https URL}{MePo}
Abstract (translated)
为了应对外部世界不确定的变化，智能系统必须不断地从复杂且不断演变的环境中学习，并实时响应。这种能力集体称为通用持续学习（General Continual Learning, GCL），它涵盖了诸如在线数据流和模糊的任务边界等实际挑战。虽然通过预训练模型（Pretrained Models, PTMs）已大大推进了传统的持续学习（Conventional Continual Learning, CL），但这些方法在处理单次传递中多样且时间上交错的信息时仍然受限，导致GCL性能不佳。 受到神经科学中的元可塑性（meta-plasticity）和重构记忆的启发，我们在这里介绍了一种基于PTMs进行GCL的新颖方法，称为Meta Post-Refinement (MePo)。此方法从预训练数据中构建伪任务序列，并开发了一个双层元学习范式来优化预先训练好的骨干模型。这一过程相当于延长了预训练阶段，但大大促进了向下游GCL任务的快速适应性表示学习。 MePo进一步初始化一个元协方差矩阵作为预训练表示空间的参考几何形状，使GCL能够利用二次统计信息实现稳健的输出对齐。作为一个无需回放策略的方法（rehearsal-free），MePo在各种GCL基准和预训练检查点上实现了显著的性能提升（例如，在Sup-21/1K条件下，CIFAR-100提升了15.10%，ImageNet-R提升了13.36%，CUB-200提升了12.56%）。我们的源代码可在\href{this https URL}{MePo}中获取。
URL
https://arxiv.org/abs/2602.07940
