Abstract
We analyze the ability of computational units to retain past responses after parameter updates, a key property for system-wide continual learning. Neural networks trained with gradient descent lack this capability, prompting us to propose Modelleyen, an alternative approach with inherent response preservation. We demonstrate through experiments on modeling the dynamics of a simple environment and on MNIST that, despite increased computational complexity and some representational limitations at its current stage, Modelleyen achieves continual learning without relying on sample replay or predefined task boundaries.
Abstract (translated)
我们分析了计算单元在参数更新后保留过去响应的能力，这是系统级持续学习的关键属性。通过梯度下降训练的神经网络缺乏这种能力，因此我们提出了Modelleyen这一替代方法，该方法具备固有的响应保持功能。通过实验模拟简单环境的动力学以及在MNIST数据集上的实验表明，尽管计算复杂性增加且目前阶段存在一些表示限制，但Modelleyen能够在不依赖样本重播或预定义任务边界的情况下实现持续学习。
URL
https://arxiv.org/abs/2503.22068
