Abstract
Despite the recent progresses, particularly in developing Language Models, there are fundamental challenges and unanswered questions about how such models can continually learn/memorize, self-improve, and find effective solutions. In this paper, we present a new learning paradigm, called Nested Learning (NL), that coherently represents a machine learning model with a set of nested, multi-level, and/or parallel optimization problems, each of which with its own context flow. Through the lenses of NL, existing deep learning methods learns from data through compressing their own context flow, and in-context learning naturally emerges in large models. NL suggests a philosophy to design more expressive learning algorithms with more levels, resulting in higher-order in-context learning and potentially unlocking effective continual learning capabilities. We advocate for NL by presenting three core contributions: (1) Expressive Optimizers: We show that known gradient-based optimizers, such as Adam, SGD with Momentum, etc., are in fact associative memory modules that aim to compress the gradients' information (by gradient descent). Building on this insight, we present other more expressive optimizers with deep memory and/or more powerful learning rules; (2) Self-Modifying Learning Module: Taking advantage of NL's insights on learning algorithms, we present a sequence model that learns how to modify itself by learning its own update algorithm; and (3) Continuum Memory System: We present a new formulation for memory system that generalizes the traditional viewpoint of long/short-term memory. Combining our self-modifying sequence model with the continuum memory system, we present a continual learning module, called Hope, showing promising results in language modeling, knowledge incorporation, and few-shot generalization tasks, continual learning, and long-context reasoning tasks.
Abstract (translated)
尽管在语言模型开发方面取得了近期的进展，但仍存在一些基本挑战和未解问题，这些问题涉及到如何让这些模型能够持续学习/记忆、自我改进以及找到有效的解决方案。在这篇论文中，我们提出了一种新的学习范式——嵌套学习（Nested Learning, NL），它通过一组嵌套的、多层次或多线程优化问题来一致地表示机器学习模型，每个问题都有其特定的上下文流。从NL的角度来看，现有的深度学习方法是通过对自身上下文流进行压缩来学习数据，而大模型中自然会出现上下文中的学习现象。NL提出了一种设计更具有表现力的学习算法的方法，这些算法可以有更多层次，从而实现更高阶的上下文学习，并有可能解锁有效的持续学习能力。 我们通过三个核心贡献来倡导嵌套学习（NL）： 1. 表达性优化器：我们知道基于梯度的优化器，如Adam、带有动量的SGD等，实际上都是关联记忆模块，旨在通过梯度下降压缩梯度信息。在此基础上，我们提出了其他更具有表现力的优化器，这些优化器包含深层记忆和/或更强的学习规则。 2. 自修改学习模块：利用NL对学习算法的理解，我们提出了一种序列模型，该模型能够通过学习自身的更新算法来自我修改。 3. 连续记忆系统：我们提出了一个新颖的记忆系统公式，它扩展了传统长短期记忆的观点。结合我们的自修改序列模型和连续记忆系统，我们提出了一种持续学习模块——希望（Hope），在语言建模、知识融合、少量示例泛化任务以及长时间上下文推理任务中表现出显著的性能。 这些贡献共同展示了嵌套学习范式如何推动机器学习技术向前发展，并提供了对未来研究方向的一些见解。
URL
https://arxiv.org/abs/2512.24695
