Abstract
There are two primary ways of incorporating new information into a language model (LM): changing its prompt or changing its parameters, e.g. via fine-tuning. Parameter updates incur no long-term storage cost for model changes. However, for many model updates, prompting is significantly more effective: prompted models can generalize robustly from single examples and draw logical inferences that do not occur under standard fine-tuning. Can models be modified so that fine-tuning does emulate prompting? This paper describes a method for meta-training LMs such that gradient updates emulate the effects of conditioning on new information. Our approach uses tools from gradient-based meta-learning but uses an LM's own prompted predictions as targets, eliminating the need for ground-truth labels. Subsequent gradient descent training recovers some (and occasionally all) of prompted model performance -- showing improvement on the ``reversal curse'' tasks, and answering questions about text passages after a single gradient update. These results suggest that, with appropriate initialization, gradient descent can be surprisingly expressive. Our results suggest new avenues for long-context modeling and offer insight into the generalization capabilities of gradient-based learning.
Abstract (translated)
将新信息融入语言模型（LM）主要有两种方式：更改其提示或调整其参数，例如通过微调来实现。参数更新不会产生长期存储成本。然而，在许多模型更新的情况下，使用提示更加有效：经过提示的模型可以从单个示例中稳健地泛化，并得出逻辑推断，而这些在标准微调过程中并不会发生。是否可以修改模型使微调的效果模拟出提示的效果呢？ 本文描述了一种方法，即通过元训练LM的方式使得梯度更新能够模仿对新信息条件化的效果。我们的方法使用基于梯度的元学习工具，但以语言模型自身受到提示后的预测作为目标，从而消除了对真实标签的需求。后续的梯度下降训练可以恢复一些（有时甚至是全部）由提示带来的模型性能——在“逆转诅咒”任务中表现出改善，并且仅通过一次梯度更新即可回答关于文本段落的问题。这些结果表明，在适当的初始化下，梯度下降具有惊人的表达能力。 我们的研究成果为长上下文建模提供了新的途径，并对基于梯度的学习泛化能力提供了见解。
URL
https://arxiv.org/abs/2506.20989
