Abstract
While test-time fine-tuning is beneficial in few-shot learning, the need for multiple backpropagation steps can be prohibitively expensive in real-time or low-resource scenarios. To address this limitation, we propose an approach that emulates gradient descent without computing gradients, enabling efficient test-time adaptation. Specifically, we formulate gradient descent as an Euler discretization of an ordinary differential equation (ODE) and train an auxiliary network to predict the task-conditional drift using only the few-shot support set. The adaptation then reduces to a simple numerical integration (e.g., via the Euler method), which requires only a few forward passes of the auxiliary network -- no gradients or forward passes of the target model are needed. In experiments on cross-domain few-shot classification using the Meta-Dataset and CDFSL benchmarks, our method significantly improves out-of-domain performance over the non-fine-tuned baseline while incurring only 6\% of the memory cost and 0.02\% of the computation time of standard fine-tuning, thus establishing a practical middle ground between direct transfer and fully fine-tuned approaches.
Abstract (translated)
虽然测试时的微调在少量样本学习中是有益的，但在实时或资源受限场景下，需要进行多次反向传播步骤的成本可能过高。为了解决这一限制，我们提出了一种方法，该方法能够在不计算梯度的情况下模拟梯度下降，从而实现高效的测试时适应性调整。具体而言，我们将梯度下降表述为普通微分方程（ODE）的欧拉离散化，并训练一个辅助网络仅使用少量样本支持集来预测任务条件下的漂移。随后的适应过程简化为简单的数值积分（例如通过欧拉方法），这只需要进行几次辅助网络的前向传递——不需要计算梯度或目标模型的前向传递。 在跨域少量样本分类实验中，我们使用了Meta-Dataset和CDFSL基准测试，并且我们的方法显著提高了跨域性能，相比于不进行微调的基础线而言，在仅占用标准微调内存成本6%和0.02%计算时间的情况下实现了这一改进。因此，该方法在直接迁移与完全微调之间建立了一种实用的中间途径。 简言之，这项研究提供了一个有效的方法来优化少量样本学习中的测试时适应性调整过程，在减少资源消耗的同时提高跨域模型性能。
URL
https://arxiv.org/abs/2504.15323
