Abstract
There is a long history of using meta learning as representation learning, specifically for determining the relevance of inputs. In this paper, we examine an instance of meta-learning in which feature relevance is learned by adapting step size parameters of stochastic gradient descent---building on a variety of prior work in stochastic approximation, machine learning, and artificial neural networks. In particular, we focus on stochastic meta-descent introduced in the Incremental Delta-Bar-Delta (IDBD) algorithm for setting individual step sizes for each feature of a linear function approximator. Using IDBD, a feature with large or small step sizes will have a large or small impact on generalization from training examples. As a main contribution of this work, we extend IDBD to temporal-difference (TD) learning---a form of learning which is effective in sequential, non i.i.d. problems. We derive a variety of IDBD generalizations for TD learning, demonstrating that they are able to distinguish which features are relevant and which are not. We demonstrate that TD IDBD is effective at learning feature relevance in both an idealized gridworld and a real-world robotic prediction task.
Abstract (translated)
使用元学习作为表示学习有着悠久的历史，特别是为了确定输入的相关性。本文研究了一个元学习的实例，其中特征相关性是通过调整随机梯度下降的步长参数来学习的——建立在随机逼近、机器学习和人工神经网络的各种先验工作的基础上。特别是，我们着重讨论了增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量增量使用idbd，具有大或小步长的特性将对训练示例中的泛化产生大或小的影响。作为这项工作的主要贡献，我们将idbd扩展到时间差（td）学习，这是一种有效解决顺序性、非i.d.问题的学习形式。我们推导了TD学习的各种IDBD归纳法，证明它们能够区分哪些特征是相关的，哪些特征不是。我们证明了TD-IDBD在理想化网格和现实机器人预测任务中都能有效地学习特征相关性。
URL
https://arxiv.org/abs/1903.03252
