Abstract
Learning with few samples is a major challenge for parameter-rich models like deep networks. In contrast, people learn complex new concepts even from very few examples, suggesting that the sample complexity of learning can often be reduced. Many approaches to few-shot learning build on transferring a representation from well-sampled classes, or using meta learning to favor architectures that can learn with few samples. Unfortunately, such approaches often struggle when learning in an online way or with non-stationary data streams. Here we describe a new approach to learn with fewer samples, by using additional information that is provided per sample. Specifically, we show how the sample complexity can be reduced by providing semantic information about the relevance of features per sample, like information about the presence of objects in a scene or confidence of detecting attributes in an image. We provide an improved generalization error bound for this case. We cast the problem of using per-sample feature relevance by using a new ellipsoid-margin loss, and develop an online algorithm that minimizes this loss effectively. Empirical evaluation on two machine vision benchmarks for scene classification and fine-grain bird classification demonstrate the benefits of this approach for few-shot learning.
Abstract (translated)
对于像深网络这样的参数丰富的模型来说，用很少的样本学习是一个主要的挑战。相比之下，人们甚至从很少的例子中学习复杂的新概念，这表明学习的样本复杂性通常可以降低。许多实现少镜头学习的方法都建立在从样本良好的类中传输一个表示的基础上，或者使用元学习来支持可以用很少的样本学习的体系结构。不幸的是，这种方法在以在线方式学习或使用非平稳数据流时往往很困难。在这里，我们描述了一种新的学习方法，通过使用每个样本提供的附加信息来减少样本。具体来说，我们展示了如何通过提供每个样本特征相关性的语义信息来降低样本的复杂性，例如场景中对象的存在信息或图像中检测属性的信心。我们为这种情况提供了一个改进的泛化误差界。我们利用一个新的椭球面边缘损失，提出了使用每样本特征相关性的问题，并开发了一种在线算法，有效地减少了这种损失。对场景分类和细粒度鸟类分类的两个机器视觉基准进行了实证评估，证明了该方法在少镜头学习中的优势。
URL
https://arxiv.org/abs/1906.03859
