Abstract
Learning good feature embeddings for images often requires substantial training data. As a consequence, in settings where training data is limited (e.g., few-shot and zero-shot learning), we are typically forced to use a generic feature embedding across various tasks. Ideally, we want to construct feature embeddings that are tuned for the given task. In this work, we propose Task-Aware Feature Embedding Networks (TAFE-Nets) to learn how to adapt the image representation to a new task in a meta learning fashion. Our network is composed of a meta learner and a prediction network. Based on a task input, the meta learner generates parameters for the feature layers in the prediction network so that the feature embedding can be accurately adjusted for that task. We show that TAFE-Net is highly effective in generalizing to new tasks or concepts and evaluate the TAFE-Net on a range of benchmarks in zero-shot and few-shot learning. Our model matches or exceeds the state-of-the-art on all tasks. In particular, our approach improves the prediction accuracy of unseen attribute-object pairs by 4 to 15 points on the challenging visual attribute-object composition task.
Abstract (translated)
学习好的图像特征嵌入通常需要大量的训练数据。因此，在训练数据有限的情况下（例如，很少放炮和零放炮学习），我们通常被迫在不同任务中使用通用功能嵌入。理想情况下，我们希望构建针对给定任务优化的特性嵌入。在这项工作中，我们提出了任务感知的特征嵌入网络（TAFE网），以学习如何以元学习的方式使图像表示适应新的任务。我们的网络由元学习者和预测网络组成。基于任务输入，元学习者为预测网络中的特征层生成参数，以便能够针对该任务精确调整特征嵌入。我们表明，TAFE网络在推广新的任务或概念方面非常有效，并且在零镜头和少镜头学习的一系列基准上评估TAFE网络。我们的模型在所有任务上都符合或超过了最先进的水平。特别是，在具有挑战性的视觉属性对象组合任务中，我们的方法提高了4到15个未看到属性对象对的预测精度。
URL
https://arxiv.org/abs/1904.05967
