Abstract
Intelligent agent naturally learns from motion. Various self-supervised algorithms have leveraged motion cues to learn effective visual representations. The hurdle here is that motion is both ambiguous and complex, rendering previous works either suffer from degraded learning efficacy, or resort to strong assumptions on object motions. In this work, we design a new learning-from-motion paradigm to bridge these gaps. Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem. Given an input image and several sparse flow guidance vectors on it, our framework seeks to recover the full-image motion. Compared to other alternatives, our framework has several appealing properties: (1) Using sparse flow guidance during training resolves the inherent motion ambiguity, and thus easing feature learning. (2) Solving the pretext task of conditional motion propagation encourages the emergence of kinematically-sound representations that poss greater expressive power. Extensive experiments demonstrate that our framework learns structural and coherent features; and achieves state-of-the-art self-supervision performance on several downstream tasks including semantic segmentation, instance segmentation, and human parsing. Furthermore, our framework is successfully extended to several useful applications such as semi-automatic pixel-level annotation. Project page: "<a href="http://mmlab.ie.cuhk.edu.hk/projects/CMP/">this http URL</a>".
Abstract (translated)
智能体自然地从运动中学习。各种自我监督算法利用运动提示学习有效的视觉表示。这里的障碍是，运动既模糊又复杂，使得以前的作品要么学习效率下降，要么诉诸于对物体运动的强烈假设。在这项工作中，我们设计了一个新的学习从运动范式来填补这些空白。我们没有明确地建模运动概率，而是将“借口”任务设计为条件运动传播问题。给出了一个输入图像和几个稀疏的流引导向量，我们的框架寻求恢复整个图像运动。与其他方法相比，我们的框架有几个吸引人的特性：（1）在训练过程中使用稀疏流引导解决了固有的运动模糊性，从而简化了特征学习。（2）解决条件运动传播的借口任务，鼓励出现具有更大表现力的运动学声音表示。大量的实验表明，我们的框架学习了结构和连贯的特性，并在语义分割、实例分割和人工解析等几个下游任务上实现了最先进的自我监督性能。此外，我们的框架还成功地扩展到了一些有用的应用程序，如半自动像素级注释。项目页面：“<a href=“http://mmlab.ie.cuhk.edu.hk/projects/cmp/”>this http url</a>”。
URL
https://arxiv.org/abs/1903.11412
