Abstract
We propose a self-supervised approach for learning representations of objects from monocular videos and demonstrate it is particularly useful in situated settings such as robotics. The main contributions of this paper are: 1) a self-supervising objective trained with contrastive learning that can discover and disentangle object attributes from video without using any labels; 2) we leverage object self-supervision for online adaptation: the longer our online model looks at objects in a video, the lower the object identification error, while the offline baseline remains with a large fixed error; 3) to explore the possibilities of a system entirely free of human supervision, we let a robot collect its own data, train on this data with our self-supervise scheme, and then show the robot can point to objects similar to the one presented in front of it, demonstrating generalization of object attributes. An interesting and perhaps surprising finding of this approach is that given a limited set of objects, object correspondences will naturally emerge when using contrastive learning without requiring explicit positive pairs. Videos illustrating online object adaptation and robotic pointing are available at: https://online-objects.github.io/.
Abstract (translated)
我们提出了一种从单目视频中学习物体表示的自监督方法，并证明它在机器人等定位环境中特别有用。本文的主要贡献是：1）一个经过对比学习训练的自我监控目标，可以在不使用任何标签的情况下发现和分离视频中的对象属性；2）我们利用对象自我监控进行在线自适应：我们的在线模型在视频中观察对象的时间越长，对象识别度越低。在误差方面，虽然离线基线仍然存在较大的固定误差；3）为了探索完全不受人监督的系统的可能性，我们让机器人收集自己的数据，并使用我们的自我监督方案对这些数据进行训练，然后显示机器人可以指向与前面显示的类似的对象，演示一般情况。对象属性的化。这种方法的一个有趣的、也许是令人惊讶的发现是，给定一组有限的对象，当使用对比学习而不需要显式的正对时，对象对应自然会出现。演示在线对象适应和机器人指向的视频可从以下网址获得：https://online objects.github.io/。
URL
https://arxiv.org/abs/1906.04312
