Abstract
Object concepts play a foundational role in human visual cognition, enabling perception, memory, and interaction in the physical world. Inspired by findings in developmental neuroscience - where infants are shown to acquire object understanding through observation of motion - we propose a biologically inspired framework for learning object-centric visual representations in an unsupervised manner. Our key insight is that motion boundary serves as a strong signal for object-level grouping, which can be used to derive pseudo instance supervision from raw videos. Concretely, we generate motion-based instance masks using off-the-shelf optical flow and clustering algorithms, and use them to train visual encoders via contrastive learning. Our framework is fully label-free and does not rely on camera calibration, making it scalable to large-scale unstructured video data. We evaluate our approach on three downstream tasks spanning both low-level (monocular depth estimation) and high-level (3D object detection and occupancy prediction) vision. Our models outperform previous supervised and self-supervised baselines and demonstrate strong generalization to unseen scenes. These results suggest that motion-induced object representations offer a compelling alternative to existing vision foundation models, capturing a crucial but overlooked level of abstraction: the visual instance. The corresponding code will be released upon paper acceptance.
Abstract (translated)
物体概念在人类视觉认知中扮演着基础性的角色，使感知、记忆和与物理世界的互动成为可能。受到发育神经科学发现的启发——即婴儿通过观察运动来获得对物体的理解——我们提出了一种生物启发式的框架，在无监督的方式下学习以对象为中心的视觉表示。我们的核心洞察是，运动边界可以作为用于将同一场景中的不同实例分组的强大信号，这可以从原始视频中推导出伪样本标签。具体来说，我们使用现成的光流和聚类算法生成基于运动的实例掩模，并利用它们通过对比学习训练视觉编码器。我们的框架完全不需要标签，也不依赖于相机校准，使其能够应用于大规模无结构视频数据。 我们在三个下游任务上评估了我们的方法，这些任务涵盖了从低级（单目深度估计）到高级（3D物体检测和占用预测）的视觉处理。我们的模型在之前监督学习及自监督学习基准测试中表现出色，并且对未见过的场景具有强大的泛化能力。这些结果表明，运动诱导的对象表示为现有视觉基础模型提供了一种有吸引力的替代方案，捕捉到了一个关键但被忽视的抽象层次：视觉实例。相应的代码将在论文接受后发布。
URL
https://arxiv.org/abs/2505.21635
