Abstract
Popular representation learning methods encourage feature invariance under transformations applied at the input. However, in 3D perception tasks like object localization and segmentation, outputs are naturally equivariant to some transformations, such as rotation. Using pre-training loss functions that encourage equivariance of features under certain transformations provides a strong self-supervision signal while also retaining information of geometric relationships between transformed feature representations. This can enable improved performance in downstream tasks that are equivariant to such transformations. In this paper, we propose a spatio-temporal equivariant learning framework by considering both spatial and temporal augmentations jointly. Our experiments show that the best performance arises with a pre-training approach that encourages equivariance to translation, scaling, and flip, rotation and scene flow. For spatial augmentations, we find that depending on the transformation, either a contrastive objective or an equivariance-by-classification objective yields best results. To leverage real-world object deformations and motion, we consider sequential LiDAR scene pairs and develop a novel 3D scene flow-based equivariance objective that leads to improved performance overall. We show our pre-training method for 3D object detection which outperforms existing equivariant and invariant approaches in many settings.
Abstract (translated)
流行的表示学习方法鼓励在应用于输入时的变换下保持特征的不变性。然而，在像物体定位和分割这样的3D感知任务中，输出自然地对某些变换（例如旋转）具有等价性。通过使用鼓励在某些变换下保持特征等价的预训练损失函数，可以提供强大的自监督信号，同时保留变换前特征表示之间几何关系的信息。这可以提高在下游具有这种变换的任务的性能。在本文中，我们提出了一种空间和时间等价的表示学习框架，通过同时考虑空间和时间增强。我们的实验表明，最佳性能通过鼓励对平移、缩放和翻转、旋转和场景流动的等价性来实现。对于空间增强，我们发现，根据变换，无论是对比性目标还是类比目标都能获得最佳结果。为了利用真实的物体变形和运动，我们考虑了连续的激光雷达场景对，并开发了一个新的基于3D场景流的三等价目标，这使得整体性能得到提高。我们证明了我们的预训练方法在许多设置中优于现有的等价和不变方法。
URL
https://arxiv.org/abs/2404.11737
