Abstract
Accurately perceiving and tracking instances over time is essential for the decision-making processes of autonomous agents interacting safely in dynamic environments. With this intention, we propose Mask4D for the challenging task of 4D panoptic segmentation of LiDAR point clouds. Mask4D is the first transformer-based approach unifying semantic instance segmentation and tracking of sparse and irregular sequences of 3D point clouds into a single joint model. Our model directly predicts semantic instances and their temporal associations without relying on any hand-crafted non-learned association strategies such as probabilistic clustering or voting-based center prediction. Instead, Mask4D introduces spatio-temporal instance queries which encode the semantic and geometric properties of each semantic tracklet in the sequence. In an in-depth study, we find that it is critical to promote spatially compact instance predictions as spatio-temporal instance queries tend to merge multiple semantically similar instances, even if they are spatially distant. To this end, we regress 6-DOF bounding box parameters from spatio-temporal instance queries, which is used as an auxiliary task to foster spatially compact predictions. Mask4D achieves a new state-of-the-art on the SemanticKITTI test set with a score of 68.4 LSTQ, improving upon published top-performing methods by at least +4.5%.
Abstract (translated)
准确的时间感知和跟踪对于动态环境中安全自主代理交互的决策过程至关重要。为了实现这一目标,我们提出了Mask4D,以解决LiDAR点云4DPanoptic分割的挑战性任务。Mask4D是第一种基于Transformer的方法,将语义实例分割和跟踪稀疏和不规则3D点云序列统一到一个联合模型中。我们的模型直接预测语义实例及其时间关联,而不需要依赖任何手制非学习性关联策略,如概率簇集或投票中心预测。相反,Mask4D引入了时间实例查询,编码每个语义跟踪条目的语义和几何属性,在序列中。在深入研究中发现,促进空间紧凑实例预测非常重要,因为时间实例查询往往会将多个语义相似实例合并,即使它们空间上相距很远。为此,我们恢复了6DOF边界框参数从时间实例查询,并将其用作辅助任务,以促进空间紧凑的预测。Mask4D在语义KITTI测试集上取得了新的先进技术水平,得分为68.4LSTQ,比已发布的最佳方法至少提高了4.5%。
URL
https://arxiv.org/abs/2309.16133
