Abstract
Mobile monocular 3D object detection (Mono3D) (e.g., on a vehicle, a drone, or a robot) is an important yet challenging task. Existing transformer-based offline Mono3D models adopt grid-based vision tokens, which is suboptimal when using coarse tokens due to the limited available computational power. In this paper, we propose an online Mono3D framework, called MonoATT, which leverages a novel vision transformer with heterogeneous tokens of varying shapes and sizes to facilitate mobile Mono3D. The core idea of MonoATT is to adaptively assign finer tokens to areas of more significance before utilizing a transformer to enhance Mono3D. To this end, we first use prior knowledge to design a scoring network for selecting the most important areas of the image, and then propose a token clustering and merging network with an attention mechanism to gradually merge tokens around the selected areas in multiple stages. Finally, a pixel-level feature map is reconstructed from heterogeneous tokens before employing a SOTA Mono3D detector as the underlying detection core. Experiment results on the real-world KITTI dataset demonstrate that MonoATT can effectively improve the Mono3D accuracy for both near and far objects and guarantee low latency. MonoATT yields the best performance compared with the state-of-the-art methods by a large margin and is ranked number one on the KITTI 3D benchmark.
Abstract (translated)
移动设备的三维物体检测(Mono3D) (例如在车辆、无人机或机器人中)是一项重要但具有挑战性的任务。现有的基于Transformer的离线Mono3D模型采用网格视觉代币，虽然使用细代币可以提高性能，但在使用粗代币时性能有所下降。在本文中，我们提出了一个在线Mono3D框架，称为MonoATT，它利用一种具有不同形状和大小的异质代币的新视觉Transformer来实现移动设备的Mono3D。MonoATT的核心思想是，在利用Transformer增强Mono3D之前，自适应地将更细的代币分配给更有意义的区域。为此，我们首先利用先前知识设计了一个评分网络，用于选择图像中最重要的区域，然后提出了一种具有注意力机制的代币簇集和融合网络，以逐步将代币集中在选定区域周围。最后，从不同代币中恢复像素级特征映射，并在使用SOTA的Mono3D检测器作为底层检测核心之前使用该框架进行物体检测。现实世界KITTI数据集的实验结果显示，MonoATT能够有效提高近远物体的Mono3D精度，并保证低延迟。MonoATT比最先进的方法表现更好，并以KITTI 3D基准排名第一。
URL
https://arxiv.org/abs/2303.13018
