Paper Reading AI Learner

MonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformer

2023-03-23 03:45:03
Yunsong Zhou, Hongzi Zhu, Quan Liu, Shan Chang, Minyi Guo


Mobile monocular 3D object detection (Mono3D) (e.g., on a vehicle, a drone, or a robot) is an important yet challenging task. Existing transformer-based offline Mono3D models adopt grid-based vision tokens, which is suboptimal when using coarse tokens due to the limited available computational power. In this paper, we propose an online Mono3D framework, called MonoATT, which leverages a novel vision transformer with heterogeneous tokens of varying shapes and sizes to facilitate mobile Mono3D. The core idea of MonoATT is to adaptively assign finer tokens to areas of more significance before utilizing a transformer to enhance Mono3D. To this end, we first use prior knowledge to design a scoring network for selecting the most important areas of the image, and then propose a token clustering and merging network with an attention mechanism to gradually merge tokens around the selected areas in multiple stages. Finally, a pixel-level feature map is reconstructed from heterogeneous tokens before employing a SOTA Mono3D detector as the underlying detection core. Experiment results on the real-world KITTI dataset demonstrate that MonoATT can effectively improve the Mono3D accuracy for both near and far objects and guarantee low latency. MonoATT yields the best performance compared with the state-of-the-art methods by a large margin and is ranked number one on the KITTI 3D benchmark.

Abstract (translated)

移动设备的三维物体检测(Mono3D) (例如在车辆、无人机或机器人中)是一项重要但具有挑战性的任务。现有的基于Transformer的离线Mono3D模型采用网格视觉代币,虽然使用细代币可以提高性能,但在使用粗代币时性能有所下降。在本文中,我们提出了一个在线Mono3D框架,称为MonoATT,它利用一种具有不同形状和大小的异质代币的新视觉Transformer来实现移动设备的Mono3D。MonoATT的核心思想是,在利用Transformer增强Mono3D之前,自适应地将更细的代币分配给更有意义的区域。为此,我们首先利用先前知识设计了一个评分网络,用于选择图像中最重要的区域,然后提出了一种具有注意力机制的代币簇集和融合网络,以逐步将代币集中在选定区域周围。最后,从不同代币中恢复像素级特征映射,并在使用SOTA的Mono3D检测器作为底层检测核心之前使用该框架进行物体检测。现实世界KITTI数据集的实验结果显示,MonoATT能够有效提高近远物体的Mono3D精度,并保证低延迟。MonoATT比最先进的方法表现更好,并以KITTI 3D基准排名第一。



3D Action Action_Localization Action_Recognition Activity Adversarial Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot