Abstract
Vehicle-centric perception plays a crucial role in many intelligent systems, including large-scale surveillance systems, intelligent transportation, and autonomous driving. Existing approaches lack effective learning of vehicle-related knowledge during pre-training, resulting in poor capability for modeling general vehicle perception representations. To handle this problem, we propose VehicleMAE-V2, a novel vehicle-centric pre-trained large model. By exploring and exploiting vehicle-related multimodal structured priors to guide the masked token reconstruction process, our approach can significantly enhance the model's capability to learn generalizable representations for vehicle-centric perception. Specifically, we design the Symmetry-guided Mask Module (SMM), Contour-guided Representation Module (CRM) and Semantics-guided Representation Module (SRM) to incorporate three kinds of structured priors into token reconstruction including symmetry, contour and semantics of vehicles respectively. SMM utilizes the vehicle symmetry constraints to avoid retaining symmetric patches and can thus select high-quality masked image patches and reduce information redundancy. CRM minimizes the probability distribution divergence between contour features and reconstructed features and can thus preserve holistic vehicle structure information during pixel-level reconstruction. SRM aligns image-text features through contrastive learning and cross-modal distillation to address the feature confusion caused by insufficient semantic understanding during masked reconstruction. To support the pre-training of VehicleMAE-V2, we construct Autobot4M, a large-scale dataset comprising approximately 4 million vehicle images and 12,693 text descriptions. Extensive experiments on five downstream tasks demonstrate the superior performance of VehicleMAE-V2.
Abstract (translated)
车辆为中心的感知在许多智能系统中扮演着关键角色，包括大规模监控系统、智能交通和自动驾驶。现有的方法在预训练过程中缺乏有效的学习与车辆相关的知识，导致模型对于通用车辆感知表示的能力较差。为了解决这个问题，我们提出了VehicleMAE-V2，这是一种新颖的以车辆为中心的大规模预训练模型。通过探索并利用与车辆相关的多模态结构先验来引导掩码令牌重构过程，我们的方法能够显著提升模型学习通用化车辆感知表示的能力。 具体来说，我们设计了由对称性指导的掩码模块(SMM)、轮廓引导表示模块(CRM)和语义引导表示模块(SRM)，分别将对称性、轮廓和语义这三种结构先验整合到令牌重构中。SMM利用车辆的对称约束来避免保留对称补丁，并因此能够选择高质量的掩码图像补丁，减少信息冗余。CRM通过最小化轮廓特征与重建特征之间的概率分布差异，从而在像素级重构过程中保持整体车辆结构的信息。SRM则通过对比学习和跨模态蒸馏方式将图像-文本特征对齐，解决由于语义理解不足而在掩码重构期间造成的特征混淆问题。 为了支持VehicleMAE-V2的预训练，我们构建了Autobot4M，这是一个包含大约400万张车辆图片及12,693条文本描述的大规模数据集。在五个下游任务上的广泛实验表明了VehicleMAE-V2的优越性能。
URL
https://arxiv.org/abs/2512.19934
