Abstract
Roadside Collaborative Perception refers to a system where multiple roadside units collaborate to pool their perceptual data, assisting vehicles in enhancing their environmental awareness. Existing roadside perception methods concentrate on model design but overlook data issues like calibration errors, sparse information, and multi-view consistency, leading to poor performance on recent published datasets. To significantly enhance roadside collaborative perception and address critical data issues, we present the first simulation framework RoCo-Sim for road-side collaborative perception. RoCo-Sim is capable of generating diverse, multi-view consistent simulated roadside data through dynamic foreground editing and full-scene style transfer of a single image. RoCo-Sim consists of four components: (1) Camera Extrinsic Optimization ensures accurate 3D to 2D projection for roadside cameras; (2) A novel Multi-View Occlusion-Aware Sampler (MOAS) determines the placement of diverse digital assets within 3D space; (3) DepthSAM innovatively models foreground-background relationships from single-frame fixed-view images, ensuring multi-view consistency of foreground; and (4) Scalable Post-Processing Toolkit generates more realistic and enriched scenes through style transfer and other enhancements. RoCo-Sim significantly improves roadside 3D object detection, outperforming SOTA methods by 83.74 on Rcooper-Intersection and 83.12 on TUMTraf-V2X for AP70. RoCo-Sim fills a critical gap in roadside perception simulation. Code and pre-trained models will be released soon: this https URL
Abstract (translated)
路边协作感知指的是一个系统，其中多个路边单元（RSU）合作共享其感知数据，协助车辆提高对周围环境的感知能力。现有的路边感知方法主要集中在模型设计上，但忽视了校准误差、信息稀疏性和多视角一致性等数据问题，导致在最近发布的数据集上的表现不佳。为了显著增强路边协作感知并解决关键的数据问题，我们提出了首个用于道路边协同感知的仿真框架RoCo-Sim（Roadside Collaborative Simulation）。该框架能够通过动态前景编辑和单张图像的全场景风格转换生成多视角一致性的多样化模拟路边数据。 RoCo-Sim包含四个组成部分： 1. **Camera Extrinsic Optimization**：确保路边摄像头从3D到2D投影的准确性； 2. **Multi-View Occlusion-Aware Sampler (MOAS)**：一种新颖的采样器，用于在三维空间内放置各种数字资产，并考虑到多视角下的遮挡情况； 3. **DepthSAM**：创新性地建模单帧固定视角图像中的前景和背景关系，确保多视角下前景的一致性； 4. **Scalable Post-Processing Toolkit**：通过风格转换和其他增强手段生成更逼真且丰富化的场景。 RoCo-Sim显著提高了路边三维物体检测的性能，在Rcooper-Intersection数据集上超过现有最佳方法（SOTA）83.74%，在TUMTraf-V2X数据集上的AP70指标得分高于SOTA 83.12%。RoCo-Sim填补了路边感知仿真领域的关键空白，相关代码和预训练模型即将发布：[此链接](this https URL)（原文中的链接地址需要替换为实际的有效URL）。
URL
https://arxiv.org/abs/2503.10410
