Abstract
Dynamic scene reconstruction for autonomous driving enables vehicles to perceive and interpret complex scene changes more precisely. Dynamic Neural Radiance Fields (NeRFs) have recently shown promising capability in scene modeling. However, many existing methods rely heavily on accurate poses inputs and multi-sensor data, leading to increased system complexity. To address this, we propose FreeDriveRF, which reconstructs dynamic driving scenes using only sequential RGB images without requiring poses inputs. We innovatively decouple dynamic and static parts at the early sampling level using semantic supervision, mitigating image blurring and artifacts. To overcome the challenges posed by object motion and occlusion in monocular camera, we introduce a warped ray-guided dynamic object rendering consistency loss, utilizing optical flow to better constrain the dynamic modeling process. Additionally, we incorporate estimated dynamic flow to constrain the pose optimization process, improving the stability and accuracy of unbounded scene reconstruction. Extensive experiments conducted on the KITTI and Waymo datasets demonstrate the superior performance of our method in dynamic scene modeling for autonomous driving.
Abstract (translated)
用于自动驾驶的动态场景重建使车辆能够更精确地感知和解释复杂的场景变化。最近，动态神经辐射场（NeRF）在场景建模方面展现出了很有前景的能力。然而，许多现有的方法严重依赖于准确的姿态输入和多传感器数据，从而增加了系统的复杂性。为了解决这个问题，我们提出了FreeDriveRF，它仅使用连续的RGB图像来重建动态驾驶场景，并且不需要姿态输入。我们在早期采样阶段创新地将动态部分与静态部分解耦，利用语义监督来减少图像模糊和伪影。为了克服单目相机在面对物体运动和遮挡时所带来的挑战，我们引入了一种扭曲光线引导的动态对象渲染一致性损失，利用光流更好地约束动态建模过程。此外，我们还采用了估计的动态流来限制姿态优化过程，从而提高了无界场景重建的稳定性和准确性。我们在KITTI和Waymo数据集上进行了一系列实验，结果表明我们的方法在自动驾驶的动态场景建模方面具有优越的表现。
URL
https://arxiv.org/abs/2505.09406
