Abstract
Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle state estimation tasks involving motion blur and high dynamic range (HDR) illumination conditions. However, the versatility of event-based visual odometry (VO) relying on handcrafted data association (either direct or indirect methods) is still unreliable, especially in field robot applications under low-light HDR conditions, where the dynamic range can be enormous and the signal-to-noise ratio is spatially-and-temporally varying. Leveraging deep neural networks offers new possibilities for overcoming these challenges. In this paper, we propose a learning-based stereo event visual odometry. Building upon Deep Event Visual Odometry (DEVO), our system (called Stereo-DEVO) introduces a novel and efficient static-stereo association strategy for sparse depth estimation with almost no additional computational burden. By integrating it into a tightly coupled bundle adjustment (BA) optimization scheme, and benefiting from the recurrent network's ability to perform accurate optical flow estimation through voxel-based event representations to establish reliable patch associations, our system achieves high-precision pose estimation in metric scale. In contrast to the offline performance of DEVO, our system can process event data of \zs{Video Graphics Array} (VGA) resolution in real time. Extensive evaluations on multiple public real-world datasets and self-collected data justify our system's versatility, demonstrating superior performance compared to state-of-the-art event-based VO methods. More importantly, our system achieves stable pose estimation even in large-scale nighttime HDR scenarios.
Abstract (translated)
基于事件的相机是受生物启发的传感器，其像素可以在微秒级分辨率下独立且异步地响应亮度变化，这为处理包含运动模糊和高动态范围（HDR）照明条件的状态估计任务提供了潜力。然而，依赖于手工制作的数据关联（直接或间接方法）的基于事件的视觉测距（VO）在低光HDR现场机器人应用中的可靠性仍然不足，因为在这种情况下，动态范围可能非常大且信噪比会随时间和空间变化而波动。利用深度神经网络为克服这些挑战提供了新的可能性。 本文中，我们提出了一种学习型立体事件视觉测距方法。基于深度事件视觉测距（DEVO），我们的系统（称为Stereo-DEVO）引入了一个新颖且高效的静态立体关联策略，用于稀疏深度估计，并几乎无需额外的计算负担。通过将其整合到紧密耦合的整体调整优化方案中，并利用递归网络的能力通过对体素表示的事件进行准确的光流估算来建立可靠的补丁关联，我们的系统能够在度量尺度上实现高精度的姿态估计。与DEVO的离线性能相比，我们的系统可以实时处理VGA分辨率的事件数据。 在多个公共真实世界的数据集和自采集数据上的广泛评估证明了我们系统的多功能性，并展示了优于现有最先进的基于事件的VO方法的卓越性能。更重要的是，我们的系统即使在大规模夜间HDR场景中也能实现稳定的姿态估计。
URL
https://arxiv.org/abs/2509.08235
