Abstract
Event cameras have the potential to capture continuous motion information over time and space, making them well-suited for optical flow estimation. However, most existing learning-based methods for event-based optical flow adopt frame-based techniques, ignoring the spatio-temporal characteristics of events. Additionally, these methods assume linear motion between consecutive events within the loss time window, which increases optical flow errors in long-time sequences. In this work, we observe that rich spatio-temporal information and accurate nonlinear motion between events are crucial for event-based optical flow estimation. Therefore, we propose E-NMSTFlow, a novel unsupervised event-based optical flow network focusing on long-time sequences. We propose a Spatio-Temporal Motion Feature Aware (STMFA) module and an Adaptive Motion Feature Enhancement (AMFE) module, both of which utilize rich spatio-temporal information to learn spatio-temporal data associations. Meanwhile, we propose a nonlinear motion compensation loss that utilizes the accurate nonlinear motion between events to improve the unsupervised learning of our network. Extensive experiments demonstrate the effectiveness and superiority of our method. Remarkably, our method ranks first among unsupervised learning methods on the MVSEC and DSEC-Flow datasets. Our project page is available at this https URL.
Abstract (translated)
事件相机有能力捕捉长时间和空间上的连续运动信息，因此非常适合用于光流估计。然而，大多数现有的基于学习的方法在处理以事件为基础的光流时采用的是帧基技术，忽视了事件的空间时间和特性特征。此外，这些方法假设在损失时间窗口内的连续事件之间存在线性运动，这会导致长时间序列中的光流误差增加。 在这项工作中，我们观察到丰富的空间和时间信息以及事件之间的准确非线性运动对于基于事件的光流估计至关重要。因此，我们提出了E-NMSTFlow，这是一种新的无监督基于事件的光流网络，专注于长时序数据。我们提出了一种时空运动特征感知（STMFA）模块和一种自适应运动特征增强（AMFE）模块，这两个模块都利用了丰富的时空信息来学习时空数据关联。同时，我们还提出了一种非线性运动补偿损失函数，该函数利用事件之间的准确非线性运动以改善我们网络的无监督学习过程。 大量的实验表明我们的方法有效且优越，在MVSEC和DSEC-Flow数据集上基于无监督学习的方法中，我们的方法排名第一。我们的项目页面可在此链接访问：[提供的URL]
URL
https://arxiv.org/abs/2505.05089
