Abstract
With their motion-responsive nature, event-based cameras offer significant advantages over traditional cameras for optical flow estimation. While deep learning has improved upon traditional methods, current neural networks adopted for event-based optical flow still face temporal and spatial reasoning limitations. We propose Perturbed State Space Feature Encoders (P-SSE) for multi-frame optical flow with event cameras to address these challenges. P-SSE adaptively processes spatiotemporal features with a large receptive field akin to Transformer-based methods, while maintaining the linear computational complexity characteristic of SSMs. However, the key innovation that enables the state-of-the-art performance of our model lies in our perturbation technique applied to the state dynamics matrix governing the SSM system. This approach significantly improves the stability and performance of our model. We integrate P-SSE into a framework that leverages bi-directional flows and recurrent connections, expanding the temporal context of flow prediction. Evaluations on DSEC-Flow and MVSEC datasets showcase P-SSE's superiority, with 8.48% and 11.86% improvements in EPE performance, respectively.
Abstract (translated)
事件相机通过其运动响应特性，在光流估计方面相对于传统相机具有显著优势。尽管深度学习已经改进了传统的光流估算方法，但目前用于基于事件的光流的神经网络仍然面临时间和空间推理的局限性。为了解决这些问题，我们提出了扰动状态空间特征编码器（P-SSE），这是一种用于多帧光流处理的方法，适用于事件相机。P-SSE以类似于Transformer方法的方式自适应地处理具有大感受野的空间时间特征，同时保持SSM（状态空间模型）特有的线性计算复杂度特点。 然而，使我们的模型达到最先进的性能的关键创新在于我们应用于控制SSM系统的状态动力学矩阵的扰动技术。这种方法显著提高了模型的稳定性和性能。我们将P-SSE整合到一个框架中，该框架利用双向流和递归连接来扩展光流预测的时间上下文。 在DSEC-Flow和MVSEC数据集上的评估展示了P-SSE的优势，分别在EPE（端点误差）性能上取得了8.48%和11.86%的提升。
URL
https://arxiv.org/abs/2504.10669
