Abstract
This paper develops a mathematical argument and algorithms for building representations of data from event-based cameras, that we call Fast Feature Field ($\text{F}^3$). We learn this representation by predicting future events from past events and show that it preserves scene structure and motion information. $\text{F}^3$ exploits the sparsity of event data and is robust to noise and variations in event rates. It can be computed efficiently using ideas from multi-resolution hash encoding and deep sets - achieving 120 Hz at HD and 440 Hz at VGA resolutions. $\text{F}^3$ represents events within a contiguous spatiotemporal volume as a multi-channel image, enabling a range of downstream tasks. We obtain state-of-the-art performance on optical flow estimation, semantic segmentation, and monocular metric depth estimation, on data from three robotic platforms (a car, a quadruped robot and a flying platform), across different lighting conditions (daytime, nighttime), environments (indoors, outdoors, urban, as well as off-road) and dynamic vision sensors (resolutions and event rates). Our implementations can predict these tasks at 25-75 Hz at HD resolution.
Abstract (translated)
本文开发了一个数学论证和算法，用于从事件相机生成数据表示，我们称之为快速特征场（Fast Feature Field, $\text{F}^3$）。通过预测过去事件的未来事件来学习这种表示，并展示了它能保留场景结构和运动信息。$\text{F}^3$ 利用了事件数据的稀疏性，在噪声和事件率变化下具有鲁棒性，可以通过多分辨率哈希编码和深度集的思想高效计算出来，在高清分辨率下实现120 Hz，在VGA分辨率下实现440 Hz。$\text{F}^3$ 将事件在连续时空体积内表示为一个多通道图像，从而支持一系列下游任务。我们在三个机器人平台（汽车、四足机器人和飞行平台）的数据上获得了光流估计、语义分割以及单目度量深度估计方面的最佳性能，在不同光照条件（白天、夜间）、环境（室内、室外、城市及越野）以及动态视觉传感器（分辨率和事件率）下的数据也取得了良好效果。我们的实现可以在高清分辨率下以25-75 Hz 的速率预测这些任务。
URL
https://arxiv.org/abs/2509.25146
