Abstract
Event-based cameras offer unique advantages such as high temporal resolution, high dynamic range, and low power consumption. However, the massive storage requirements and I/O burdens of existing synthetic data generation pipelines and the scarcity of real data prevent event-based training datasets from scaling up, limiting the development and generalization capabilities of event vision models. To address this challenge, we introduce Video-to-Voxel (V2V), an approach that directly converts conventional video frames into event-based voxel grid representations, bypassing the storage-intensive event stream generation entirely. V2V enables a 150 times reduction in storage requirements while supporting on-the-fly parameter randomization for enhanced model robustness. Leveraging this efficiency, we train several video reconstruction and optical flow estimation model architectures on 10,000 diverse videos totaling 52 hours--an order of magnitude larger than existing event datasets, yielding substantial improvements.
Abstract (translated)
基于事件的相机提供了诸如高时间分辨率、宽动态范围和低功耗等独特优势。然而，现有合成数据生成管道的巨大存储需求和输入输出(I/O)负担，以及真实数据的稀缺性，限制了基于事件的训练数据集的扩展能力，从而阻碍了事件视觉模型的发展和泛化能力。为了解决这一挑战，我们引入了一种名为视频到体素（Video-to-Voxel, V2V）的方法，该方法可以直接将传统的视频帧转换为基于事件的体素网格表示，完全绕过了存储密集型的事件流生成过程。通过采用V2V技术，我们可以实现150倍的存储需求减少，并支持实时参数随机化以增强模型鲁棒性。利用这一效率，我们在总计长达52小时（包含10,000多个多样视频）的数据集上训练了几个视频重建和光流估计模型架构——这比现有的基于事件数据集大了一个数量级，从而显著提高了性能。
URL
https://arxiv.org/abs/2505.16797
