Abstract
Event camera, a novel neuromorphic vision sensor, records data with high temporal resolution and wide dynamic range, offering new possibilities for accurate visual representation in challenging scenarios. However, event data is inherently sparse and noisy, mainly reflecting brightness changes, which complicates effective feature extraction. To address this, we propose a self-supervised pre-training framework to fully reveal latent information in event data, including edge information and texture cues. Our framework consists of three stages: Difference-guided Masked Modeling, inspired by the event physical sampling process, reconstructs temporal intensity difference maps to extract enhanced information from raw event data. Backbone-fixed Feature Transition contrasts event and image features without updating the backbone to preserve representations learned from masked modeling and stabilizing their effect on contrastive learning. Focus-aimed Contrastive Learning updates the entire model to improve semantic discrimination by focusing on high-value regions. Extensive experiments show our framework is robust and consistently outperforms state-of-the-art methods on various downstream tasks, including object recognition, semantic segmentation, and optical flow estimation. The code and dataset are available at this https URL.
Abstract (translated)
事件相机是一种新型的神经形态视觉传感器，它以高时间分辨率和宽动态范围记录数据，在具有挑战性的场景中为准确的视觉表示提供了新的可能性。然而，由于事件数据本身稀疏且噪声较多，并主要反映亮度变化，这使得有效特征提取变得复杂。为了应对这一问题，我们提出了一种自我监督的预训练框架，旨在从事件数据中充分揭示潜在信息，包括边缘信息和纹理线索。 我们的框架包含三个阶段： 1. 差异导向的掩码建模（Difference-guided Masked Modeling），受事件物理采样过程启发，重建时间强度差异图以增强从原始事件数据提取的信息。 2. 背骨固定的特征转换（Backbone-fixed Feature Transition），对比事件和图像特征而不更新背骨部分，以此保持通过掩码建模所学到的表示，并稳定其对对比学习的影响。 3. 目标导向的对比性学习（Focus-aimed Contrastive Learning），通过专注于高价值区域来更新整个模型以提高语义判别能力。 广泛的实验表明，我们的框架具有鲁棒性，在各种下游任务上始终优于现有方法，包括物体识别、语义分割和光流估计。代码和数据集可在此网址获得（请将“https URL”替换为实际链接）。
URL
https://arxiv.org/abs/2508.05507
