Abstract
Event cameras deliver visual data with high temporal resolution, low latency, and minimal redundancy, yet their asynchronous, sparse sequential nature challenges standard tensor-based machine learning (ML). While the recent asynchronous-to-synchronous (A2S) paradigm aims to bridge this gap by asynchronously encoding events into learned representations for ML pipelines, existing A2S approaches often sacrifice representation expressivity and generalizability compared to dense, synchronous methods. This paper introduces EVA (EVent Asynchronous representation learning), a novel A2S framework to generate highly expressive and generalizable event-by-event representations. Inspired by the analogy between events and language, EVA uniquely adapts advances from language modeling in linear attention and self-supervised learning for its construction. In demonstration, EVA outperforms prior A2S methods on recognition tasks (DVS128-Gesture and N-Cars), and represents the first A2S framework to successfully master demanding detection tasks, achieving a remarkable 47.7 mAP on the Gen1 dataset. These results underscore EVA's transformative potential for advancing real-time event-based vision applications.
Abstract (translated)
事件相机提供具有高时间分辨率、低延迟和最小冗余的视觉数据，但它们异步且稀疏的特性给传统的基于张量的方法在机器学习（ML）中带来了挑战。尽管最近提出的异步到同步（A2S）范式旨在通过将事件异步编码为适合于机器学习流水线的学习表示来弥合这一差距，但现有的A2S方法通常牺牲了与密集、同步方法相比的表达能力和泛化能力。本文介绍了EVA（EVent Asynchronous 表示学习），这是一种新颖的A2S框架，用于生成高度表达和通用化的事件到事件表示。受事件与语言之间类比的启发，EVA独特地借鉴了线性注意力和自监督学习在语言建模中的进展来构建自身。实验表明，EVA在识别任务（DVS128-Gesture 和 N-Cars）上超越了先前的A2S方法，并且是首个成功掌握具有挑战性的检测任务的A2S框架，在Gen1数据集上达到了惊人的47.7 mAP。这些结果突显了EVA对实时事件基础视觉应用发展的变革潜力。
URL
https://arxiv.org/abs/2505.11165
