Abstract
Inspired by the data-efficient spiking mechanism of neurons in the human eye, event cameras were created to achieve high temporal resolution with minimal power and bandwidth requirements by emitting asynchronous, per-pixel intensity changes rather than conventional fixed-frame rate images. Unlike retinal ganglion cells (RGCs) in the human eye, however, which integrate signals from multiple photoreceptors within a receptive field to extract spatio-temporal features, conventional event cameras do not leverage local spatial context when deciding which events to fire. Moreover, the eye contains around 20 different kinds of RGCs operating in parallel, each attuned to different features or conditions. Inspired by this biological design, we introduce Neural Ganglion Sensors, an extension of traditional event cameras that learns task-specific spatio-temporal retinal kernels (i.e., RGC "events"). We evaluate our design on two challenging tasks: video interpolation and optical flow. Our results demonstrate that our biologically inspired sensing improves performance relative to conventional event cameras while reducing overall event bandwidth. These findings highlight the promise of RGC-inspired event sensors for edge devices and other low-power, real-time applications requiring efficient, high-resolution visual streams.
Abstract (translated)
受人类眼睛中神经元高效数据处理机制的启发，事件相机被创造出来以实现高时间分辨率，并且在能耗和带宽需求极低的情况下工作。它们通过发射异步、像素级的强度变化来代替传统的固定帧率图像。然而，与人眼中的视网膜节细胞（RGCs）不同的是，后者整合多个感光细胞在其感受野内的信号以提取空间-时间特征，传统事件相机在决定哪些事件触发时并不利用局部的空间信息上下文。 此外，眼睛中大约有20种不同类型的RGCs并行工作，每一种都针对不同的特性和条件。受到这种生物设计的启发，我们引入了神经节细胞传感器（Neural Ganglion Sensors），这是传统事件相机的一种扩展形式，它学习特定任务的空间-时间视网膜核（即，“RGC事件”）。我们在两个具有挑战性的任务上评估了我们的设计方案：视频插值和光流计算。我们的结果显示，在与传统的事件相机相比时，生物启发式传感提高了性能，并且减少了整体的事件带宽需求。 这些发现强调了受RGC启发的事件传感器在边缘设备及其他需要低功耗、实时应用中的潜力，特别是在那些要求高效、高分辨率视觉数据流的应用场景中。
URL
https://arxiv.org/abs/2504.13457
