Abstract
Event-based camera has emerged as a promising paradigm for robot perception, offering advantages with high temporal resolution, high dynamic range, and robustness to motion blur. However, existing deep learning-based event processing methods often fail to fully leverage the sparse nature of event data, complicating their integration into resource-constrained edge applications. While neuromorphic computing provides an energy-efficient alternative, spiking neural networks struggle to match of performance of state-of-the-art models in complex event-based vision tasks, like object detection and optical flow. Moreover, achieving high activation sparsity in neural networks is still difficult and often demands careful manual tuning of sparsity-inducing loss terms. Here, we propose Context-aware Sparse Spatiotemporal Learning (CSSL), a novel framework that introduces context-aware thresholding to dynamically regulate neuron activations based on the input distribution, naturally reducing activation density without explicit sparsity constraints. Applied to event-based object detection and optical flow estimation, CSSL achieves comparable or superior performance to state-of-the-art methods while maintaining extremely high neuronal sparsity. Our experimental results highlight CSSL's crucial role in enabling efficient event-based vision for neuromorphic processing.
Abstract (translated)
基于事件的相机已成为机器人感知领域的一个有前景的范式，它提供了高时间分辨率、宽动态范围以及对运动模糊的强大鲁棒性。然而，现有的基于深度学习的事件处理方法往往未能充分利用事件数据的稀疏特性，从而使其难以在资源受限的应用中集成。虽然神经形态计算提供了一种节能替代方案，但脉冲神经网络（SNN）在复杂基于事件的视觉任务，如物体检测和光流估计上，仍难以达到当前最佳模型的表现水平。此外，在神经网络中实现高激活稀疏性仍然具有挑战性，并且通常需要对诱导稀疏性的损失项进行仔细的手动调整。 为此，我们提出了一种新颖的框架——感知上下文的稀疏时空学习（CSSL），该框架引入了基于输入分布动态调节神经元激活的感知上下文阈值法。这种方法自然减少了激活密度，而无需明确的稀疏约束条件。在基于事件的物体检测和光流估计的应用中，CSSL实现了与现有最佳方法相当或更优的表现水平，并且保持了极高的神经元稀疏性。 实验结果强调了CSSL在实现高效的基于事件视觉处理以支持神经形态计算中的关键作用。
URL
https://arxiv.org/abs/2508.19806
