Abstract
The combination of spiking neural networks and event-based vision sensors holds the potential of highly efficient and high-bandwidth optical flow estimation. This paper presents the first hierarchical spiking architecture in which motion (direction and speed) selectivity emerges in an unsupervised fashion from the raw stimuli generated with an event-based camera. A novel adaptive neuron model and spike-timing-dependent plasticity formulation are at the core of this neural network governing its spike-based processing and learning, respectively. After convergence, the neural architecture exhibits the main properties of biological visual motion systems, namely feature extraction and local and global motion perception. To assess the outcome of the learning, a shallow conventional artificial neural network is trained to map the activation traces of the penultimate layer to the optical flow visual observables of ventral flow. The proposed solution is validated for simulated event sequences with ground-truth measurements. Experimental results show that accurate estimates of these parameters can be obtained over a wide range of speeds.
Abstract (translated)
尖峰神经网络和基于事件的视觉传感器的组合具有高效和高带宽光流估计的潜力。本文介绍了第一种分层尖峰结构，其中运动（方向和速度）选择性以无监督的方式从基于事件的相机产生的原始刺激中出现。一种新的自适应神经元模型和尖峰定时依赖的可塑性公式分别是这个神经网络的核心，它管理其基于尖峰的处理和学习。在收敛之后，神经结构展示了生物视觉运动系统的主要特性，即特征提取和局部和全局运动感知。为了评估学习的结果，训练浅的常规人工神经网络以将倒数第二层的激活轨迹映射到腹侧流的光流视觉可观察物。所提出的解决方案被验证用于具有地面实况测量的模拟事件序列。实验结果表明，可以在很宽的速度范围内获得这些参数的准确估计。
URL
https://arxiv.org/abs/1807.10936
