Abstract
In recent years, spiking neural networks (SNNs) have gained momentum due to their high potential in time-series processing combined with minimal energy consumption. However, they still lack a dedicated and efficient training algorithm. The popular backpropagation with surrogate gradients, adapted from stochastic gradient descent (SGD)-derived algorithms, has several drawbacks when used as an optimizer for SNNs. Specifically, it suffers from low scalability and numerical imprecision. In this paper, we propose a novel SNN training method based on the alternating direction method of multipliers (ADMM). Our ADMM-based training aims to solve the problem of the SNN step function's non-differentiability. We formulate the problem, derive closed-form updates, and empirically show the optimizer's convergence properties, great potential, and possible new research directions to improve the method in a simulated proof-of-concept.
Abstract (translated)
近年来，由于脉冲神经网络（SNN）在时间序列处理方面具有巨大潜力，并且能耗极低，因此它们得到了迅速的发展。然而，目前这些网络仍缺乏一个专门和高效的训练算法。从随机梯度下降（SGD）衍生出来的流行反向传播方法与替代梯度，在作为SNN优化器使用时存在一些缺点，特别是可扩展性和数值精度方面的问题。在本文中，我们提出了一种基于交替方向乘子法（ADMM）的新颖的SNN训练方法。我们的基于ADMM的训练旨在解决SNN步函数不可微分的问题。我们对问题进行了建模，推导了封闭形式的更新公式，并通过模拟的概念验证实验展示了优化器的收敛特性、巨大的潜力以及可能的研究新方向来改进这种方法。
URL
https://arxiv.org/abs/2505.05527
