Abstract
Multi-bit spiking neural networks (SNNs) have recently become a heated research spot, pursuing energy-efficient and high-accurate AI. However, with more bits involved, the associated memory and computation demands escalate to the point where the performance improvements become disproportionate. Based on the insight that different layers demonstrate different importance and extra bits could be wasted and interfering, this paper presents an adaptive bit allocation strategy for direct-trained SNNs, achieving fine-grained layer-wise allocation of memory and computation resources. Thus, SNN's efficiency and accuracy can be improved. Specifically, we parametrize the temporal lengths and the bit widths of weights and spikes, and make them learnable and controllable through gradients. To address the challenges caused by changeable bit widths and temporal lengths, we propose the refined spiking neuron, which can handle different temporal lengths, enable the derivation of gradients for temporal lengths, and suit spike quantization better. In addition, we theoretically formulate the step-size mismatch problem of learnable bit widths, which may incur severe quantization errors to SNN, and accordingly propose the step-size renewal mechanism to alleviate this issue. Experiments on various datasets, including the static CIFAR and ImageNet and the dynamic CIFAR-DVS and DVS-GESTURE, demonstrate that our methods can reduce the overall memory and computation cost while achieving higher accuracy. Particularly, our SEWResNet-34 can achieve a 2.69\% accuracy gain and 4.16$\times$ lower bit budgets over the advanced baseline work on ImageNet. This work will be fully open-sourced.
Abstract (translated)
最近，多比特脉冲神经网络（SNNs）已成为研究热点，旨在追求能源效率高且准确度高的AI。然而，随着涉及更多位数，相关的内存和计算需求上升到一定程度，性能改进变得不成比例。基于不同层表现出不同的重要性以及多余的位可能会浪费和相互干扰这一见解，本文提出了一种用于直接训练SNN的自适应位分配策略，实现了细粒度、逐层的内存和计算资源分配。因此，可以提高SNN的效率和准确性。 具体来说，我们参数化了权重和脉冲的时间长度以及位宽，并通过梯度使其可学习且可控。为了解决由变化的位宽和时间长度带来的挑战，我们提出了精炼的脉冲神经元模型，它可以处理不同的时间长度、支持时间长度的梯度导出，并更好地适应脉冲量化。此外，我们从理论上对可学习位宽度的步长不匹配问题进行了建模，这种问题可能会导致严重的量化误差影响SNN，因此提出了一种步长更新机制来缓解这一问题。 在包括静态CIFAR和ImageNet以及动态CIFAR-DVS和DVS-GESTURE等不同数据集上的实验表明，我们的方法可以在减少整体内存和计算成本的同时实现更高的准确度。特别是，在ImageNet上，我们的SEWResNet-34模型相比最先进的基线工作，获得了2.69%的精度提升，并且位预算减少了4.16倍。本研究将完全开源。
URL
https://arxiv.org/abs/2506.23717
