Abstract
Bio-inspired neural networks are attractive for their adversarial robustness, energy frugality, and closer alignment with cortical physiology, yet they often lag behind back-propagation (BP) based models in accuracy and ability to scale. We show that allowing the use of different bio-inspired learning rules in different layers, discovered automatically by a tailored neural-architecture-search (NAS) procedure, bridges this gap. Starting from standard NAS baselines, we enlarge the search space to include bio-inspired learning rules and use NAS to find the best architecture and learning rule to use in each layer. We show that neural networks that use different bio-inspired learning rules for different layers have better accuracy than those that use a single rule across all the layers. The resulting NN that uses a mix of bio-inspired learning rules sets new records for bio-inspired models: 95.16% on CIFAR-10, 76.48% on CIFAR-100, 43.42% on ImageNet16-120, and 60.51% top-1 on ImageNet. In some regimes, they even surpass comparable BP-based networks while retaining their robustness advantages. Our results suggest that layer-wise diversity in learning rules allows better scalability and accuracy, and motivates further research on mixing multiple bio-inspired learning rules in the same network.
Abstract (translated)
受生物启发的神经网络因其对抗鲁棒性、节能性和与皮质生理学更接近的一致性而受到青睐，但它们在准确性及扩展能力方面通常落后于基于反向传播（BP）的模型。我们证明，允许不同层使用由定制化的神经架构搜索（NAS）过程自动发现的不同生物启发式学习规则，在一定程度上弥合了这一差距。从标准NAS基线开始，我们将搜索空间扩大以包括生物启发式学习规则，并利用NAS来寻找在每一层中使用的最佳架构和学习规则。 我们展示了使用不同生物启发式学习规则的神经网络比在整个网络中使用单一规则具有更高的准确性。这些混合了多种生物启发式学习规则的结果神经网络为基于生物启发的学习模型设定了新的记录：CIFAR-10上的准确率为95.16%，CIFAR-100上的准确率为76.48%，ImageNet16-120上的准确率为43.42%，以及在ImageNet上达到60.51%的top-1准确率。在某些情况下，这些模型甚至超越了与其性能相当的BP基网络，同时保持了它们的稳健性优势。 我们的研究结果表明，在不同层使用多样化的学习规则有助于提高扩展性和准确性，并鼓励进一步探索在同一神经网络中混合多种生物启发式学习规则的研究方向。
URL
https://arxiv.org/abs/2507.13485
