Abstract
Learning from the interaction is the primary way biological agents know about the environment and themselves. Modern deep reinforcement learning (DRL) explores a computational approach to learning from interaction and has significantly progressed in solving various tasks. However, the powerful DRL is still far from biological agents in energy efficiency. Although the underlying mechanisms are not fully understood, we believe that the integration of spiking communication between neurons and biologically-plausible synaptic plasticity plays a prominent role. Following this biological intuition, we optimize a spiking policy network (SPN) by a genetic algorithm as an energy-efficient alternative to DRL. Our SPN mimics the sensorimotor neuron pathway of insects and communicates through event-based spikes. Inspired by biological research that the brain forms memories by forming new synaptic connections and rewires these connections based on new experiences, we tune the synaptic connections instead of weights in SPN to solve given tasks. Experimental results on several robotic control tasks show that our method can achieve the performance level of mainstream DRL methods and exhibit significantly higher energy efficiency.
Abstract (translated)
学习从互动是生物Agent了解自身和环境的主要方式。现代深度强化学习(DRL)探索了从互动中学习的计算方式,并在解决各种任务方面取得了显著进展。然而,强大的DRL仍然比生物Agent在能源效率上落后。尽管基本机制尚未完全理解,但我们相信,神经元之间的突触交流和生物学上可信的神经递质可塑性在Integration中发挥了重要作用。遵循这种生物学直觉,我们使用遗传算法优化了突触行为网络(SPN),将其作为DRL的高效替代方案。我们的SPN模拟了昆虫感觉运动神经元路径,通过基于事件的突触交流进行通信。受到生物学研究启发,大脑通过建立新的突触连接并基于新经验重构这些连接来形成记忆,因此我们在SPN中优化了突触连接,而不是权重,以解决给定任务。对多个机器人控制任务的实验结果表明,我们的方法可以实现主流DRL方法的性能水平,并表现出显著的更高效。
URL
https://arxiv.org/abs/2301.10292
