Abstract
Biological neural networks (like the hippocampus) can internally generate "replay" resembling stimulus-driven activity. Recent computational models of replay use noisy recurrent neural networks (RNNs) trained to path-integrate. Replay in these networks has been described as Langevin sampling, but new modifiers of noisy RNN replay have surpassed this description. We re-examine noisy RNN replay as sampling to understand or improve it in three ways: (1) Under simple assumptions, we prove that the gradients replay activity should follow are time-varying and difficult to estimate, but readily motivate the use of hidden state leakage in RNNs for replay. (2) We confirm that hidden state adaptation (negative feedback) encourages exploration in replay, but show that it incurs non-Markov sampling that also slows replay. (3) We propose the first model of temporally compressed replay in noisy path-integrating RNNs through hidden state momentum, connect it to underdamped Langevin sampling, and show that, together with adaptation, it counters slowness while maintaining exploration. We verify our findings via path-integration of 2D triangular and T-maze paths and of high-dimensional paths of synthetic rat place cell activity.
Abstract (translated)
生物学神经网络（如海马体）可以内部生成类似刺激驱动活动的“回放”。最近关于回放的计算模型使用了通过路径积分训练的带有噪声的递归神经网络（RNN）。在这些网络中的回放被描述为兰乔VIN采样，但对带噪RNN回放的新修正已经超越了这一描述。我们从三个角度重新审视带噪RNN回放作为采样的理解或改进：(1) 在简单的假设下，我们证明回放活动应该遵循的时间变化梯度难以估计，但却能轻易地推动在RNN中使用隐藏状态泄漏进行回放。(2) 我们确认了隐藏状态适应（负反馈）鼓励回放在探索中的作用，但同时也表明这导致非马尔可夫采样，并且减缓了回放的速度。(3) 我们提出首个通过隐藏状态动量在带噪路径积分RNN中实现时间压缩回放的模型，并将其与欠阻尼兰乔VIN采样联系起来，展示出它结合适应性时能克服缓慢性并保持探索。我们通过二维三角和T形迷宫以及合成大鼠位置细胞活动的高维路径中的路径积来验证我们的发现。
URL
https://arxiv.org/abs/2602.18401
