Abstract
Predictive coding (PC) is an influential theory of information processing in the brain, providing a biologically plausible alternative to backpropagation. It is motivated in terms of Bayesian inference, as hidden states and parameters are optimised via gradient descent on variational free energy. However, implementations of PC rely on maximum \textit{a posteriori} (MAP) estimates of hidden states and maximum likelihood (ML) estimates of parameters, limiting their ability to quantify epistemic uncertainty. In this work, we investigate a Bayesian extension to PC that estimates a posterior distribution over network parameters. This approach, termed Bayesian Predictive coding (BPC), preserves the locality of PC and results in closed-form Hebbian weight updates. Compared to PC, our BPC algorithm converges in fewer epochs in the full-batch setting and remains competitive in the mini-batch setting. Additionally, we demonstrate that BPC offers uncertainty quantification comparable to existing methods in Bayesian deep learning, while also improving convergence properties. Together, these results suggest that BPC provides a biologically plausible method for Bayesian learning in the brain, as well as an attractive approach to uncertainty quantification in deep learning.
Abstract (translated)
预测编码（Predictive Coding，简称PC）是大脑信息处理的一个有影响力的理论模型，提供了一种生物上合理的反向传播算法替代方案。它基于贝叶斯推理原理，在变分自由能量的梯度下降过程中优化隐含状态和参数。然而，现有的预测编码实现主要依赖于最大后验概率（MAP）来估计隐藏状态，并使用最大似然法（ML）来估计参数，这限制了它们量化知识不确定性（epistemic uncertainty）的能力。 在这项工作中，我们探讨了一种贝叶斯扩展的预测编码方法，该方法可以对网络参数进行后验分布的估计。这种方法被称为贝叶斯预测编码（Bayesian Predictive Coding, 简称BPC），它保持了预测编码的局部性，并且生成了封闭形式的赫布式权重更新公式。 相比传统的预测编码算法，在全批次设置下，我们的BPC算法在更少的迭代周期内收敛；而在小批量设置下，该方法仍然具有竞争力。此外，我们还展示了BPC能够在不确定性量化方面与现有的贝叶斯深度学习方法相媲美，并且还能改进其收敛性能。 这些结果表明，BPC为大脑中的贝叶斯学习提供了一种生物上合理的方案，同时在深度学习中提供了评估不确定性的有吸引力的方法。
URL
https://arxiv.org/abs/2503.24016
