Abstract
Efficient training of artificial neural networks remains a key challenge in deep learning. Backpropagation (BP), the standard learning algorithm, relies on gradient descent and typically requires numerous iterations for convergence. In this study, we introduce Expectation Reflection (ER), a novel learning approach that updates weights multiplicatively based on the ratio of observed to predicted outputs. Unlike traditional methods, ER maintains consistency without requiring ad hoc loss functions or learning rate hyperparameters. We extend ER to multilayer networks and demonstrate its effectiveness in performing image classification tasks. Notably, ER achieves optimal weight updates in a single iteration. Additionally, we reinterpret ER as a modified form of gradient descent incorporating the inverse mapping of target propagation. These findings suggest that ER provides an efficient and scalable alternative for training neural networks.
Abstract (translated)
人工神经网络的高效训练仍然是深度学习中的一个关键挑战。反向传播（BP）作为标准的学习算法，依赖于梯度下降，并且通常需要多次迭代才能收敛。在本研究中，我们引入了期望反射（ER），这是一种新的学习方法，它根据观察到的输出与预测输出的比例来乘法更新权重。不同于传统的方法，ER能够在不使用临时损失函数或学习率超参数的情况下保持一致性。我们将ER扩展到了多层网络，并展示了其在执行图像分类任务方面的有效性。特别地，ER可以在一次迭代中实现最优的权重更新。此外，我们重新诠释了ER为一种结合目标传播逆映射的修改版梯度下降法。这些发现表明，ER为训练神经网络提供了一种高效且可扩展的替代方案。
URL
https://arxiv.org/abs/2503.10144
