Abstract
Dynamic Spectral Backpropagation (DSBP) enhances neural network training under resource constraints by projecting gradients onto principal eigenvectors, reducing complexity and promoting flat minima. Five extensions are proposed, dynamic spectral inference, spectral architecture optimization, spectral meta learning, spectral transfer regularization, and Lie algebra inspired dynamics, to address challenges in robustness, fewshot learning, and hardware efficiency. Supported by a third order stochastic differential equation (SDE) and a PAC Bayes limit, DSBP outperforms Sharpness Aware Minimization (SAM), Low Rank Adaptation (LoRA), and Model Agnostic Meta Learning (MAML) on CIFAR 10, Fashion MNIST, MedMNIST, and Tiny ImageNet, as demonstrated through extensive experiments and visualizations. Future work focuses on scalability, bias mitigation, and ethical considerations.
Abstract (translated)
动态光谱反向传播（DSBP）通过将梯度投影到主特征向量上来增强资源受限条件下的神经网络训练，从而降低复杂性并促进平坦极小值的形成。提出了五种扩展方法：动态光谱推理、光谱架构优化、光谱元学习、光谱迁移正则化和李代数启发的动力学，以应对鲁棒性、少量样本学习以及硬件效率方面的挑战。DSBP基于一个三阶随机微分方程（SDE）和支持概率可放大贝叶斯限的理论，其在CIFAR 10、Fashion MNIST、MedMNIST和Tiny ImageNet数据集上的性能超过了尖锐度感知最小化（SAM）、低秩适应（LoRA）和模型无关元学习（MAML），这一结论通过广泛的实验与可视化分析得到了验证。未来的工作将重点放在可扩展性、偏差缓解以及伦理考量上。
URL
https://arxiv.org/abs/2505.23369
