Abstract
Based on Stochastic Gradient Descent (SGD), the paper introduces two optimizers, named Interpolational Accelerating Gradient Descent (IAGD) as well as Noise-Regularized Stochastic Gradient Descent (NRSGD). IAGD leverages second-order Newton Interpolation to expedite the convergence process during training, assuming relevancy in gradients between iterations. To avoid over-fitting, NRSGD incorporates a noise regularization technique that introduces controlled noise to the gradients during the optimization process. Comparative experiments of this research are conducted on the CIFAR-10, and CIFAR-100 datasets, benchmarking different CNNs(Convolutional Neural Networks) with IAGD and NRSGD against classical optimizers in Keras Package. Results demonstrate the potential of those two viable improvement methods in SGD, implicating the effectiveness of the advancements.
Abstract (translated)
基于随机梯度下降（SGD）算法，该论文提出了两种优化器：插值加速梯度下降法(IAGD)和噪声正则化随机梯度下降法(NRSGD)。IAGD利用了二阶牛顿插值技术来加快训练过程中的收敛速度，并假设不同迭代之间的梯度相关性。为了防止过拟合，NRSGD在优化过程中引入了一种噪声正则化技术，通过控制性的添加噪声到梯度中实现。 该研究的比较实验是在CIFAR-10和CIFAR-100数据集上进行的，并且使用了不同的卷积神经网络（CNNs）与Keras包中的经典优化器进行对比测试。结果表明，这两种改进SGD的方法具有潜在的价值，证明了这些方法的有效性及进步意义。
URL
https://arxiv.org/abs/2510.13040
