Abstract
The paper presents the formulation, implementation, and evaluation of the ArcGD optimiser. The evaluation is conducted initially on a non-convex benchmark function and subsequently on a real-world ML dataset. The initial comparative study using the Adam optimiser is conducted on a stochastic variant of the highly non-convex and notoriously challenging Rosenbrock function, renowned for its narrow, curved valley, across dimensions ranging from 2D to 1000D and an extreme case of 50,000D. Two configurations were evaluated to eliminate learning-rate bias: (i) both using ArcGD's effective learning rate and (ii) both using Adam's default learning rate. ArcGD consistently outperformed Adam under the first setting and, although slower under the second, achieved super ior final solutions in most cases. In the second evaluation, ArcGD is evaluated against state-of-the-art optimizers (Adam, AdamW, Lion, SGD) on the CIFAR-10 image classification dataset across 8 diverse MLP architectures ranging from 1 to 5 hidden layers. ArcGD achieved the highest average test accuracy (50.7%) at 20,000 iterations, outperforming AdamW (46.6%), Adam (46.8%), SGD (49.6%), and Lion (43.4%), winning or tying on 6 of 8 architectures. Notably, while Adam and AdamW showed strong early convergence at 5,000 iterations, but regressed with extended training, whereas ArcGD continued improving, demonstrating generalization and resistance to overfitting without requiring early stopping tuning. Strong performance on geometric stress tests and standard deep-learning benchmarks indicates broad applicability, highlighting the need for further exploration. Moreover, it is also shown that a variant of ArcGD can be interpreted as a special case of the Lion optimiser, highlighting connections between the inherent mechanisms of such optimisation methods.
Abstract (translated)
这篇论文介绍了ArcGD优化器的公式化、实现和评估。评估首先在一个非凸基准函数上进行，随后在实际机器学习数据集上进行。最初的比较研究使用Adam优化器，在高度非凸且极具挑战性的Rosenbrock函数（以其狭窄而弯曲的山谷著称）的一个随机变体上展开，测试维度从2D到1000D不等，并包括极端情况下的50,000D。为了消除学习率偏差，评估了两种配置：(i) 两者均使用ArcGD的有效学习率；(ii) 两者均使用Adam的默认学习率。在第一种设置下，ArcGD始终优于Adam，在第二种设置下虽然速度较慢，但在大多数情况下仍达到了更优的最终解决方案。 在第二次评估中，ArcGD与最先进的优化器（包括Adam、AdamW、Lion和SGD）在CIFAR-10图像分类数据集上进行比较，涵盖了8种不同的多层感知机架构，从单隐藏层到五隐藏层。经过20,000次迭代后，ArcGD达到了最高的平均测试准确率（50.7%），超过了AdamW（46.6%）、Adam（46.8%）、SGD（49.6%）和Lion（43.4%），在八个架构中的六个中胜出或打平。值得注意的是，尽管Adam和AdamW在初始的5,000次迭代时表现出强烈的快速收敛性，在进一步训练后却退步了，而ArcGD则持续改进，显示出泛化能力和对过拟合的抗性，并且不需要提前停止调整。 几何压力测试和标准深度学习基准上的强劲表现表明其广泛适用性，突显了需要进行更多探索。此外，研究表明ArcGD的一个变体可以解释为Lion优化器的一种特殊情况，强调了此类优化方法内在机制之间的联系。
URL
https://arxiv.org/abs/2512.06737
