Abstract
This work explores optimization methods on hyperbolic manifolds. Building on Riemannian optimization principles, we extend the Hyperbolic Stochastic Gradient Descent (a specialization of Riemannian SGD) to a Hyperbolic Adam optimizer. While these methods are particularly relevant for learning on the Poincaré ball, they may also provide benefits in Euclidean and other non-Euclidean settings, as the chosen optimization encourages the learning of Poincaré embeddings. This representation, in turn, accelerates convergence in the early stages of training, when parameters are far from the optimum. As a case study, we train diffusion models using the hyperbolic optimization methods with hyperbolic time-discretization of the Langevin dynamics, and show that they achieve faster convergence on certain datasets without sacrificing generative quality.
Abstract (translated)
这项工作探讨了双曲流形上的优化方法。基于黎曼优化原则，我们将双曲随机梯度下降（黎曼随机梯度下降的一种特例）扩展为双曲Adam优化器。虽然这些方法特别适用于庞加莱球面上的学习，但它们也可能在欧几里得和其他非欧几里得环境中提供好处，因为所选择的优化鼓励学习庞加莱嵌入。这种表示反过来可以在训练初期参数远离最优解时加速收敛。作为案例研究，我们使用双曲优化方法和朗之万动力学的双曲时间离散化来训练扩散模型，并展示了它们在某些数据集上可以实现更快的收敛速度而不牺牲生成质量。
URL
https://arxiv.org/abs/2509.25206
