Abstract
Diffusion generative modelling (DGM) based on stochastic differential equations (SDEs) with score matching has achieved unprecedented results in data generation. In this paper, we propose a novel fast high-quality generative modelling method based on high-order Langevin dynamics (HOLD) with score matching. This motive is proved by third-order Langevin dynamics. By augmenting the previous SDEs, e.g. variance exploding or variance preserving SDEs for single-data variable processes, HOLD can simultaneously model position, velocity, and acceleration, thereby improving the quality and speed of the data generation at the same time. HOLD is composed of one Ornstein-Uhlenbeck process and two Hamiltonians, which reduce the mixing time by two orders of magnitude. Empirical experiments for unconditional image generation on the public data set CIFAR-10 and CelebA-HQ show that the effect is significant in both Frechet inception distance (FID) and negative log-likelihood, and achieves the state-of-the-art FID of 1.85 on CIFAR-10.
Abstract (translated)
基于随机微分方程（SDEs）的扩散生成建模（DGM）取得了空前的数据生成结果。在本文中，我们提出了一种基于高阶Langevin动力（HOLD）的新的快速高质量生成建模方法。这一动机由高阶Langevin动力证明。通过增加以前SDEs，例如单数据变量过程的方差爆炸或方差保持SDE，HOLD可以同时建模位置、速度和加速度，从而提高数据生成的质量和速度。HOLD由一个Ornstein-Uhlenbeck过程和两个哈密顿组成，它们减少了混合时间的两倍。在公共数据集CIFAR-10和CelebA-HQ上进行无条件图像生成的实证实验表明，该效果在弗雷歇创新距离（FID）和负对数似然上都有显著影响，并在CIFAR-10上实现了最先进的FID值1.85。
URL
https://arxiv.org/abs/2404.12814
