Abstract
Photo-realistic image restoration algorithms are typically evaluated by distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures (e.g., FID, NIQE), where the desire is to attain the lowest possible distortion without compromising on perceptual quality. To achieve this goal, current methods typically attempt to sample from the posterior distribution, or to optimize a weighted sum of a distortion loss (e.g., MSE) and a perceptual quality loss (e.g., GAN). Unlike previous works, this paper is concerned specifically with the optimal estimator that minimizes the MSE under a constraint of perfect perceptual index, namely where the distribution of the reconstructed images is equal to that of the ground-truth ones. A recent theoretical result shows that such an estimator can be constructed by optimally transporting the posterior mean prediction (MMSE estimate) to the distribution of the ground-truth images. Inspired by this result, we introduce Posterior-Mean Rectified Flow (PMRF), a simple yet highly effective algorithm that approximates this optimal estimator. In particular, PMRF first predicts the posterior mean, and then transports the result to a high-quality image using a rectified flow model that approximates the desired optimal transport map. We investigate the theoretical utility of PMRF and demonstrate that it consistently outperforms previous methods on a variety of image restoration tasks.
Abstract (translated)
通常,照片现实感的图像修复算法通过畸变度量(如 PSNR 和 SSIM)以及感知质量度量(如 FID 和 NIQE)进行评估。其目标是在不牺牲感知质量的情况下实现最低的畸变。为了实现这一目标,现有方法通常尝试从后验分布中采样,或者通过优化一个加权畸变损失(如 MSE)和一个感知质量损失(如 GAN)来优化。与之前的工作不同,本文关注的是在满足完美感知指数的约束条件下,实现最低 MSE 的最优估计器,即重建图像的分布与真实图像的分布相等。 最近的一个理论结果表明,可以通过通过最优地传输后验均预测(MMSE估计)到真实图像分布来构建这样的最优估计器。受到这一结果的启发,我们引入了后验均值平滑流动(PMRF)算法,这是一种简单而高效的照片现实感图像修复算法。 特别是,PMRF 首先预测后验均值,然后使用平滑流动模型将结果传输到具有仿真实定理的图像中。我们研究了 PMRF 的理论应用价值,并证明了它在各种图像修复任务上 consistently优于之前的方法。
URL
https://arxiv.org/abs/2410.00418