Paper Reading AI Learner

Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration

2024-10-01 05:54:07
Guy Ohayon, Tomer Michaeli, Michael Elad

Abstract

Photo-realistic image restoration algorithms are typically evaluated by distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures (e.g., FID, NIQE), where the desire is to attain the lowest possible distortion without compromising on perceptual quality. To achieve this goal, current methods typically attempt to sample from the posterior distribution, or to optimize a weighted sum of a distortion loss (e.g., MSE) and a perceptual quality loss (e.g., GAN). Unlike previous works, this paper is concerned specifically with the optimal estimator that minimizes the MSE under a constraint of perfect perceptual index, namely where the distribution of the reconstructed images is equal to that of the ground-truth ones. A recent theoretical result shows that such an estimator can be constructed by optimally transporting the posterior mean prediction (MMSE estimate) to the distribution of the ground-truth images. Inspired by this result, we introduce Posterior-Mean Rectified Flow (PMRF), a simple yet highly effective algorithm that approximates this optimal estimator. In particular, PMRF first predicts the posterior mean, and then transports the result to a high-quality image using a rectified flow model that approximates the desired optimal transport map. We investigate the theoretical utility of PMRF and demonstrate that it consistently outperforms previous methods on a variety of image restoration tasks.

Abstract (translated)

通常,照片现实感的图像修复算法通过畸变度量(如 PSNR 和 SSIM)以及感知质量度量(如 FID 和 NIQE)进行评估。其目标是在不牺牲感知质量的情况下实现最低的畸变。为了实现这一目标,现有方法通常尝试从后验分布中采样,或者通过优化一个加权畸变损失(如 MSE)和一个感知质量损失(如 GAN)来优化。与之前的工作不同,本文关注的是在满足完美感知指数的约束条件下,实现最低 MSE 的最优估计器,即重建图像的分布与真实图像的分布相等。 最近的一个理论结果表明,可以通过通过最优地传输后验均预测(MMSE估计)到真实图像分布来构建这样的最优估计器。受到这一结果的启发,我们引入了后验均值平滑流动(PMRF)算法,这是一种简单而高效的照片现实感图像修复算法。 特别是,PMRF 首先预测后验均值,然后使用平滑流动模型将结果传输到具有仿真实定理的图像中。我们研究了 PMRF 的理论应用价值,并证明了它在各种图像修复任务上 consistently优于之前的方法。

URL

https://arxiv.org/abs/2410.00418

PDF

https://arxiv.org/pdf/2410.00418.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot