Abstract
Diffusion models have recently gained traction as a powerful class of deep generative priors, excelling in a wide range of image restoration tasks due to their exceptional ability to model data distributions. To solve image restoration problems, many existing techniques achieve data consistency by incorporating additional likelihood gradient steps into the reverse sampling process of diffusion models. However, the additional gradient steps pose a challenge for real-world practical applications as they incur a large computational overhead, thereby increasing inference time. They also present additional difficulties when using accelerated diffusion model samplers, as the number of data consistency steps is limited by the number of reverse sampling steps. In this work, we propose a novel diffusion-based image restoration solver that addresses these issues by decoupling the reverse process from the data consistency steps. Our method involves alternating between a reconstruction phase to maintain data consistency and a refinement phase that enforces the prior via diffusion purification. Our approach demonstrates versatility, making it highly adaptable for efficient problem-solving in latent space. Additionally, it reduces the necessity for numerous sampling steps through the integration of consistency models. The efficacy of our approach is validated through comprehensive experiments across various image restoration tasks, including image denoising, deblurring, inpainting, and super-resolution.
Abstract (translated)
扩散模型最近因能够在图像修复任务中建模数据分布而获得了广泛关注，成为了一种强大的深度生成priors。由于其出色的数据分布建模能力，扩散模型在广泛的图像修复任务中表现出色。为解决图像修复问题，许多现有技术通过将扩散模型的反向采样过程引入 additional likelihood gradient 步骤来实现数据一致性。然而，这些额外的梯度步骤对现实世界的实际应用造成了较大的计算开销，从而增加了推理时间。当使用加速扩散模型采样器时，它们还提出了其他困难，因为数据一致性步骤的数量限制了反向采样步骤的数量。在本文中，我们提出了一种新型的基于扩散的图像修复求解器，通过将反向过程与数据一致性步骤解耦，从而解决了这些问题。我们的方法包括重建阶段和优化阶段。在重建阶段，我们维持数据一致性，通过扩散净化来实施先验。在优化阶段，我们通过扩散除杂来强制实施先验。通过将一致性模型集成到我们的方法中，我们减少了需要进行的采样步骤数量。我们通过在各种图像修复任务上进行全面的实验来验证我们方法的效力，包括图像去噪、去模糊、修复和超分辨率。
URL
https://arxiv.org/abs/2403.06054
