Abstract
Diffusion models (DM) have achieved remarkable promise in image super-resolution (SR). However, most of them are tailored to solving non-blind inverse problems with fixed known degradation settings, limiting their adaptability to real-world applications that involve complex unknown degradations. In this work, we propose BlindDiff, a DM-based blind SR method to tackle the blind degradation settings in SISR. BlindDiff seamlessly integrates the MAP-based optimization into DMs, which constructs a joint distribution of the low-resolution (LR) observation, high-resolution (HR) data, and degradation kernels for the data and kernel priors, and solves the blind SR problem by unfolding MAP approach along with the reverse process. Unlike most DMs, BlindDiff firstly presents a modulated conditional transformer (MCFormer) that is pre-trained with noise and kernel constraints, further serving as a posterior sampler to provide both priors simultaneously. Then, we plug a simple yet effective kernel-aware gradient term between adjacent sampling iterations that guides the diffusion model to learn degradation consistency knowledge. This also enables to joint refine the degradation model as well as HR images by observing the previous denoised sample. With the MAP-based reverse diffusion process, we show that BlindDiff advocates alternate optimization for blur kernel estimation and HR image restoration in a mutual reinforcing manner. Experiments on both synthetic and real-world datasets show that BlindDiff achieves the state-of-the-art performance with significant model complexity reduction compared to recent DM-based methods. Code will be available at \url{this https URL}
Abstract (translated)
扩散模型（DM）在图像超分辨率（SR）方面取得了显著的进展。然而，大多数DM都是为了解决具有固定已知退化设置的非盲反问题，从而限制了其在现实世界涉及复杂未知退化的应用中的适应性。在这项工作中，我们提出了BlindDiff，一种基于DM的盲SR方法，用于解决SISR中的盲退化设置。BlindDiff无缝地将基于MAP的优化集成到DM中，构建了数据和核先验的联合分布，并利用展开的MAP方法与反向过程一起解决盲SR问题。与大多数DM不同，BlindDiff首先引入了一个预训练带有噪声和核约束的调制条件Transformer（MCFormer），作为后验采样器，同时提供核先验。然后，我们在相邻采样迭代之间插入一个简单的但有效的核关注度梯度项，引导扩散模型学习退化一致性知识。这还允许通过观察前 denoised 样本来联合优化降解模型和HR图像。与基于MAP的反向扩散过程相结合，我们证明了BlindDiff在模糊核估计和HR图像恢复方面主张交替优化。在合成和现实世界数据集上的实验表明，与最近的DM基于方法相比，BlindDiff在模型复杂度降低的情况下实现了最先进的性能。代码将在\url{这个https URL}上可用。
URL
https://arxiv.org/abs/2403.10211
