Abstract
Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training, while more complex cases could happen in the real world. This gap between the assumed and actual degradation hurts the restoration performance where artifacts are often observed in the output. However, it is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. To tackle this robustness issue, we propose Diffusion-based Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image. By leveraging a well-performing denoising diffusion probabilistic model, our DR2 diffuses input images to a noisy status where various types of degradation give way to Gaussian noise, and then captures semantic information through iterative denoising steps. As a result, DR2 is robust against common degradation (e.g. blur, resize, noise and compression) and compatible with different designs of enhancement modules. Experiments in various settings show that our framework outperforms state-of-the-art methods on heavily degraded synthetic and real-world datasets.
Abstract (translated)
Blind face restoration通常将质量下降的低质量数据与预先定义的退化模型一起合成用于训练,而在现实生活中可能会出现更复杂的情况。这种假设和实际退化之间的差距常常导致恢复性能的损失,在输出中常常观察到 artifacts。然而,将每种类型的退化都包括在训练数据中是非常昂贵和不可能的。为了解决这个问题的稳健性问题,我们提出了基于扩散的稳健退化去除器(DR2)。首先将退化图像转换为一个粗但退化不变的预测,然后使用增强模块将粗预测恢复为高质量的图像。通过利用表现良好的去噪扩散概率模型,我们的DR2将输入图像扩散到噪声状态,其中各种不同类型的退化让高斯噪声取代,然后通过迭代去噪步骤捕获语义信息。因此,DR2对常见的退化(例如模糊、缩放、噪声和压缩)具有鲁棒性,并与不同的增强模块设计兼容。在各种设置下的实验表明,我们的框架在严重退化的合成数据和实际数据集上优于最先进的方法。
URL
https://arxiv.org/abs/2303.06885
