Abstract
In recent years, we have witnessed the great advancement of Deep neural networks (DNNs) in image restoration. However, a critical limitation is that they cannot generalize well to real-world degradations with different degrees or types. In this paper, we are the first to propose a novel training strategy for image restoration from the causality perspective, to improve the generalization ability of DNNs for unknown degradations. Our method, termed Distortion Invariant representation Learning (DIL), treats each distortion type and degree as one specific confounder, and learns the distortion-invariant representation by eliminating the harmful confounding effect of each degradation. We derive our DIL with the back-door criterion in causality by modeling the interventions of different distortions from the optimization perspective. Particularly, we introduce counterfactual distortion augmentation to simulate the virtual distortion types and degrees as the confounders. Then, we instantiate the intervention of each distortion with a virtual model updating based on corresponding distorted images, and eliminate them from the meta-learning perspective. Extensive experiments demonstrate the effectiveness of our DIL on the generalization capability for unseen distortion types and degrees. Our code will be available at this https URL.
Abstract (translated)
近年来,我们见证了深度学习神经网络(DNN)在图像恢复方面的重大突破。然而,一个关键限制是,它们无法很好地适用于不同程度或类型的实际退化。在本文中,我们首先提出了从因果关系的角度提出的一种新的图像恢复训练策略,以提高未知退化的泛化能力。我们的方法被称为“失真不变表示学习(DIL)”,将每个失真类型和程度视为一个特定的干扰变量,并通过消除每个退化的有害混淆效应来学习失真不变的表示。我们从优化的角度来看建模不同失真的影响。特别地,我们引入了反事实失真增强来模拟虚拟失真类型和程度作为干扰变量。然后,我们基于相应的失真图像更新虚拟模型,并从元学习的角度来看消除每个失真的影响。广泛的实验表明,我们的DIL对于未观察到的失真类型和程度的泛化能力的有效性。我们的代码将在这个httpsURL上可用。
URL
https://arxiv.org/abs/2303.06859
