Abstract
Recent advances in image deraining have focused on training powerful models on mixed multiple datasets comprising diverse rain types and backgrounds. However, this approach tends to overlook the inherent differences among rainy images, leading to suboptimal results. To overcome this limitation, we focus on addressing various rainy images by delving into meaningful representations that encapsulate both the rain and background components. Leveraging these representations as instructive guidance, we put forth a Context-based Instance-level Modulation (CoI-M) mechanism adept at efficiently modulating CNN- or Transformer-based models. Furthermore, we devise a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and potent algorithm tailored for training models on mixed datasets. Moreover, CoIC offers insight into modeling relationships of datasets, quantitatively assessing the impact of rain and details on restoration, and unveiling distinct behaviors of models given diverse inputs. Extensive experiments validate the efficacy of CoIC in boosting the deraining ability of CNN and Transformer models. CoIC also enhances the deraining prowess remarkably when real-world dataset is included.
Abstract (translated)
近年来，图像去雾技术的发展主要集中在在包含多种雨类型和背景的混合数据集上训练强大的模型。然而，这种方法往往忽视了雨图片之间的固有差异，导致性能较低。为了克服这一局限，我们专注于通过深入挖掘有意义的表现来解决各种雨图片，从而实现更好的结果。利用这些表现作为有指导性的提示，我们提出了一个基于上下文的实例级调制（CoI-M）机制，该机制能够有效地对基于CNN或Transformer的模型进行调制。此外，我们还设计了一个雨-/细节敏感的对比学习策略，以帮助提取联合雨-/细节感知的表示。通过将CoI-M与雨-/细节感知的对比学习相结合，我们开发了CoIC，一种专为在混合数据集上训练模型而设计的创新且强大的算法。此外，CoIC揭示了数据集之间的建模关系，定量评估了雨和细节对恢复的影响，并揭示了给定不同输入的模型具有显著的差异行为。大量的实验证实了CoIC在提高CNN和Transformer模型的去雾能力方面的有效性。当包含真实世界数据时，CoIC的去雾能力显著增强。
URL
https://arxiv.org/abs/2404.12091
