Abstract
The data bottleneck has emerged as a fundamental challenge in learning based image restoration methods. Researchers have attempted to generate synthesized training data using paired or unpaired samples to address this challenge. This study proposes SeNM-VAE, a semi-supervised noise modeling method that leverages both paired and unpaired datasets to generate realistic degraded data. Our approach is based on modeling the conditional distribution of degraded and clean images with a specially designed graphical model. Under the variational inference framework, we develop an objective function for handling both paired and unpaired data. We employ our method to generate paired training samples for real-world image denoising and super-resolution tasks. Our approach excels in the quality of synthetic degraded images compared to other unpaired and paired noise modeling methods. Furthermore, our approach demonstrates remarkable performance in downstream image restoration tasks, even with limited paired data. With more paired data, our method achieves the best performance on the SIDD dataset.
Abstract (translated)
数据瓶颈已成为基于图像修复方法的学习中的一个基本挑战。研究人员试图通过成对或非成对样本来生成合成训练数据来解决这个挑战。本研究提出了一种半监督噪声建模方法——SeNM-VAE，该方法利用成对和未成对数据集来生成真实 degradation数据。我们的方法基于使用专门设计的图形模型建模降解和清洁图像的条件分布。在变分推理框架下，我们开发了一个处理成对和未成对数据的共同目标函数。我们将该方法应用于真实世界图像去噪和超分辨率任务。与其它未成对和成对噪声建模方法相比，我们的方法在合成降解图像的质量方面具有卓越的表现。此外，即使只有很少的成对数据，我们的方法在下游图像修复任务中也表现出优异的性能。随着更多成对数据的增加，我们的方法在SIDD数据集上实现最佳性能。
URL
https://arxiv.org/abs/2403.17502
