Abstract
Diffusion models have recently received a surge of interest due to their impressive performance for image restoration, especially in terms of noise robustness. However, existing diffusion-based methods are trained on a large amount of training data and perform very well in-distribution, but can be quite susceptible to distribution shift. This is especially inappropriate for data-starved hyperspectral image (HSI) restoration. To tackle this problem, this work puts forth a self-supervised diffusion model for HSI restoration, namely Denoising Diffusion Spatio-Spectral Model (\texttt{DDS2M}), which works by inferring the parameters of the proposed Variational Spatio-Spectral Module (VS2M) during the reverse diffusion process, solely using the degraded HSI without any extra training data. In VS2M, a variational inference-based loss function is customized to enable the untrained spatial and spectral networks to learn the posterior distribution, which serves as the transitions of the sampling chain to help reverse the diffusion process. Benefiting from its self-supervised nature and the diffusion process, \texttt{DDS2M} enjoys stronger generalization ability to various HSIs compared to existing diffusion-based methods and superior robustness to noise compared to existing HSI restoration methods. Extensive experiments on HSI denoising, noisy HSI completion and super-resolution on a variety of HSIs demonstrate \texttt{DDS2M}'s superiority over the existing task-specific state-of-the-arts.
Abstract (translated)
扩散模型最近吸引了大量关注,因为它们在图像恢复方面表现出令人印象深刻的性能,特别是在噪声鲁棒性方面。然而,现有的扩散方法需要大量的训练数据进行训练,并在分布范围内表现得很好,但很容易受到分布偏差的影响。这对于缺乏训练数据的高分辨率图像恢复(HSI)来说尤其不合适。为了解决这一问题,这项工作提出了一种自我监督的扩散模型,即Denoising DiffusionSpatio-Spectral Model(DDS2M),它通过在反向扩散过程中推断提议的变分超平面模块(VS2M)的参数,仅使用退化的高分辨率图像,而不需要额外的训练数据。在VS2M中,一种变分推断based的损失函数被定制,以便未训练的空间和谱网络学习后分布,作为采样链的过渡,以帮助反向扩散过程。得益于其自我监督性和扩散过程,DDS2M对各种HSI的泛化能力比现有的扩散方法更强,对噪声的鲁棒性也比现有的HSI恢复方法更好。在HSI去噪、噪声HSI完成和超分辨率多个HSI样本上开展的实验表明,DDS2M比现有的任务特定标准方法更具优越性。
URL
https://arxiv.org/abs/2303.06682
