Abstract
This paper addresses the fusion of a pair of spatially unregistered hyperspectral image (HSI) and multispectral image (MSI) covering roughly overlapping regions. HSIs offer high spectral but low spatial resolution, while MSIs provide the opposite. The goal is to integrate their complementary information to enhance both HSI spatial resolution and MSI spectral resolution. While hyperspectral-multispectral fusion (HMF) has been widely studied, the unregistered setting remains challenging. Many existing methods focus solely on MSI super-resolution, leaving HSI unchanged. Supervised deep learning approaches were proposed for HSI super-resolution, but rely on accurate training data, which is often unavailable. Moreover, theoretical analyses largely address the co-registered case, leaving unregistered HMF poorly understood. In this work, an unsupervised framework is proposed to simultaneously super-resolve both MSI and HSI. The method integrates coupled spectral unmixing for MSI super-resolution with latent-space adversarial learning for HSI super-resolution. Theoretical guarantees on the recoverability of the super-resolution MSI and HSI are established under reasonable generative models -- providing, to our best knowledge, the first such insights for unregistered HMF. The approach is validated on semi-real and real HSI-MSI pairs across diverse conditions.
Abstract (translated)
本文针对空间未配准的高光谱图像（HSI）与多光谱图像（MSI）融合问题展开研究，两者覆盖区域大致重叠。高光谱图像具有高光谱分辨率但空间分辨率低，而多光谱图像则相反。研究目标在于整合二者互补信息，以同时提升高光谱图像的空间分辨率与多光谱图像的光谱分辨率。尽管高光谱-多光谱融合（HMF）已得到广泛研究，但未配准场景仍具挑战性。现有方法多仅关注多光谱图像超分辨率，未改变高光谱图像质量。虽有监督深度学习方法被提出用于高光谱超分辨率，但其依赖难以获取的精确训练数据。此外，现有理论分析多针对配准场景，未配准条件下的融合问题尚缺乏深入理解。 本研究提出一种无监督框架，可同时对多光谱与高光谱图像进行超分辨率重建。该方法将耦合光谱解混用于多光谱超分辨率，并结合潜在空间对抗学习实现高光谱超分辨率。在合理的生成模型下，本文建立了超分辨率重建多光谱与高光谱图像可恢复性的理论保证——据我们所知，这是首个针对未配准高光谱-多光谱融合的理论分析。通过在多样条件下对半真实与真实HSI-MSI数据对进行验证，证实了该方法的有效性。
URL
https://arxiv.org/abs/2603.21510
