Abstract
Hyperspectral images super-resolution aims to improve the spatial resolution, yet its performance is often limited at high-resolution ratios. The recent adoption of high-resolution reference images for super-resolution is driven by the poor spatial detail found in low-resolution HSIs, presenting it as a favorable method. However, these approaches cannot effectively utilize information from the reference image, due to the inaccuracy of alignment and its inadequate interaction between alignment and fusion modules. In this paper, we introduce a Spatial-Spectral Concordance Hyperspectral Super-Resolution (SSC-HSR) framework for unaligned reference RGB guided HSI SR to address the issues of inaccurate alignment and poor interactivity of the previous approaches. Specifically, to ensure spatial concordance, i.e., align images more accurately across resolutions and refine textures, we construct a Two-Stage Image Alignment with a synthetic generation pipeline in the image alignment module, where the fine-tuned optical flow model can produce a more accurate optical flow in the first stage and warp model can refine damaged textures in the second stage. To enhance the interaction between alignment and fusion modules and ensure spectral concordance during reconstruction, we propose a Feature Aggregation module and an Attention Fusion module. In the feature aggregation module, we introduce an Iterative Deformable Feature Aggregation block to achieve significant feature matching and texture aggregation with the fusion multi-scale results guidance, iteratively generating learnable offset. Besides, we introduce two basic spectral-wise attention blocks in the attention fusion module to model the inter-spectra interactions. Extensive experiments on three natural or remote-sensing datasets show that our method outperforms state-of-the-art approaches on both quantitative and qualitative evaluations.
Abstract (translated)
超光谱图像的超分辨率处理旨在提高空间分辨率，但其性能在高比例尺时往往受限。近年来，采用高分辨率参考图像是为了克服低分辨率HSI（超光谱图像）中发现的空间细节不足的问题，这被认为是一种有利的方法。然而，这些方法无法有效地利用参考图的信息，原因在于对齐的不准确性以及对齐和融合模块之间交互不足。 在本文中，我们提出了一种新的空间-光谱一致性超光谱超分辨率（SSC-HSR）框架，该框架用于未对准的RGB图像引导HSI SR，并解决了以往方法中存在的对齐不准确及互动性差的问题。具体而言，为了确保空间一致性，即跨不同分辨率更精确地对齐图像并细化纹理，我们在图像对齐模块中构建了一个两阶段图像对齐与合成生成管道，其中经过微调的光流模型可以在第一阶段产生更为精确的光流，而变形模型则在第二阶段可以修复受损的纹理。为了增强对齐和融合模块之间的互动性，并确保重建过程中的光谱一致性，我们还提出了特征聚合模块和注意力融合模块。 在特征聚合模块中，我们引入了迭代可变形特征聚合块（Iterative Deformable Feature Aggregation block），该块可以在多尺度结果的指导下实现显著的特征匹配与纹理聚合，并且通过迭代生成学习到偏移量。此外，在注意力融合模块中，我们还介绍了两个基本的光谱注意块来建模不同光谱之间的相互作用。 在三个自然或遥感数据集上的大量实验表明，我们的方法在定量和定性评估方面均优于最先进的方法。
URL
https://arxiv.org/abs/2505.02109
