Abstract
Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at this https URL.
Abstract (translated)
未注册的高光谱图像（HSI）超分辨率（SR）通常旨在利用未经配准的高分辨率参考图像增强低分辨率的HSI。本文中，我们提出了一种基于解混的融合框架，该框架将空间-光谱信息分离出来，以同时缓解未注册融合的影响并提高SR模型的学习能力。具体而言，首先使用奇异值分解进行初始光谱解混，在保留原始端元的同时使后续网络专注于提升初始丰度图。为了利用未经配准参考图像的空间纹理，我们引入了一个从粗到细的可变形聚合模块，该模块先通过一个粗金字塔预测器估计像素级流和相似性图，然后执行精细亚像素修正以实现参考特征的可变形聚合。接着，聚合特征会经过一系列空间-通道丰度交叉注意力块进行细化处理。此外，还提出了一种基于空间-通道调制的融合模块，该模块利用动态门控权重合并编码器-解码器特征，生成高质量、高分辨率的HSI。在模拟和真实数据集上的实验结果确认了我们所提方法达到了当前最佳的超分辨率性能。代码将在此URL提供。
URL
https://arxiv.org/abs/2603.07918
