Abstract
We study cross-modal super-resolution (SR) on real-world misaligned data, where only a limited number of low-resolution (LR) source and high-resolution (HR) guide image pairs with complex spatial misalignments are available. To address this challenge, we propose RobSelf--a fully self-supervised model that is optimized online, requiring no training data, ground-truth supervision, or pre-alignment. RobSelf features two key techniques: a misalignment-aware feature translator and a content-aware reference filter. The translator reformulates unsupervised cross-modal and cross-resolution alignment as a weakly-supervised, misalignment-aware translation subtask, producing an aligned guide feature with inherent redundancy. Guided by this feature, the filter performs reference-based discriminative self-enhancement on the source, enabling SR predictions with high resolution and high fidelity. Across a variety of tasks, we demonstrate that RobSelf achieves state-of-the-art performance and superior efficiency. Additionally, we introduce a real-world dataset, RealMisSR, to advance research on this topic. Dataset and code: this https URL.
Abstract (translated)
我们研究了在现实世界中不匹配数据上的跨模态超分辨率（SR）问题，其中仅提供了少量低分辨率（LR）源图像与高分辨率（HR）引导图像的配对组合，并且这些图像存在复杂的空间错位。为了解决这一挑战，我们提出了一种名为RobSelf的完全自监督模型，该模型在线优化，无需训练数据、真实标签监督或预对齐步骤。 RobSelf具备两个关键技术： 1. 错位感知特征翻译器：将无监督跨模态和跨分辨率对齐重新定义为一个弱监督、错位感知的翻译子任务。此过程会产生一种包含冗余信息且与源图像对齐的引导特征。 2. 内容感知参考过滤器：基于上述生成的特征，该过滤器执行以参考为基础的判别性自我增强操作，从而在源图像上实现高分辨率和高质量的超分辨率预测。 我们在多种任务中展示了RobSelf取得了最先进的性能，并且具有卓越的效率。此外，我们还引入了一个真实世界的实验数据集RealMisSR，用以推进该领域的研究进展。有关数据集和代码，请参见此链接：this https URL.
URL
https://arxiv.org/abs/2602.18822
