Abstract
Multimodal large models have shown excellent ability in addressing image super-resolution in real-world scenarios by leveraging language class as condition information, yet their abilities in degraded images remain limited. In this paper, we first revisit the capabilities of the Recognize Anything Model (RAM) for degraded images by calculating text similarity. We find that directly using contrastive learning to fine-tune RAM in the degraded space is difficult to achieve acceptable results. To address this issue, we employ a degradation selection strategy to propose a Real Embedding Extractor (REE), which achieves significant recognition performance gain on degraded image content through contrastive learning. Furthermore, we use a Conditional Feature Modulator (CFM) to incorporate the high-level information of REE for a powerful Mamba-based network, which can leverage effective pixel information to restore image textures and produce visually pleasing results. Extensive experiments demonstrate that the REE can effectively help image super-resolution networks balance fidelity and perceptual quality, highlighting the great potential of Mamba in real-world applications. The source code of this work will be made publicly available at: this https URL
Abstract (translated)
多模态大型模型在通过利用语言类别作为条件信息来处理现实场景中的图像超分辨率问题时展现了出色的能力，但它们对退化图像的处理能力仍然有限。在这篇论文中，我们首先通过计算文本相似度重新审视了识别任何模型（RAM）在退化图像上的性能表现。我们发现直接使用对比学习对RAM进行微调以适应退化空间难以获得满意的结果。为了解决这个问题，我们采用了一种退化选择策略，提出了真实嵌入提取器（REE），通过对比学习，它能在退化图像内容上实现显著的识别性能提升。 此外，我们还利用了条件特征调节器（CFM）将REE的高级信息融入到强大的基于Mamba网络中，该网络能够利用有效的像素信息来恢复图像纹理，并生成视觉上令人愉悦的结果。广泛的实验表明，REE可以有效地帮助图像超分辨率网络在保真度和感知质量之间取得平衡，突显了Mamba在网络真实应用场景中的巨大潜力。这项工作的源代码将公开发布在：[提供的URL]
URL
https://arxiv.org/abs/2602.23890
