Abstract
Existing real-world super-resolution (RSR) methods based on generative priors have achieved remarkable progress in producing high-quality and globally consistent reconstructions. However, they often struggle to recover fine-grained details of diverse object instances in complex real-world scenes. This limitation primarily arises because commonly adopted denoising losses (e.g., MSE) inherently favor global consistency while neglecting instance-level perception and restoration. To address this issue, we propose InstanceRSR, a novel RSR framework that jointly models semantic information and introduces instance-level feature alignment. Specifically, we employ low-resolution (LR) images as global consistency guidance while jointly modeling image data and semantic segmentation maps to enforce semantic relevance during sampling. Moreover, we design an instance representation learning module to align the diffusion latent space with the instance latent space, enabling instance-aware feature alignment, and further incorporate a scale alignment mechanism to enhance fine-grained perception and detail recovery. Benefiting from these designs, our approach not only generates photorealistic details but also preserves semantic consistency at the instance level. Extensive experiments on multiple real-world benchmarks demonstrate that InstanceRSR significantly outperforms existing methods in both quantitative metrics and visual quality, achieving new state-of-the-art (SOTA) performance.
Abstract (translated)
现有基于生成先验的真实世界超分辨率（RSR）方法在生成高质量、全局一致的图像重建方面取得了显著进展。然而，它们在复杂真实场景中恢复多样目标实例的细粒度细节方面往往存在不足。这一局限性主要源于常用的去噪损失函数（如MSE）本质上倾向于全局一致性，而忽略了实例级的感知与修复。针对该问题，我们提出了InstanceRSR，一种新型RSR框架，通过联合建模语义信息并引入实例级特征对齐来解决问题。具体而言，我们采用低分辨率（LR）图像作为全局一致性指导，同时联合建模图像数据与语义分割图，以在采样过程中强化语义相关性。此外，我们设计了实例表征学习模块，将扩散隐空间与实例隐空间进行对齐，实现实例感知的特征对齐，并进一步引入尺度对齐机制以增强细粒度感知与细节恢复能力。得益于上述设计，我们的方法不仅能生成照片般真实的细节，还能在实例层面保持语义一致性。在多个真实场景基准数据集上的大量实验表明，InstanceRSR在定量指标与视觉质量上均显著优于现有方法，达到了新的最先进（SOTA）性能。
URL
https://arxiv.org/abs/2603.24240
