Abstract
With the advent of Generative AI, Single Image Super-Resolution (SISR) quality has seen substantial improvement, as the strong priors learned by Text-2-Image Diffusion (T2IDiff) Foundation Models (FM) can bridge the gap between High-Resolution (HR) and Low-Resolution (LR) images. However, flagship smartphone cameras have been slow to adopt generative models because strong generation can lead to undesirable hallucinations. For substantially degraded LR images, as seen in academia, strong generation is required and hallucinations are more tolerable because of the wide gap between LR and HR images. In contrast, in consumer photography, the LR image has substantially higher fidelity, requiring only minimal hallucination-free generation. We hypothesize that generation in SISR is controlled by the stringency and richness of the FM's conditioning feature. First, text features are high level features, which often cannot describe subtle textures in an image. Additionally, Smartphone LR images are at least $12MP$, whereas SISR networks built on T2IDiff FM are designed to perform inference on much smaller images ($<1MP$). As a result, SISR inference has to be performed on small patches, which often cannot be accurately described by text feature. To address these shortcomings, we introduce an SISR network built on a FM with lower-level feature conditioning, specifically DINOv2 features, which we call a Feature-to-Image Diffusion (F2IDiff) Foundation Model (FM). Lower level features provide stricter conditioning while being rich descriptors of even small patches.
Abstract (translated)
随着生成式AI的出现，单幅图像超分辨率（SISR）的质量有了显著提升。这是因为文本到图像扩散模型（T2I-Diffusion 模型）学习到了强大的先验知识，可以缩小高分辨率（HR）和低分辨率（LR）图像之间的差距。然而，旗舰智能手机摄像头在采用生成式模型方面进展缓慢，因为强烈的生成能力可能会导致不希望出现的幻觉现象。对于学术界看到的高度退化的LR图像来说，强烈生成是必需的，并且由于LR与HR图像之间的巨大差距，这些幻觉是可以容忍的。相比之下，在消费级摄影中，低分辨率图像具有较高的保真度，仅需要微小而无幻觉干扰的生成。 我们假设在SISR中的生成受到基础模型（Foundation Model, FM）条件特征严格性和丰富性的控制。首先，文本特征是高层次的特征，往往无法描述图像中的细微纹理。此外，智能手机拍摄的LR图像至少为1200万像素，而基于T2I-Diffusion FM构建的SISR网络被设计用于较小尺寸的图像（<1百万像素）进行推理。因此，SISR推理必须在小块上进行，并且这些小块通常无法用文本特征准确描述。 为了弥补这一不足，我们引入了一个基于低级特征条件的基础模型的SISR网络，具体来说是DINOv2特征，我们称其为特征到图像扩散（F2I-Diffusion 模型）。低级特征提供了严格的条件约束，同时对小块提供丰富的描述。
URL
https://arxiv.org/abs/2512.24473
