Diffusion-based image super-resolution (ISR) has shown strong potential, but it still struggles in real-world scenarios where degradations are unknown and spatially non-uniform, often resulting in lost details or visual artifacts. To address this challenge, we propose a novel super-resolution diffusion model, QUSR, which integrates a Quality-Aware Prior (QAP) with an Uncertainty-Guided Noise Generation (UNG) module. The UNG module adaptively adjusts the noise injection intensity, applying stronger perturbations to high-uncertainty regions (e.g., edges and textures) to reconstruct complex details, while minimizing noise in low-uncertainty regions (e.g., flat areas) to preserve original information. Concurrently, the QAP leverages an advanced Multimodal Large Language Model (MLLM) to generate reliable quality descriptions, providing an effective and interpretable quality prior for the restoration process. Experimental results confirm that QUSR can produce high-fidelity and high-realism images in real-world scenarios. The source code is available at this https URL.
基于扩散的图像超分辨率(ISR)展现了强大的潜力,但在现实场景中却面临挑战,尤其是在退化情况未知且空间非均匀的情况下,往往会导致细节丢失或视觉伪影。为了解决这一问题,我们提出了一种新的超分辨率扩散模型QUSR,它集成了质量感知先验(QAP)和不确定性引导噪声生成(UNG)模块。UNG模块能够自适应地调整噪声注入强度,在高不确定区域(如边缘和纹理)应用更强的扰动以重建复杂细节,而在低不确定区域(如平坦区域)则尽量减少噪声以保留原有信息。同时,QAP利用先进的多模态大型语言模型(MLLM)生成可靠的质量描述,为恢复过程提供有效且可解释的质量先验。实验结果证实了QUSR能够在现实场景中产生高保真度和高真实感的图像。源代码可在该网址获得(原链接未提供具体URL,请访问相关研究论文或官方发布渠道获取)。
https://arxiv.org/abs/2603.09125
Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at this https URL.
未注册的高光谱图像(HSI)超分辨率(SR)通常旨在利用未经配准的高分辨率参考图像增强低分辨率的HSI。本文中,我们提出了一种基于解混的融合框架,该框架将空间-光谱信息分离出来,以同时缓解未注册融合的影响并提高SR模型的学习能力。具体而言,首先使用奇异值分解进行初始光谱解混,在保留原始端元的同时使后续网络专注于提升初始丰度图。为了利用未经配准参考图像的空间纹理,我们引入了一个从粗到细的可变形聚合模块,该模块先通过一个粗金字塔预测器估计像素级流和相似性图,然后执行精细亚像素修正以实现参考特征的可变形聚合。接着,聚合特征会经过一系列空间-通道丰度交叉注意力块进行细化处理。此外,还提出了一种基于空间-通道调制的融合模块,该模块利用动态门控权重合并编码器-解码器特征,生成高质量、高分辨率的HSI。在模拟和真实数据集上的实验结果确认了我们所提方法达到了当前最佳的超分辨率性能。代码将在此URL提供。
https://arxiv.org/abs/2603.07918
In bandwidth-limited online video streaming, videos are usually downsampled and compressed. Although recent online video super-resolution (online VSR) approaches achieve promising results, they are still compute-intensive and fall short of real-time processing at higher resolutions, due to complex motion estimation for alignment and redundant processing of consecutive frames. To address these issues, we propose a compressed-domain-aware network (CDA-VSR) for online VSR, which utilizes compressed-domain information, including motion vectors, residual maps, and frame types to balance quality and efficiency. Specifically, we propose a motion-vector-guided deformable alignment module that uses motion vectors for coarse warping and learns only local residual offsets for fine-tuned adjustments, thereby maintaining accuracy while reducing computation. Then, we utilize a residual map gated fusion module to derive spatial weights from residual maps, suppressing mismatched regions and emphasizing reliable details. Further, we design a frame-type-aware reconstruction module for adaptive compute allocation across frame types, balancing accuracy and efficiency. On the REDS4 dataset, our CDA-VSR surpasses the state-of-the-art method TMP, with a maximum PSNR improvement of 0.13 dB while delivering more than double the inference speed. The code will be released at this https URL.
在带宽受限的在线视频流媒体中,通常会将视频进行降采样和压缩。尽管最近的一些在线视频超分辨率(online VSR)方法取得了令人满意的结果,但由于复杂的运动估计以实现对齐以及连续帧处理中的冗余计算,这些方法仍然需要大量的计算资源,并且在高分辨率下的实时处理方面仍存在不足。 为了解决这些问题,我们提出了一种压缩域感知网络(CDA-VSR),该网络利用包括运动向量、残差图和帧类型在内的压缩域信息,在保证质量的同时提高效率。具体来说,我们提出了一个基于运动矢量引导的可变形对齐模块,该模块使用运动向量进行粗略扭曲,并仅学习局部残差偏移以实现精细调整,从而在减少计算需求的情况下保持准确性。此外,我们设计了一个从残差图中推导空间权重的残差图门控融合模块,抑制不匹配区域并强调可靠细节。另外,为了在不同类型的帧之间进行自适应计算分配,平衡准确性和效率,我们还设计了一个基于帧类型感知重建模块。 在RED-S4数据集上,我们的CDA-VSR方法超越了最新的TMP方法,在最大PSNR改善达到0.13分贝的同时,推理速度提高了两倍以上。代码将在以下链接发布:[此处插入URL]。
https://arxiv.org/abs/2603.07694
Image Super-Resolution (SR) aims to reconstruct high-resolution images from degraded low-resolution inputs. While diffusion-based SR methods offer powerful generative capabilities, their performance heavily depends on how semantic priors are structured and integrated into the generation process. Existing approaches often rely on entangled or coarse-grained priors that mix global layout with local details, or conflate structural and textural cues, thereby limiting semantic controllability and interpretability. In this work, we propose DTPSR, a novel diffusion-based SR framework that introduces disentangled textual priors along two complementary dimensions: spatial hierarchy (global vs. local) and frequency semantics (low- vs. high-frequency). By explicitly separating these priors, DTPSR enables the model to simultaneously capture scene-level structure and object-specific details with frequency-aware semantic guidance. The corresponding embeddings are injected via specialized cross-attention modules, forming a progressive generation pipeline that reflects the semantic granularity of visual content, from global layout to fine-grained textures. To support this paradigm, we construct DisText-SR, a large-scale dataset containing approximately 95,000 image-text pairs with carefully disentangled global, low-frequency, and high-frequency descriptions. To further enhance controllability and consistency, we adopt a multi-branch classifier-free guidance strategy with frequency-aware negative prompts to suppress hallucinations and semantic drift. Extensive experiments on synthetic and real-world benchmarks show that DTPSR achieves high perceptual quality, competitive fidelity, and strong generalization across diverse degradation scenarios.
图像超分辨率(SR)的目标是从退化的低分辨率输入中重建高质量的高分辨率图像。虽然基于扩散的方法提供了强大的生成能力,但其性能很大程度上取决于语义先验如何被构建和整合到生成过程中。现有方法常常依赖于纠缠不清或粗粒度的先验信息,这些信息混合了全局布局与局部细节,或者混淆了结构和纹理线索,从而限制了语义控制能力和解释性。 在这项工作中,我们提出了DTPSR(Disentangled Textual Priors for Image Super-Resolution),这是一种新型基于扩散的方法框架。它在两个互补维度上引入了非纠缠的文本先验:空间层级(全局 vs. 局部)和频率语义(低频 vs. 高频)。通过明确分离这些先验,DTPSR使模型能够同时捕捉场景级别的结构和对象特定的细节,并且具有频率感知的语义引导。相应的嵌入信息通过专门设计的交叉注意力模块注入,形成了一个渐进生成管道,该管道从全局布局到精细纹理反映视觉内容的语义粒度。 为了支持这一范式,我们构建了DisText-SR,这是一个大规模的数据集,包含大约95,000个图像-文本对,并且这些描述被仔细地分离成了全局、低频和高频成分。为提高控制能力和一致性,我们在频率感知负面提示的引导下采用了无分类器多分支策略来抑制幻觉效应和语义漂移。 在合成数据集和真实世界基准上的广泛实验表明,DTPSR达到了高感知质量、竞争性的保真度,并且在各种退化场景中具备强大的泛化能力。
https://arxiv.org/abs/2603.07430
The effectiveness of super resolution (SR) models hinges on their ability to recover high frequency structure without introducing artifacts. Diffusion based approaches have recently advanced the state of the art in SR. However, most diffusion based SR pipelines operate purely in the spatial domain, which may yield high frequency details that are not well supported by the underlying low resolution evidence. On the other hand, unlike supervised SR models that may inject dataset specific textures, single image SR relies primarily on internal image statistics and can therefore be less prone to dataset-driven hallucinations; nevertheless, ambiguity in the LR observation can still lead to inconsistent high frequency details. To tackle this problem, we introduce BATDiff, an unsupervised Bivariate A trous Wavelet Diffusion model designed to provide structured cross scale guidance during the generative process. BATDiff employs an a Trous wavelet transform that constructs an undecimated multiscale representation in which high frequency components are progressively revealed while the full spatial resolution is preserved. As the core inference mechanism, BATDiff includes a bivariate cross scale module that models parent child dependencies between adjacent scales. It improves high frequency coherence and reduces mismatch artifacts in diffusion based SR. Experiments on standard benchmarks demonstrate that BATDiff produces sharper and more structurally consistent reconstructions than existing diffusion and non diffusion baselines, achieving improvements in fidelity and perceptual quality.
超分辨率(SR)模型的有效性在于其能否在不引入伪影的情况下恢复高频结构。基于扩散的方法最近在超分辨率领域取得了最先进的成果。然而,大多数基于扩散的超分辨率流水线仅在空间域内操作,这可能会产生低分辨率输入数据无法充分支持的高频细节。另一方面,与可能注入特定于数据集纹理的监督学习SR模型不同,单图像SR主要依赖于内部图像统计信息,因此不太容易出现由数据驱动导致的幻觉;不过,低分辨率观察中的模糊性仍然可能导致不一致的高频细节。 为了解决这个问题,我们引入了BATDiff,这是一种无监督的双变量A-trous小波扩散模型,旨在在生成过程中提供结构化的跨尺度指导。BATDiff采用了一个A-trous小波变换,构建了一个未下采样的多尺度表示,在该表示中逐步揭示高频成分的同时保持完整的空间分辨率不变。作为核心推断机制,BATDiff包含一个双变量跨尺度模块,用于建模相邻尺度之间的父代-子代依赖关系。这可以提高高频一致性和减少基于扩散的SR中的不匹配伪影。在标准基准测试上的实验表明,与现有的基于扩散和非扩散的方法相比,BATDiff能够生成更清晰、结构上更加一致的重建图像,在保真度和感知质量方面取得了改进。
https://arxiv.org/abs/2603.07234
Diffusion transformer (DiT) architectures show great potential for real-world image super-resolution (Real-ISR). However, their computationally expensive iterative sampling necessitates one-step distillation. Existing one-step distillation methods struggle with Real-ISR on DiT. They suffer from fundamental trajectory mismatch and generate severe grid-like periodic artifacts. To tackle these challenges, we propose StrSR, a novel one-step adversarial distillation framework featuring spectral and trajectory regularization. Specifically, we propose an asymmetric discriminative distillation architecture to bridge the trajectory gap. Additionally, we design a frequency distribution matching strategy to effectively suppress DiT-specific periodic artifacts caused by high-frequency spectral leakage. Extensive experiments demonstrate that StrSR achieves state-of-the-art performance in Real-ISR, across both quantitative metrics and visual perception. The code and models will be released at this https URL .
扩散变换器(DiT)架构在现实世界中的图像超分辨率(Real-ISR)方面表现出巨大的潜力。然而,其计算成本高昂的迭代采样需要采用一次性蒸馏方法来优化效率。现有的单一步骤蒸馏方法在处理基于DiT的Real-ISR任务时存在困难,它们遇到了根本性的轨迹不匹配问题,并且生成了严重的网格状周期性伪影。 为了应对这些挑战,我们提出了一种新的单一步骤对抗性蒸馏框架StrSR,该框架具有频谱和轨迹正则化特性。具体而言,我们设计了一个不对称的判别式蒸馏架构来弥合轨迹差距。此外,我们制定了一种频率分布匹配策略,以有效抑制由高频谱泄漏引发的DiT特有的周期性伪影。 广泛的实验表明,StrSR在Real-ISR任务中实现了当前最先进的性能,在定量度量和视觉感知方面均表现出色。代码和模型将在[这个链接](https://此链接应为实际URL)上发布。
https://arxiv.org/abs/2603.06275
Recent Super-Resolution~(SR) methods mainly adopt Transformers for their strong long-range modeling capability and exceptional representational capacity. However, most SR Transformers rely heavily on relative positional bias~(RPB), which prevents them from leveraging hardware-efficient attention kernels such as FlashAttention. This limitation imposes a prohibitive computational burden during both training and inference, severely restricting attempts to scale SR Transformers by enlarging the training patch size or the self-attention window. Consequently, unlike other domains that actively exploit the inherent scalability of Transformers, SR Transformers remain heavily focused on effectively utilizing limited receptive fields. In this paper, we propose Rank-factorized Implicit Neural Bias~(RIB), an alternative to RPB that enables FlashAttention in SR Transformers. Specifically, RIB approximates positional bias using low-rank implicit neural representations and concatenates them with pixel content tokens in a channel-wise manner, turning the element-wise bias addition in attention score computation into a dot-product operation. Further, we introduce a convolutional local attention and a cyclic window strategy to fully leverage the advantages of long-range interactions enabled by RIB and FlashAttention. We enlarge the window size up to \textbf{96$\times$96} while jointly scaling the training patch size and the dataset size, maximizing the benefits of Transformers in the SR task. As a result, our network achieves \textbf{35.63\,dB PSNR} on Urban100$\times$2, while reducing training and inference time by \textbf{2.1$\times$} and \textbf{2.9$\times$}, respectively, compared to the RPB-based SR Transformer~(PFT).
近期的超分辨率(SR)方法主要采用Transformer架构,因其强大的长程建模能力和卓越的表现力。然而,大多数SR Transformer严重依赖相对位置偏差(RPB),这阻碍了它们利用高效的硬件注意力内核(如FlashAttention)。这种限制在训练和推理过程中带来了巨大的计算负担,并且极大地限制了通过增加训练补丁尺寸或自注意力窗口来扩展SR Transformers的尝试。因此,与积极利用Transformer固有可扩展性的其他领域不同,SR Transformer仍然集中于有效地使用有限的感受野。 在这篇论文中,我们提出了一个名为秩因子化隐式神经偏差(RIB)的方法,这是一种替代RPB的方法,它使SR Transformer能够采用FlashAttention。具体来说,RIB通过低秩的隐式神经表示来近似位置偏差,并以通道方式将其与像素内容令牌拼接在一起,从而将注意力得分计算中的逐元素偏差相加转换为点积操作。 此外,我们还引入了卷积局部注意机制和循环窗口策略,以便充分利用由RIB和FlashAttention提供的长程交互的优势。我们将窗口大小扩大到**96×96**,同时扩展训练补丁尺寸和数据集的规模,最大化Transformer在SR任务中的优势。结果表明,我们的网络在Urban100x2数据集上实现了**35.63 dB PSNR** 的性能,相比基于RPB的SR Transformer(PFT)减少了**2.1倍**的训练时间以及**2.9倍**的推理时间。
https://arxiv.org/abs/2603.06738
Generative real-world image super-resolution (Real-ISR) can synthesize visually convincing details from severely degraded low-resolution (LR) inputs, yet its stochastic sampling makes a critical failure mode hard to avoid: outputs may look sharp but be unfaithful to the LR evidence (semantic and structural hallucination), while such LR-anchored faithfulness is difficult to assess without HR ground truth. Preference-based reinforcement learning (RL) is a natural fit because each LR input yields a rollout group of candidates to compare. However, effective alignment in Real-ISR is hindered by (i) the lack of a degradation-robust LR-referenced faithfulness signal, and (ii) a rollout-group optimization bottleneck where naive multi-reward scalarization followed by normalization compresses objective-wise contrasts, causing advantage collapse and weakening the reward-weighted updates in DiffusionNFT-style forward fine-tuning. Moreover, (iii) limited coverage of real degradations restricts rollout diversity and preference signal quality. We propose LucidNFT, a multi-reward RL framework for flow-matching Real-ISR. LucidNFT introduces LucidConsistency, a degradation-robust semantic evaluator that makes LR-anchored faithfulness measurable and optimizable; a decoupled advantage normalization strategy that preserves objective-wise contrasts within each LR-conditioned rollout group before fusion, preventing advantage collapse; and LucidLR, a large-scale collection of real-world degraded images to support robust RL fine-tuning. Experiments show that LucidNFT consistently improves strong flow-based Real-ISR baselines, achieving better perceptual-faithfulness trade-offs with stable optimization dynamics across diverse real-world scenarios.
生成型真实世界图像超分辨率(Real-ISR)能够从严重降级的低分辨率(LR)输入中合成视觉上令人信服的细节,但其随机采样使得一个关键失败模式难以避免:输出可能看起来很锐利但实际上却与LR证据不符(语义和结构错觉),而没有高分辨率(HR)真实情况作为参考,这种基于LR锚定的真实性很难评估。基于偏好的强化学习(RL)是一个自然的解决方案,因为每个低分辨率输入都会产生一组候选者来进行比较。然而,在Real-ISR中有效对齐受到以下问题的影响:(i) 缺乏一种抗降级且基于LR的真实度信号;以及(ii) 一个滚动组优化瓶颈,其中简单的多奖励标量化和随后的归一化压缩了每个目标之间的对比度,导致优势崩溃并减弱了DiffusionNFT式正向微调中的加权更新。此外,(iii) 真实降级范围有限也限制了滚动多样性及偏好信号质量。 为了解决这些问题,我们提出了LucidNFT,这是一个用于流匹配Real-ISR的多奖励RL框架。LucidNFT引入了一个新的方法——LucidConsistency,这是一种抗降级且基于语义的真实度评估器,使基于LR锚定的真实性变得可测量和优化;一种分离优势归一化策略,在融合之前保留每个基于低分辨率条件滚动组内的目标对比度,防止优势崩溃;以及LucidLR,一个大规模的包含真实世界降级图像的数据集,支持鲁棒RL微调。实验表明,LucidNFT能够持续改进强大的流型Real-ISR基线,并在各种现实场景中实现更好的感知真实性权衡和稳定的优化动态。 这个框架解决了现有方法面临的关键挑战,通过引入新的评估标准、优化策略以及大规模的真实数据集,从而提高了生成图像的质量与真实性。
https://arxiv.org/abs/2603.05947
Digital subtraction angiography (DSA) is a key imaging technique for the auxiliary diagnosis and treatment of cerebrovascular diseases. Recent advancements in gaussian splatting and dynamic neural representations have enabled robust 3D vessel reconstruction from sparse dynamic inputs. However, these methods are fundamentally constrained by the resolution of input projections, where performing naive upsampling to enhance rendering resolution inevitably results in severe blurring and aliasing artifacts. Such lack of super-resolution capability prevents the reconstructed 4D models from recovering fine-grained vascular details and intricate branching structures, which restricts their application in precision diagnosis and treatment. To solve this problem, this paper proposes DSA-SRGS, the first super-resolution gaussian splatting framework for dynamic sparse-view DSA reconstruction. Specifically, we introduce a Multi-Fidelity Texture Learning Module that integrates high-quality priors from a fine-tuned DSA-specific super-resolution model, into the 4D reconstruction optimization. To mitigate potential hallucination artifacts from pseudo-labels, this module employs a Confidence-Aware Strategy to adaptively weight supervision signals between the original low-resolution projections and the generated high-resolution pseudo-labels. Furthermore, we develop Radiative Sub-Pixel Densification, an adaptive strategy that leverages gradient accumulation from high-resolution sub-pixel sampling to refine the 4D radiative gaussian kernels. Extensive experiments on two clinical DSA datasets demonstrate that DSA-SRGS significantly outperforms state-of-the-art methods in both quantitative metrics and qualitative visual fidelity.
数字减影血管造影(DSA)是辅助诊断和治疗脑血管疾病的关键成像技术。近年来,高斯点光源技术和动态神经表示的发展使得从稀疏的动态输入中实现稳健的三维血管重建成为可能。然而,这些方法从根本上受到输入投影分辨率的限制:简单地放大以提高渲染分辨率不可避免地会导致严重模糊和混叠伪影。这种缺乏超分辨率能力阻碍了重建的4D模型恢复精细的血管细节和复杂的分支结构,从而限制其在精准诊断和治疗中的应用。 为了解决这个问题,本文提出了一种名为DSA-SRGS(Dynamic Sparse-view DSA Reconstruction with Super-Resolution Gaussian Splats)的首个动态稀疏视图DSA重建超分辨率高斯点光源框架。具体而言,我们引入了一个多保真度纹理学习模块,该模块将精细调优后的特定于DSA的超分辨率模型中的高质量先验知识整合到4D重建优化中。为了减轻伪标签可能带来的幻觉伪影,此模块采用了一种基于信心感知策略自适应地在原始低分辨率投影和生成的高分辨率伪标签之间平衡监督信号。 此外,我们还开发了辐射亚像素密度化技术,这是一种利用从高分辨率子像素采样积累的梯度来优化4D放射高斯核的自适应策略。通过对两个临床DSA数据集进行广泛的实验研究证明,DSA-SRGS在量化指标和定性视觉保真度方面均显著优于现有最先进方法。
https://arxiv.org/abs/2603.04770
Infrared image super-resolution (IISR) under real-world conditions is a practically significant yet rarely addressed task. Pioneering works are often trained and evaluated on simulated datasets or neglect the intrinsic differences between infrared and visible imaging. In practice, however, real infrared images are affected by coupled optical and sensing degradations that jointly deteriorate both structural sharpness and thermal fidelity. To address these challenges, we propose Real-IISR, a unified autoregressive framework for real-world IISR that progressively reconstructs fine-grained thermal structures and clear backgrounds in a scale-by-scale manner via thermal-structural guided visual autoregression. Specifically, a Thermal-Structural Guidance module encodes thermal priors to mitigate the mismatch between thermal radiation and structural edges. Since non-uniform degradations typically induce quantization bias, Real-IISR adopts a Condition-Adaptive Codebook that dynamically modulates discrete representations based on degradation-aware thermal priors. Also, a Thermal Order Consistency Loss enforces a monotonic relation between temperature and pixel intensity, ensuring relative brightness order rather than absolute values to maintain physical consistency under spatial misalignment and thermal drift. We build FLIR-IISR, a real-world IISR dataset with paired LR-HR infrared images acquired via automated focus variation and motion-induced blur. Extensive experiments demonstrate the promising performance of Real-IISR, providing a unified foundation for real-world IISR and benchmarking. The dataset and code are available at: this https URL.
在现实条件下进行红外图像超分辨率(IISR)是一项具有实际意义但较少被关注的任务。早期的工作通常是在模拟数据集上训练和评估,或者忽视了红外成像与可见光成像之间的内在差异。然而,在实践中,真实的红外图像受到光学和感测退化的共同影响,这些退化同时降低了结构清晰度和热辐射的准确性。为了应对这一挑战,我们提出了Real-IISR,这是一个统一的自回归框架,用于现实世界的IISR任务。该框架通过基于热-结构引导的视觉自回归逐步以逐级的方式重建精细的热结构和清晰的背景。 具体而言,一个热-结构引导模块编码了热先验信息来缓解热辐射与结构边缘之间的不匹配问题。由于非均匀退化通常会引起量化偏置,Real-IISR采用了一个条件自适应码本,该码本根据对退化的感知性热先验动态地调节离散表示。此外,一种温度顺序一致性损失被提出以强制执行温度和像素强度之间单调关系,确保相对亮度排序而不是绝对值,在空间错位和热漂移的情况下保持物理一致性。 我们构建了FLIR-IISR数据集,这是一个包含配对的低分辨率-高分辨率红外图像的真实世界IISR数据集。这些图像通过自动化焦距变化和运动引起的模糊获得。广泛的实验展示了Real-IISR的出色性能,为真实世界的IISR提供了统一的基础并设立了基准。该数据集和代码可在以下网址获取:此 https URL 链接。
https://arxiv.org/abs/2603.04745
Deep learning (DL) methods are currently being explored to restore images from sparse-view-, limited-data-, and undersampled-based acquisitions in medical applications. Although outputs from DL may appear visually appealing based on likability/subjective criteria (such as less noise, smooth features), they may also suffer from hallucinations. This issue is further exacerbated by a lack of easy-to-use techniques and robust metrics for the identification of hallucinations in DL outputs. In this work, we propose performing Fourier Ring Correlation (FRC) analysis over small patches and concomitantly (s)canning across DL outputs and their reference counterparts to detect hallucinations (termed as sFRC). We describe the rationale behind sFRC and provide its mathematical formulation. The parameters essential to sFRC may be set using predefined hallucinated features annotated by subject matter experts or using imaging theory-based hallucination maps. We use sFRC to detect hallucinations for three undersampled medical imaging problems: CT super-resolution, CT sparse view, and MRI subsampled restoration. In the testing phase, we demonstrate sFRC's effectiveness in detecting hallucinated features for the CT problem and sFRC's agreement with imaging theory-based outputs on hallucinated feature maps for the MR problem. Finally, we quantify the hallucination rates of DL methods on in-distribution versus out-of-distribution data and under increasing subsampling rates to characterize the robustness of DL methods. Beyond DL-based methods, sFRC's effectiveness in detecting hallucinations for a conventional regularization-based restoration method and a state-of-the-art unrolled method is also shown.
深度学习(DL)方法目前被探索用于从稀疏视图、有限数据和欠采样获取的图像中恢复医学应用中的影像。虽然基于美观性/主观标准(如减少噪声,平滑特征),深度学习输出可能看起来很有吸引力,但它们也可能存在幻觉问题。这一问题由于缺乏易于使用的技术和稳健指标来识别深度学习输出中的幻觉而进一步加剧。在本工作中,我们提出对小块进行傅里叶环相关性(FRC)分析,并同时扫描深度学习输出及其参考图像以检测幻觉(称为sFRC)。我们描述了sFRC的原理并提供了其数学公式化方法。用于设置sFRC的关键参数可以通过使用领域专家注释的幻觉特征或基于成像理论的幻觉图来设定。 我们利用sFRC对三种欠采样医学影像问题进行幻觉检测:CT超分辨率、CT稀疏视图以及MRI欠采样恢复。在测试阶段,我们展示了sFRC在CT问题中检测到幻觉特征的有效性,并且对于MR问题,显示了sFRC与基于成像理论的输出对幻觉特征图的一致性。 最后,我们通过量化深度学习方法在分布内与分布外数据以及随着欠采样率增加时的幻觉频率来表征这些方法的鲁棒性。除深度学习方法外,还展示了sFRC对于传统正则化恢复方法和最新展开式方法检测幻觉的有效性。
https://arxiv.org/abs/2603.04673
Generating high-quality 360° panoramic videos from perspective input is one of the crucial applications for virtual reality (VR), whereby high-resolution videos are especially important for immersive experience. Existing methods are constrained by computational limitations of vanilla diffusion models, only supporting $\leq$ 1K resolution native generation and relying on suboptimal post super-resolution to increase resolution. We introduce CubeComposer, a novel spatio-temporal autoregressive diffusion model that natively generates 4K-resolution 360° videos. By decomposing videos into cubemap representations with six faces, CubeComposer autoregressively synthesizes content in a well-planned spatio-temporal order, reducing memory demands while enabling high-resolution output. Specifically, to address challenges in multi-dimensional autoregression, we propose: (1) a spatio-temporal autoregressive strategy that orchestrates 360° video generation across cube faces and time windows for coherent synthesis; (2) a cube face context management mechanism, equipped with a sparse context attention design to improve efficiency; and (3) continuity-aware techniques, including cube-aware positional encoding, padding, and blending to eliminate boundary seams. Extensive experiments on benchmark datasets demonstrate that CubeComposer outperforms state-of-the-art methods in native resolution and visual quality, supporting practical VR application scenarios. Project page: this https URL
从视角输入生成高质量的360°全景视频是虚拟现实(VR)的关键应用之一,而高分辨率视频对于沉浸式体验尤为重要。现有的方法受限于传统扩散模型在计算上的限制,只能支持原生生成不超过1K分辨率的视频,并且依赖于次优的后处理超分辨率技术来提升分辨率。我们介绍了CubeComposer,这是一种新颖的空间-时间自回归扩散模型,能够直接生成4K分辨率的360°视频。通过将视频分解为具有六个面的立方图表示,CubeComposer按照精心规划的空间-时间顺序自回归地合成内容,在减少内存需求的同时支持高分辨率输出。 具体而言,为了应对多维自回归中的挑战,我们提出了以下解决方案: 1. 一种空间-时间自回归策略,通过协调立方体各个面和时间窗口的360°视频生成过程,实现连贯的合成; 2. 一个配备稀疏上下文注意力设计的立方图面上下文管理机制,以提高效率; 3. 连续性感知技术,包括位置编码、填充和平滑处理,以消除边界接缝。 在基准数据集上的广泛实验表明,CubeComposer在原生分辨率和视觉质量方面超越了现有最先进的方法,支持实际的VR应用场景。项目主页:[此链接](this%20https%20URL)
https://arxiv.org/abs/2603.04291
Audio Super-Resolution is a set of techniques aimed at high-quality estimation of the given signal as if it would be sampled with higher sample rate. Among suggested methods there are diffusion and flow models (which are considered slower), generative adversarial networks (which are considered faster), however both approaches are currently presented by high-parametric networks, requiring high computational costs both for training and inference. We propose a solution to both these problems by re-considering the recent advances in the training of diffusion models and applying them to super-resolution from any to 48 kHz sample rate. Our approach shows better results than NU-Wave 2 and is comparable to state-of-the-art models. Our model called FastWave has around 50 GFLOPs of computational complexity and 1.3 M parameters and can be trained with less resources and significantly faster than the majority of recently proposed diffusion- and flow-based solutions. The code has been made publicly available.
音频超分辨率是一系列技术,旨在通过更高采样率对给定信号进行高质量估计。其中建议的方法包括扩散模型和流模型(被认为较慢),以及生成对抗网络(被认为较快)。然而,这两种方法当前都需要高参数的网络,这导致了在训练和推理过程中较高的计算成本。我们提出了一种解决方案,重新审视最近在训练扩散模型方面的进展,并将其应用于从任何采样率到48 kHz的超分辨率任务中。我们的方法比NU-Wave 2表现更好,并且与现有最佳模型相当。我们的模型名为FastWave,在计算复杂度方面约为50 GFLOPs,参数量为1.3M,能够以较少资源并且显著快于大多数最近提出的基于扩散和流的方法进行训练。代码已经公开发布。
https://arxiv.org/abs/2603.04122
Diffusion-based approaches have recently driven remarkable progress in real-world image super-resolution (SR). However, existing methods still struggle to simultaneously preserve fine details and ensure high-fidelity reconstruction, often resulting in suboptimal visual quality. In this paper, we propose FiDeSR, a high-fidelity and detail-preserving one-step diffusion super-resolution framework. During training, we introduce a detail-aware weighting strategy that adaptively emphasizes regions where the model exhibits higher prediction errors. During inference, low- and high-frequency adaptive enhancers further refine the reconstruction without requiring model retraining, enabling flexible enhancement control. To further improve the reconstruction accuracy, FiDeSR incorporates a residual-in-residual noise refinement, which corrects prediction errors in the diffusion noise and enhances fine detail recovery. FiDeSR achieves superior real-world SR performance compared to existing diffusion-based methods, producing outputs with both high perceptual quality and faithful content restoration. The source code will be released at: this https URL.
基于扩散的方法最近在真实世界的图像超分辨率(SR)领域取得了显著的进步。然而,现有的方法仍然难以同时保留细节并确保高保真度重建,在视觉质量上常常不尽如人意。在这篇论文中,我们提出了FiDeSR,这是一种高度保真且能保持细节的一步式扩散超分辨率框架。 在训练阶段,我们引入了一种感知细节的加权策略,该策略能够自适应地强调模型预测误差较高的区域。在推理阶段,低频和高频自适应增强器进一步细化重建结果,而无需重新训练模型,从而实现灵活的增强控制。为了进一步提高重建精度,FiDeSR集成了残差内残差噪声精炼机制,这种机制可以修正扩散噪声中的预测错误,并提升细部恢复能力。 与现有的基于扩散的方法相比,FiDeSR在真实世界的超分辨率性能方面表现出色,能够生成既具有高感知质量又忠实还原内容的输出。源代码将在以下链接发布:this https URL.
https://arxiv.org/abs/2603.02692
Recently, Transformers have gained significant popularity in image restoration tasks such as image super-resolution and denoising, owing to their superior performance. However, balancing performance and computational burden remains a long-standing problem for transformer-based architectures. Due to the quadratic complexity of self-attention, existing methods often restrict attention to local windows, resulting in limited receptive field and suboptimal performance. To address this issue, we propose Adaptive Token Dictionary (ATD), a novel transformer-based architecture for image restoration that enables global dependency modeling with linear complexity relative to image size. The ATD model incorporates a learnable token dictionary, which summarizes external image priors (i.e., typical image structures) during the training process. To utilize this information, we introduce a token dictionary cross-attention (TDCA) mechanism that enhances the input features via interaction with the learned dictionary. Furthermore, we exploit the category information embedded in the TDCA attention maps to group input features into multiple categories, each representing a cluster of similar features across the image and serving as an attention group. We also integrate the learned category information into the feed-forward network to further improve feature fusion. ATD and its lightweight version ATD-light, achieve state-of-the-art performance on multiple image super-resolution benchmarks. Moreover, we develop ATD-U, a multi-scale variant of ATD, to address other image restoration tasks, including image denoising and JPEG compression artifacts removal. Extensive experiments demonstrate the superiority of out proposed models, both quantitatively and qualitatively.
最近,Transformer模型在图像恢复任务(如超分辨率和去噪)中因其优越的性能而获得了广泛应用。然而,如何平衡性能与计算负担仍是基于Transformer架构的一个长期难题。由于自注意力机制具有二次复杂度,现有方法通常限制注意范围为局部窗口,导致感受野受限且性能不理想。为了应对这一问题,我们提出了适应性令牌字典(Adaptive Token Dictionary, ATD),这是一种新颖的用于图像恢复的Transformer架构,能够在与图像大小呈线性关系的情况下实现全局依赖建模。 ATD模型中引入了一个可学习的令牌字典,在训练过程中它会总结外部图像先验信息(即典型的图像结构)。为了利用这一信息,我们提出了一种令牌字典交叉注意力(Token Dictionary Cross-Attention, TDCA)机制,通过与所学字典互动来增强输入特征。此外,我们还利用了嵌入在TDCA注意图中的类别信息,将输入特征划分为多个类别,每个类别代表图像中相似特征的集群,并作为注意力组使用。同时,我们将学习到的类别信息整合进前馈网络中,以进一步改进特征融合。 ATD及其轻量级版本ATD-light,在多个图像超分辨率基准测试上达到了最先进的性能水平。此外,我们还开发了ATD-U,这是ATD的一个多尺度变体,用于解决包括图像去噪和JPEG压缩痕迹去除在内的其他图像恢复任务。广泛的实验结果从定量和定性两个方面证明了所提出模型的优越性。
https://arxiv.org/abs/2603.02581
Global token mixing, implemented via self-attention or state-space sequence models, has become a popular model design choice for MRI restoration. However, MRI restoration tasks differ substantially in how their degradations vary over image and k-space domains, and in the degree to which global coupling is already imposed by physics-driven data consistency terms. In this work, we ask the question whether global token mixing is actually beneficial in each individual task across three representative settings: accelerated MRI reconstruction with explicit data consistency, MRI super-resolution with k-space center cropping, and denoising of clinical carotid MRI data with spatially heteroscedastic noise. To reduce confounding factors, we establish a controlled testbed comparing a minimal local gated CNN and its large-field variant, benchmarking them directly against state-of-the-art global models under aligned training and evaluation protocols. For accelerated MRI reconstruction, the minimal unrolled gated-CNN baseline is already highly competitive compared to recent token-mixing approaches in public reconstruction benchmarks, suggesting limited additional benefits when the forward model and data-consistency steps provide strong global constraints. For super-resolution, where low-frequency k-space data are largely preserved by the controlled low-pass degradation, local gated models remain competitive, and a lightweight large-field variant yields only modest improvements. In contrast, for denoising with pronounced spatially heteroscedastic noise, token-mixing models achieve the strongest overall performance, consistent with the need to estimate spatially varying reliability. In conclusion, our results demonstrate that the utility of global token mixing in MRI restoration is task-dependent, and it should be tailored to the underlying imaging physics and degradation structure.
全球令牌混合(通过自注意力或状态空间序列模型实现)已成为MRI重建任务中流行的模型设计选择。然而,不同MRI重建任务在图像和k-空间领域中的退化变化方式以及物理驱动数据一致性项所带来的全局耦合程度上存在显著差异。在这项工作中,我们探讨了全球令牌混合是否真的对每一单独的任务都有益处,具体是在三个具有代表性的设置中:加速MRI重建(采用显式数据一致性)、基于k-空间中心裁剪的MRI超分辨率以及临床颈动脉MRI数据去噪(含空间异质噪声)。为了减少混淆因素的影响,我们建立了一个受控测试环境,在该环境中比较了最小局部门控CNN及其大范围变体,并直接将其与最先进的全局模型在一致的训练和评估协议下进行基准对比。 对于加速MRI重建任务,最小展开的门控-CNN基线已与最近的令牌混合方法相比在公共重建基准中具有高度竞争力,这表明当正向模型及数据一致性步骤提供了强大的全球约束时,额外采用令牌混合所带来的好处有限。对于超分辨率任务,在受控制低通滤波退化下大多数低频k-空间数据被保留的情况下,局部门控模型仍然保持竞争性,并且轻量级的大范围变体仅带来轻微的改进。相比之下,在存在显著空间异质噪声的去噪任务中,令牌混合模型实现了最强的整体性能,这与估计空间变化可靠性的需求一致。 综上所述,我们的研究结果表明在MRI重建中的全球令牌混合效用取决于具体任务,并且应根据底层成像物理和退化结构进行量身定制。
https://arxiv.org/abs/2603.01449
Tensor Ring (TR) decomposition is a powerful tool for high-order data modeling, but is inherently restricted to discrete forms defined on fixed meshgrids. In this work, we propose a TR functional decomposition for both meshgrid and non-meshgrid data, where factors are parameterized by Implicit Neural Representations (INRs). However, optimizing this continuous framework to capture fine-scale details is intrinsically difficult. Through a frequency-domain analysis, we demonstrate that the spectral structure of TR factors determines the frequency composition of the reconstructed tensor and limits the high-frequency modeling capacity. To mitigate this, we propose a reparameterized TR functional decomposition, in which each TR factor is a structured combination of a learnable latent tensor and a fixed basis. This reparameterization is theoretically shown to improve the training dynamics of TR factor learning. We further derive a principled initialization scheme for the fixed basis and prove the Lipschitz continuity of our proposed model. Extensive experiments on image inpainting, denoising, super-resolution, and point cloud recovery demonstrate that our method achieves consistently superior performance over existing approaches. Code is available at this https URL.
张量环(TR)分解是高阶数据分析的强大工具,但本质上局限于在固定网格上的离散形式。在这项工作中,我们提出了一种适用于网格数据和非网格数据的TR函数分解方法,其中因子由隐式神经表示(INR)参数化。然而,优化这种连续框架以捕捉细粒度细节在本质上是困难的。通过频域分析,我们展示了TR因子的谱结构决定了重构张量的频率构成,并限制了高频建模的能力。为了解决这个问题,我们提出了一个重新参数化的TR函数分解方法,在这种方法中每个TR因子是由可学习的潜在张量和固定基底有结构地组合而成。理论上证明,这种重参数化可以改善TR因子的学习动态过程。此外,我们还推导出了一种用于固定基底的原则性初始化方案,并证明了所提出模型的Lipschitz连续性。在图像修复、去噪、超分辨率和点云恢复等广泛的实验中表明,我们的方法比现有方法达到了一致且优越的表现水平。代码可在提供的URL处获取。
https://arxiv.org/abs/2603.01034
The rapid progress in radar and communication places increasing demands on low-latency and energy-efficiency array signal processing methods. There is an emerging direction of constructing analog computing processors for directly processing electromagnetic (EM) waves. However, the existing methods are constrained by 2D physical aperture and imprecise design process with inefficient computing architecture, resulting in limited sensing resolution and number of separated sources. Here, we present a fully-analog array signal processor (FASP) using 3D aperture engineering framework to perform super-resolution direction-of-arrival estimation, source number estimation, and multi-channel source separation in parallel for both coherent and incoherent sources. 3D aperture engineering is realized by constructing deep cascaded metasurface layers so that the diffractive propagation from oblique incident fields can be layer-wise modulated and piecewise encoded for perceiving EM fields far exceeding physical aperture limits. The multi-dimensional synthetic aperture (MSA) training is developed to characterize the metasurface modulation and optimize the neuro-augmented physical model for extending system aperture and generating high-order nonlinear angular response. FASP orthogonalizes the array response vectors of communication channels to map them into antenna detectors in the analog domain. The $N$-layer FASP has the capability to achieve ~N times higher angular resolution than the Rayleigh diffraction limit. Experiments further validate the source number estimation and independent channel separation of 10-target that can suppress radar jamming signals by ~20 dB and enhance channel communication capacity by 13.5 times at 36~41 GHz. FASP heralds a paradigm shift in signal processing for super-resolution optics, advanced radar, and 6G communications.
雷达和通信技术的快速进步对低延迟和能源效率高的阵列信号处理方法提出了越来越高的要求。一种新兴的方向是构建用于直接处理电磁(EM)波的模拟计算处理器。然而,现有的方法受到二维物理孔径限制以及设计过程不精确且计算架构效率低下的制约,导致感知分辨率有限及分离源的数量受限。在这里,我们提出了一种使用三维孔径工程技术进行全模拟阵列信号处理器(FASP),能够同时并行执行超分辨到达方向估计、源数量估计和多通道源分离,适用于相干和非相干源。 通过构建深层级联的元表面层来实现三维孔径工程,这样可以逐层调节来自斜入射场的衍射传播,并分段编码感知超过物理孔径限制的电磁场。开发了多维合成孔径(MSA)训练技术以表征元表面调制并优化神经增强的物理模型,以便扩展系统孔径并生成高阶非线性角度响应。 FASP通过正交化通信通道的阵列响应向量,并将其映射到模拟域中的天线检测器来工作。具有$N$层的FASP能够实现约$N$倍于瑞利衍射极限的角度分辨率。 实验进一步验证了10个目标源的数量估计和独立通道分离,可以抑制雷达干扰信号大约20分贝,并在36~41 GHz频段上将信道通信容量提高13.5倍。FASP预示着超分辨光学、先进雷达及第六代(6G)通信领域中的信号处理范式转变的到来。
https://arxiv.org/abs/2603.00995
We propose a simple yet effective UHDPromer, a neural discrimination-prompted Transformer, for Ultra-High-Definition (UHD) image restoration and enhancement. Our UHDPromer is inspired by an interesting observation that there implicitly exist neural differences between high-resolution and low-resolution features, and exploring such differences can facilitate low-resolution feature representation. To this end, we first introduce Neural Discrimination Priors (NDP) to measure the differences and then integrate NDP into the proposed Neural Discrimination-Prompted Attention (NDPA) and Neural Discrimination-Prompted Network (NDPN). The proposed NDPA re-formulates the attention by incorporating NDP to globally perceive useful discrimination information, while the NDPN explores a continuous gating mechanism guided by NDP to selectively permit the passage of beneficial content. To enhance the quality of restored images, we propose a super-resolution-guided reconstruction approach, which is guided by super-resolving low-resolution features to facilitate final UHD image restoration. Experiments show that UHDPromer achieves the best computational efficiency while still maintaining state-of-the-art performance on $3$ UHD image restoration and enhancement tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes and pre-trained models will be made available at this https URL.
我们提出了一种简单而有效的UHDPromer,这是一种神经区分提示Transformer,用于超高清(UHD)图像的恢复和增强。我们的UHDPromer受到一个有趣观察的启发,即高分辨率特征与低分辨率特征之间隐含地存在神经差异,并且探索这些差异可以促进低分辨率特征的表现形式。为此,我们首先引入了神经区分先验(NDP),用于衡量这些差异,然后将NDP整合到提出的神经区分提示注意机制(NDPA)和神经区分提示网络(NDPN)中。所提出的NDPA通过融合NDP重新定义注意力机制,以全局感知有用的区分信息,而NDPN则探索一种由NDP指导的连续门控机制,以便选择性地允许有益内容通过。为了提升恢复图像的质量,我们提出了一种超分辨率引导重建方法,该方法受低分辨率特征超解析过程的引导,从而促进最终的UHD图像恢复。 实验结果表明,UHDPromer在计算效率上表现出最佳水平,并且仍能在三个UHD图像恢复和增强任务(包括低光图像增强、去雾化和去除模糊)中保持最先进的性能。源代码及预训练模型可在上述网址获取。
https://arxiv.org/abs/2603.00853
Low-field magnetic resonance imaging (MRI) provides affordable access to diagnostic imaging but suffers from prolonged acquisition and limited image quality. Accelerated imaging can be achieved with k-space undersampling, while super-resolution (SR) and image quality transfer (IQT) methods typically rely on spatial-domain post-processing. In this work, we propose a novel framework for reconstructing high-field MR like images directly from undersampled low-field k-space. Our approach employs a k-space dual channel U-Net that processes the real and imaginary components of undersampled k-space to restore missing frequency content. Experiments on low-field brain MRI demonstrate that our k-space-driven image enhancement consistently outperforms the counterpart spatial-domain method. Furthermore, reconstructions from undersampled k-space achieve image quality comparable to full k-space acquisitions. To the best of our knowledge, this is the first work that investigates low-field MRI SR/IQT directly from undersampled k-space.
低场磁共振成像(MRI)提供了经济实惠的诊断影像访问途径,但存在采集时间长和图像质量有限的问题。通过k空间欠采样可以实现加速成像,而超分辨率(SR)和图像质量转换(IQT)方法通常依赖于空间域后的处理步骤。在本研究中,我们提出了一种新的框架,直接从欠采样的低场k空间重构高场MR类似的图像。我们的方法采用了一个k空间双通道U-Net架构,该架构能够处理欠采样k空间的实部和虚部组件,以恢复缺失的频率内容。 实验结果表明,在低场脑MRI数据上的应用中,我们提出的基于k空间的图像增强方法在性能上显著优于传统的空间域方法。此外,从欠采样的k空间重构出的图像质量可以与完全采样的k空间采集得到的质量相媲美。据我们所知,这是第一个直接从欠采样k空间研究低场MRI超分辨率和图像质量转换的工作。
https://arxiv.org/abs/2603.00668