Super_Resolution
-
FinPercep-RM: A Fine-grained Reward Model and Co-evolutionary Curriculum for RL-based Real-world Super-Resolution
2025-12-27 16:55:21Yidi Liu, Zihao Fan, Jie Huang, Jie Xiao, Dong Li, Wenlong Zhang, Lei Bai, Xueyang Fu, Zheng-Jun ZhaAbstract
Reinforcement Learning with Human Feedback (RLHF) has proven effective in image generation field guided by reward models to align human preferences. Motivated by this, adapting RLHF for Image Super-Resolution (ISR) tasks has shown promise in optimizing perceptual quality with Image Quality Assessment (IQA) model as reward models. However, the traditional IQA model usually output a single global score, which are exceptionally insensitive to local and fine-grained distortions. This insensitivity allows ISR models to produce perceptually undesirable artifacts that yield spurious high scores, misaligning optimization objectives with perceptual quality and results in reward hacking. To address this, we propose a Fine-grained Perceptual Reward Model (FinPercep-RM) based on an Encoder-Decoder architecture. While providing a global quality score, it also generates a Perceptual Degradation Map that spatially localizes and quantifies local defects. We specifically introduce the FGR-30k dataset to train this model, consisting of diverse and subtle distortions from real-world super-resolution models. Despite the success of the FinPercep-RM model, its complexity introduces significant challenges in generator policy learning, leading to training instability. To address this, we propose a Co-evolutionary Curriculum Learning (CCL) mechanism, where both the reward model and the ISR model undergo synchronized curricula. The reward model progressively increases in complexity, while the ISR model starts with a simpler global reward for rapid convergence, gradually transitioning to the more complex model outputs. This easy-to-hard strategy enables stable training while suppressing reward hacking. Experiments validates the effectiveness of our method across ISR models in both global quality and local realism on RLHF methods.
Abstract (translated)
基于人类反馈的强化学习（RLHF）在通过奖励模型引导图像生成领域时，已被证明能够有效地对齐人类偏好。受此启发，在图像超分辨率（ISR）任务中采用RLHF显示出潜力，可以利用图像质量评估（IQA）模型作为奖励模型来优化感知质量。然而，传统的IQA模型通常输出单一的全局评分，对于局部和细微失真非常不敏感。这种不敏感性允许ISR模型产生一些在视觉上令人不满但能获得高分数的伪影，从而导致优化目标与感知质量脱节，并且会导致奖励欺骗（reward hacking）。 为了解决这个问题，我们提出了一种基于编码器-解码器架构的细粒度感知奖励模型（Fine-grained Perceptual Reward Model, FinPercep-RM）。除了提供一个全局评分之外，该模型还会生成一张感知退化图，用于空间定位并量化局部缺陷。为了训练这一模型，我们特别引入了FGR-30k数据集，其中包含来自实际超分辨率模型的各种细微失真。 尽管FinPercep-RM模型取得了一定的成功，但其复杂性为生成器策略学习带来了重大挑战，并导致了训练的不稳定性。为此，我们提出了一种协同进化课程学习（Co-evolutionary Curriculum Learning, CCL）机制，在这种机制中，奖励模型和ISR模型都经历了同步的课程。在这一过程中，奖励模型逐步增加其复杂性，而ISR模型则从简单的全局奖励开始快速收敛，然后逐渐过渡到更复杂的模型输出。这一由简至难的战略不仅能够使训练过程更加稳定，还能抑制奖励欺骗。 实验验证了我们方法的有效性，在各种ISR模型中，无论是全局质量还是局部真实性方面，RLHF方法都取得了显著的改进。
URL
https://arxiv.org/abs/2512.22647
PDF
-
Efficient Vision Mamba for MRI Super-Resolution via Hybrid Selective Scanning
2025-12-25 22:41:51Mojtaba Safari, Shansong Wang, Vanessa L Wildman, Mingzhe Hu, Zach Eidex, Chih-Wei Chang, Erik H Middlebrooks, Richard L. J Qiu, Pretesh Patel, Ashesh B. Jani, Hui Mao, Zhen Tian, Xiaofeng YangAbstract
Background: High-resolution MRI is critical for diagnosis, but long acquisition times limit clinical use. Super-resolution (SR) can enhance resolution post-scan, yet existing deep learning methods face fidelity-efficiency trade-offs. Purpose: To develop a computationally efficient and accurate deep learning framework for MRI SR that preserves anatomical detail for clinical integration. Materials and Methods: We propose a novel SR framework combining multi-head selective state-space models (MHSSM) with a lightweight channel MLP. The model uses 2D patch extraction with hybrid scanning to capture long-range dependencies. Each MambaFormer block integrates MHSSM, depthwise convolutions, and gated channel mixing. Evaluation used 7T brain T1 MP2RAGE maps (n=142) and 1.5T prostate T2w MRI (n=334). Comparisons included Bicubic interpolation, GANs (CycleGAN, Pix2pix, SPSR), transformers (SwinIR), Mamba (MambaIR), and diffusion models (I2SB, Res-SRDiff). Results: Our model achieved superior performance with exceptional efficiency. For 7T brain data: SSIM=0.951+-0.021, PSNR=26.90+-1.41 dB, LPIPS=0.076+-0.022, GMSD=0.083+-0.017, significantly outperforming all baselines (p<0.001). For prostate data: SSIM=0.770+-0.049, PSNR=27.15+-2.19 dB, LPIPS=0.190+-0.095, GMSD=0.087+-0.013. The framework used only 0.9M parameters and 57 GFLOPs, reducing parameters by 99.8% and computation by 97.5% versus Res-SRDiff, while outperforming SwinIR and MambaIR in accuracy and efficiency. Conclusion: The proposed framework provides an efficient, accurate MRI SR solution, delivering enhanced anatomical detail across datasets. Its low computational demand and state-of-the-art performance show strong potential for clinical translation.
Abstract (translated)
背景：高分辨率MRI在诊断中至关重要，但长时间的获取时间限制了其临床应用。超分辨率(SR)可以提高扫描后的图像质量，然而现有的深度学习方法面临着精度与效率之间的权衡。 目的：开发一种计算高效且准确的深度学习框架用于MRI超分辨率处理，以保持解剖细节并便于临床上的应用整合。 材料和方法：我们提出了一种结合多头选择性状态空间模型(MHSSM)与轻量级通道MLP（多层感知器）的新颖SR框架。该模型采用2D补丁提取与混合扫描来捕捉长距离依赖关系。每个MambaFormer块集成了MHSSM、深度卷积和门控通道混合。评估使用了7T脑部的T1 MP2RAGE映射（n=142）和1.5T前列腺T2w MRI图像（n=334）。对比方法包括双三次插值，生成对抗网络(GANs)如CycleGAN、Pix2pix、SPSR，变换器模型如SwinIR，Mamba模型(MambaIR)，以及扩散模型如I2SB和Res-SRDiff。 结果：我们的模型在效率上表现出色且性能优越。对于7T脑部数据集：SSIM（结构相似性指数）为0.951±0.021、PSNR（峰值信噪比）为26.90±1.41 dB、LPIPS（感知线性判别编码器距离）为0.076±0.022、GMSD（梯度平均标准差）为0.083±0.017，显著优于所有基准方法（p<0.001）。对于前列腺数据集：SSIM为0.770±0.049、PSNR为27.15±2.19 dB、LPIPS为0.190±0.095、GMSD为0.087±0.013。我们的框架仅使用了0.9M参数和57 GFLOPs，相较于Res-SRDiff模型减少了99.8%的参数以及97.5%的计算量，并在准确性和效率上超越了SwinIR和MambaIR。 结论：所提出的框架提供了一种高效且精确的MRI超分辨率解决方案，在不同数据集中增强了解剖细节。其较低的计算需求及顶尖性能显示出了巨大的临床转化潜力。
URL
https://arxiv.org/abs/2512.19676
PDF
-
Diffusion Posterior Sampling for Super-Resolution under Gaussian Measurement Noise
2025-12-25 22:22:53Abu Hanif Muhammad SyarubanyAbstract
This report studies diffusion posterior sampling (DPS) for single-image super-resolution (SISR) under a known degradation model. We implement a likelihood-guided sampling procedure that combines an unconditional diffusion prior with gradient-based conditioning to enforce measurement consistency for $4\times$ super-resolution with additive Gaussian noise. We evaluate posterior sampling (PS) conditioning across guidance scales and noise levels, using PSNR and SSIM as fidelity metrics and a combined selection score $(\mathrm{PSNR}/40)+\mathrm{SSIM}$. Our ablation shows that moderate guidance improves reconstruction quality, with the best configuration achieved at PS scale $0.95$ and noise standard deviation $\sigma=0.01$ (score $1.45231$). Qualitative results confirm that the selected PS setting restores sharper edges and more coherent facial details compared to the downsampled inputs, while alternative conditioning strategies (e.g., MCG and PS-annealed) exhibit different texture fidelity trade-offs. These findings highlight the importance of balancing diffusion priors and measurement-gradient strength to obtain stable, high-quality reconstructions without retraining the diffusion model for each operator.
Abstract (translated)
这份报告研究了在已知退化模型下的单幅图像超分辨率（SISR）中的扩散后验采样（DPS）。我们实现了一种似然引导的采样过程，该过程结合了一个无条件扩散先验与基于梯度的条件处理来强制执行$4\times$超分辨率中加性高斯噪声的一致性测量。我们在指导比例和噪声水平上评估后验采样的（PS）条件，并使用峰值信噪比（PSNR）和结构相似性指数（SSIM）作为保真度指标，以及一个组合选择分数$(\mathrm{PSNR}/40)+\mathrm{SSIM}$来衡量性能。我们的消融研究表明，适度的指导可以提高重建质量，在PS比例为$0.95$且噪声标准差$\sigma=0.01$时达到最佳配置（得分为$1.45231$）。定性结果表明，所选的PS设置恢复了更清晰的边缘和更为一致的人脸细节，这与降采样的输入相比得到了改进。相比之下，其他条件处理策略（例如MCG和PS退火）表现出不同的纹理保真度权衡。这些发现强调了在不针对每个操作器重新训练扩散模型的情况下，为了获得稳定且高质量的重建结果，平衡扩散先验和测量梯度强度的重要性。
URL
https://arxiv.org/abs/2512.21797
PDF
-
A Large-Depth-Range Layer-Based Hologram Dataset for Machine Learning-Based 3D Computer-Generated Holography
2025-12-24 08:07:39Jaehong Lee, You Chan No, YoungWoo Kim, Duksu KimAbstract
Machine learning-based computer-generated holography (ML-CGH) has advanced rapidly in recent years, yet progress is constrained by the limited availability of high-quality, large-scale hologram datasets. To address this, we present KOREATECH-CGH, a publicly available dataset comprising 6,000 pairs of RGB-D images and complex holograms across resolutions ranging from 256*256 to 2048*2048, with depth ranges extending to the theoretical limits of the angular spectrum method for wide 3D scene coverage. To improve hologram quality at large depth ranges, we introduce amplitude projection, a post-processing technique that replaces amplitude components of hologram wavefields at each depth layer while preserving phase. This approach enhances reconstruction fidelity, achieving 27.01 dB PSNR and 0.87 SSIM, surpassing a recent optimized silhouette-masking layer-based method by 2.03 dB and 0.04 SSIM, respectively. We further validate the utility of KOREATECH-CGH through experiments on hologram generation and super-resolution using state-of-the-art ML models, confirming its applicability for training and evaluating next-generation ML-CGH systems.
Abstract (translated)
基于机器学习的计算机全息技术（ML-CGH）近年来取得了迅速的发展，然而其进步受到高质量、大规模全息图数据集稀缺性的限制。为了解决这个问题，我们提出了KOREATECH-CGH，这是一个公开可用的数据集，包含6,000对不同分辨率的RGB-D图像和复数全息图（从256*256到2048*2048），深度范围扩展到了角谱法理论极限，以实现广泛的三维场景覆盖。为了在大深度范围内提高全息图的质量，我们引入了一种后处理技术——振幅投影，在每个深度层中用相位保持不变的方式替换全息波场中的振幅分量。这种方法增强了重建的保真度，PSNR达到了27.01 dB，SSIM为0.87，分别超过了最近优化的轮廓掩膜层方法2.03 dB和0.04 SSIM。通过使用最先进的ML模型进行全息图生成和超分辨率实验，我们进一步验证了KOREATECH-CGH数据集的应用价值，证明其适用于下一代ML-CGH系统的训练和评估。
URL
https://arxiv.org/abs/2512.21040
PDF
-
Next-Scale Prediction: A Self-Supervised Approach for Real-World Image Denoising
2025-12-24 08:06:17Yiwen Shan, Haiyu Zhao, Peng Hu, Xi Peng, Yuanbiao GouAbstract
Self-supervised real-world image denoising remains a fundamental challenge, arising from the antagonistic trade-off between decorrelating spatially structured noise and preserving high-frequency details. Existing blind-spot network (BSN) methods rely on pixel-shuffle downsampling (PD) to decorrelate noise, but aggressive downsampling fragments fine structures, while milder downsampling fails to remove correlated noise. To address this, we introduce Next-Scale Prediction (NSP), a novel self-supervised paradigm that decouples noise decorrelation from detail preservation. NSP constructs cross-scale training pairs, where BSN takes low-resolution, fully decorrelated sub-images as input to predict high-resolution targets that retain fine details. As a by-product, NSP naturally supports super-resolution of noisy images without retraining or modification. Extensive experiments demonstrate that NSP achieves state-of-the-art self-supervised denoising performance on real-world benchmarks, significantly alleviating the long-standing conflict between noise decorrelation and detail preservation.
Abstract (translated)
自监督的真实世界图像去噪仍然是一个基本的挑战，这源于解耦空间结构化噪声与保持高频细节之间的对抗性权衡。现有的盲点网络（BSN）方法依赖于像素混洗下采样（PD）来解相关噪声，但激进的下采样会破坏精细结构，而温和的下采样则无法去除相关的噪声。为了应对这一问题，我们引入了跨尺度预测（NSP），这是一种新颖的自监督范式，将噪声解相关与细节保持分离开来。在NSP中，构建了跨尺度训练对，BSN以低分辨率、完全去噪的子图像为输入，预测保留精细细节的高分辨率目标。作为副产品，NSP自然支持无需重新训练或修改即可提高含噪图像的超分辨率。广泛的实验表明，NSP在真实世界的基准测试中实现了最先进的自监督去噪性能，并显著缓解了长期以来噪声解相关与细节保持之间的冲突。
URL
https://arxiv.org/abs/2512.21038
PDF
-
Multiscale Corrections by Continuous Super-Resolution
2025-12-23 14:44:20Zhi-Song Liu, Roland Maier, Andreas RuppAbstract
Finite element methods typically require a high resolution to satisfactorily approximate micro and even macro patterns of an underlying physical model. This issue can be circumvented by appropriate multiscale strategies that are able to obtain reasonable approximations on under-resolved scales. In this paper, we study the implicit neural representation and propose a continuous super-resolution network as a correction strategy for multiscale effects. It can take coarse finite element data to learn both in-distribution and out-of-distribution high-resolution finite element predictions. Our highlight is the design of a local implicit transformer, which is able to learn multiscale features. We also propose Gabor wavelet-based coordinate encodings, which can overcome the bias of neural networks learning low-frequency features. Finally, perception is often preferred over distortion, so scientists can recognize the visual pattern for further investigation. However, implicit neural representation is known for its lack of local pattern supervision. We propose to use stochastic cosine similarities to compare the local feature differences between prediction and ground truth. It shows better performance on structural alignments. Our experiments show that our proposed strategy achieves superior performance as an in-distribution and out-of-distribution super-resolution strategy.
Abstract (translated)
有限元方法通常需要高分辨率才能满意地近似底层物理模型的微尺度和宏尺度模式。这一问题可以通过适当的多尺度策略来解决，这些策略能够获得在低分辨率下合理近似的解。本文研究了隐式神经表示，并提出了一种连续超分辨率网络作为多重尺度效应的校正策略。该方法可以利用粗糙的有限元数据学习分布内及分布外的高分辨率有限元预测。 我们的亮点在于设计了一个局部隐式变压器，它可以学习多尺度特征。此外，我们还提出了基于Gabor小波的坐标编码，这可以克服神经网络在学习低频特征时产生的偏差。然而，感知通常比失真更受科学家们的青睐，因此他们可以通过视觉模式进一步进行研究。 尽管隐式神经表示缺乏局部模式监督，我们提出使用随机余弦相似度来比较预测和真实值之间的局部特征差异。这种方法在结构对齐上表现更好。 我们的实验表明，在分布内及分布外的超分辨率策略中，所提出的策略表现出优越性。
URL
https://arxiv.org/abs/2411.07576
PDF
-
Field-Space Attention for Structure-Preserving Earth System Transformers
2025-12-23 13:31:21Maximilian Witte, Johannes Meuer, \'Etienne Pl\'esiat, Christopher KadowAbstract
Accurate and physically consistent modeling of Earth system dynamics requires machine-learning architectures that operate directly on continuous geophysical fields and preserve their underlying geometric structure. Here we introduce Field-Space attention, a mechanism for Earth system Transformers that computes attention in the physical domain rather than in a learned latent space. By maintaining all intermediate representations as continuous fields on the sphere, the architecture enables interpretable internal states and facilitates the enforcement of scientific constraints. The model employs a fixed, non-learned multiscale decomposition and learns structure-preserving deformations of the input field, allowing coherent integration of coarse and fine-scale information while avoiding the optimization instabilities characteristic of standard single-scale Vision Transformers. Applied to global temperature super-resolution on a HEALPix grid, Field-Space Transformers converge more rapidly and stably than conventional Vision Transformers and U-Net baselines, while requiring substantially fewer parameters. The explicit preservation of field structure throughout the network allows physical and statistical priors to be embedded directly into the architecture, yielding improved fidelity and reliability in data-driven Earth system modeling. These results position Field-Space Attention as a compact, interpretable, and physically grounded building block for next-generation Earth system prediction and generative modeling frameworks.
Abstract (translated)
地球系统动态的准确和物理一致性的建模需要能够在连续地物理场直接操作并保持其内在几何结构的机器学习架构。在这里，我们介绍了Field-Space注意力机制，这是为地球系统的Transformer设计的一种方法，在该机制中计算是在物理域而非在学习到的潜在空间进行的。通过在整个过程中将所有中间表示维护为球面上的连续字段，这种架构能够支持可解释的内部状态，并促进了科学约束条件的应用。该模型采用了一个固定的、非学习的多尺度分解，并学会了输入场保持结构的变形，从而可以连贯地集成粗粒度和细粒度信息，同时避免了标准单尺度Vision Transformer中的优化不稳定性问题。 当我们应用这种Field-Space Transformer方法对HEALPix网格上的全球温度超分辨率进行处理时，与传统的Vision Transformers以及U-Net基准模型相比，它能够更快、更稳定地收敛，并且所需的参数要少得多。在整个网络中显式保持字段结构允许物理和统计先验被直接嵌入到架构中，从而提高了数据驱动的地球系统建模的准确性和可靠性。 这些结果将Field-Space Attention定位为下一代地球系统预测和生成模型框架中的一个紧凑、可解释且具有物理学基础的重要构建模块。
URL
https://arxiv.org/abs/2512.20350
PDF
-
Super-Resolution Enhancement of Medical Images Based on Diffusion Model: An Optimization Scheme for Low-Resolution Gastric Images
2025-12-22 03:37:47Haozhe JiaAbstract
Capsule endoscopy has enabled minimally invasive gastrointestinal imaging, but its clinical utility is limited by the inherently low resolution of captured images due to hardware, power, and transmission constraints. This limitation hampers the identification of fine-grained mucosal textures and subtle pathological features essential for early diagnosis. This work investigates a diffusion-based super-resolution framework to enhance capsule endoscopy images in a data-driven and anatomically consistent manner. We adopt the SR3 (Super-Resolution via Repeated Refinement) framework built upon Denoising Diffusion Probabilistic Models (DDPMs) to learn a probabilistic mapping from low-resolution to high-resolution images. Unlike GAN-based approaches that often suffer from training instability and hallucination artifacts, diffusion models provide stable likelihood-based training and improved structural fidelity. The HyperKvasir dataset, a large-scale publicly available gastrointestinal endoscopy dataset, is used for training and evaluation. Quantitative results demonstrate that the proposed method significantly outperforms bicubic interpolation and GAN-based super-resolution methods such as ESRGAN, achieving PSNR of 27.5 dB and SSIM of 0.65 for a baseline model, and improving to 29.3 dB and 0.71 with architectural enhancements including attention mechanisms. Qualitative results show improved preservation of anatomical boundaries, vascular patterns, and lesion structures. These findings indicate that diffusion-based super-resolution is a promising approach for enhancing non-invasive medical imaging, particularly in capsule endoscopy where image resolution is fundamentally constrained.
Abstract (translated)
胶囊内镜技术实现了微创胃肠成像，但其临床应用受限于图像固有的低分辨率，这主要是由于硬件、电源和传输的限制。这种局限性阻碍了对细微粘膜纹理及早期诊断所需的小病变特征的识别。本研究探讨了一种基于扩散的超分辨率框架，以数据驱动且解剖学一致的方式增强胶囊内镜图像。我们采用了SR3（通过重复细化实现的超分辨率）框架，并在其基础上构建了去噪扩散概率模型（DDPMs），以此来学习从低分辨率到高分辨率图像的概率映射。与常因训练不稳定和幻觉伪影问题而受困的GAN（生成对抗网络）方法不同，扩散模型提供了基于似然性的稳定训练以及结构保真度的提升。 为了训练和评估，我们使用了HyperKvasir数据集——一个大规模公开可用的胃肠内镜图像数据集。定量结果显示所提出的方法在基准模型上达到了27.5 dB的PSNR（峰值信噪比）和0.65的SSIM（结构相似性指数），显著优于双三次插值及GAN超分辨率方法如ESRGAN；通过加入注意力机制等架构改进，性能进一步提升至29.3 dB PSNR和0.71 SSIM。定性结果表明该模型能更好地保留解剖边界、血管模式以及病变结构。 这些发现表明基于扩散的超分辨率技术是增强非侵入性医学成像的一个有前景的方法，特别是在图像分辨率固有限制的胶囊内镜检查中尤为显著。
URL
https://arxiv.org/abs/2512.22209
PDF
-
Complex Swin Transformer for Accelerating Enhanced SMWI Reconstruction
2025-12-21 08:58:04Muhammad Usman, Sung-Min GhoAbstract
Susceptibility Map Weighted Imaging (SMWI) is an advanced magnetic resonance imaging technique used to detect nigral hyperintensity in Parkinsons disease. However, full resolution SMWI acquisition is limited by long scan times. Efficient reconstruction methods are therefore required to generate high quality SMWI from reduced k space data while preserving diagnostic relevance. In this work, we propose a complex valued Swin Transformer based network for super resolution reconstruction of multi echo MRI data. The proposed method reconstructs high quality SMWI images from low resolution k space inputs. Experimental results demonstrate that the method achieves a structural similarity index of 0.9116 and a mean squared error of 0.076 when reconstructing SMWI from 256 by 256 k space data, while maintaining critical diagnostic features. This approach enables high quality SMWI reconstruction from reduced k space sampling, leading to shorter scan times without compromising diagnostic detail. The proposed method has the potential to improve the clinical applicability of SMWI for Parkinsons disease and support faster and more efficient neuroimaging workflows.
Abstract (translated)
Susceptibility Map Weighted Imaging (SMWI) 是一种先进的磁共振成像技术，用于检测帕金森病中的黑质高信号。然而，全分辨率的 SMWI 获取受到长时间扫描的影响。因此，需要高效的重建方法来利用减少的空间(k空间)数据生成高质量的 SMWI 图像并保持诊断相关性。 在这项工作中，我们提出了一种基于复数Swin Transformer网络的方法，用于多回波MRI数据的超分辨率重建。该方法可以从低分辨率k空间输入中重构出高质量的SMWI图像。实验结果表明，在从256×256 k空间数据重构 SMWI 时，所提方法在结构相似性指数 (SSIM) 和均方误差 (MSE) 分别达到0.9116和0.076的同时，保持了关键的诊断特征。这种方法使SMWI可以从减少的空间采样中进行高质量重建，从而缩短扫描时间而不损害诊断细节。 该方法有望提高 SMWI 在帕金森病临床应用中的适用性，并支持更快、更高效的神经影像工作流程。
URL
https://arxiv.org/abs/2512.22202
PDF
-
An Empirical Study of Sampling Hyperparameters in Diffusion-Based Super-Resolution
2025-12-19 15:17:12Yudhistira Arief WibowoAbstract
Diffusion models have shown strong potential for solving inverse problems such as single-image super-resolution, where a high-resolution image is recovered from a low-resolution observation using a pretrained unconditional prior. Conditioning methods, including Diffusion Posterior Sampling (DPS) and Manifold Constrained Gradient (MCG), can substantially improve reconstruction quality, but they introduce additional hyperparameters that require careful tuning. In this work, we conduct an empirical ablation study on FFHQ super-resolution to identify the dominant factors affecting performance when applying conditioning to pretrained diffusion models, and show that the conditioning step size has a significantly greater impact than the diffusion step count, with step sizes in the range of [2.0, 3.0] yielding the best overall performance in our experiments.
Abstract (translated)
扩散模型在解决诸如单图像超分辨率等逆向问题方面展现出强大的潜力，即通过预训练的无条件先验从低分辨率观测中恢复高分辨率图像。条件方法（包括扩散后验采样(DPS)和流形约束梯度(MCG)）可以显著提高重建质量，但它们引入了需要仔细调整的额外超参数。在这项工作中，我们通过对FFHQ超分辨率进行实证消融研究，识别出在将条件应用于预训练扩散模型时影响性能的主要因素，并表明条件步骤大小比扩散步数对性能的影响更大，在我们的实验中，范围为[2.0, 3.0]的步骤大小产生了最佳的整体效果。
URL
https://arxiv.org/abs/2512.17675
PDF
-
Self-Supervised Weighted Image Guided Quantitative MRI Super-Resolution
2025-12-19 14:15:31Alireza Samadifardheris, Dirk H. J. Poot, Florian Wiesinger, Stefan Klein, Juan A. Hernandez-TamamesAbstract
High-resolution (HR) quantitative MRI (qMRI) relaxometry provides objective tissue characterization but remains clinically underutilized due to lengthy acquisition times. We propose a physics-informed, self-supervised framework for qMRI super-resolution that uses routinely acquired HR weighted MRI (wMRI) scans as guidance, thus, removing the necessity for HR qMRI ground truth during training. We formulate super-resolution as Bayesian maximum a posteriori inference, minimizing two discrepancies: (1) between HR images synthesized from super-resolved qMRI maps and acquired wMRI guides via forward signal models, and (2) between acquired LR qMRI and downsampled predictions. This physics-informed objective allows the models to learn from clinical wMRI without HR qMRI supervision. To validate the concept, we generate training data by synthesizing wMRI guides from HR qMRI using signal equations, then degrading qMRI resolution via k-space truncation. A deep neural network learns the super-resolution mapping. Ablation experiments demonstrate that T1-weighted images primarily enhance T1 maps, T2-weighted images improve T2 maps, and combined guidance optimally enhances all parameters simultaneously. Validation on independently acquired in-vivo data from a different qMRI sequence confirms cross-qMRI sequence generalizability. Models trained on synthetic data can produce super-resolved maps from a 1-minute acquisition with quality comparable to a 5-minute reference scan, leveraging the scanner-independent nature of relaxometry parameters. By decoupling training from HR qMRI requirement, our framework enables fast qMRI acquisitions enhanced via routine clinical images, offering a practical pathway for integrating quantitative relaxometry into clinical workflows with acceptable additional scan time.
Abstract (translated)
高分辨率（HR）定量磁共振成像（qMRI）弛豫测量能够提供客观的组织特征描述，但由于采集时间过长，在临床上的应用相对较少。我们提出了一种基于物理信息的自监督框架，用于qMRI超分辨率处理，该框架使用常规获取的HR加权MRI (wMRI)扫描作为指导，因此在训练过程中不需要高分辨率qMRI的真实数据。我们将超分辨率问题形式化为贝叶斯最大后验概率推理，并最小化两种差异：（1）从超分辨率qMRI映射合成的HR图像与通过前向信号模型获得的wMRI引导之间的差异；（2）实际低分辨率（LR）qMRI和下采样预测之间的差异。这种基于物理信息的目标使得模型能够在没有高分辨率qMRI监督的情况下，从临床wMRI中学习。 为了验证这一概念的有效性，我们生成训练数据的方法是：通过信号方程从HR qMRI合成wMRI引导，然后通过对k空间截断来降低qMRI的分辨率。深度神经网络学习超分辨率映射。消融实验表明，T1加权图像主要增强T1图谱，T2加权图像改善T2图谱，而联合指导则最优地同时增强了所有参数。 在独立获得的不同qMRI序列的体内数据上进行验证，证明了模型具有跨qMRI序列的泛化能力。基于合成数据训练得到的模型可以从一个1分钟采集的数据中生成与5分钟参考扫描质量相当的超分辨率图谱，这得益于弛豫参数不受扫描仪限制的特性。 通过将训练过程从高分辨率qMRI需求中解耦出来，我们的框架能够利用常规临床图像来加速qMRI采集。这一方法提供了一条实用路径，可以在可接受的额外扫描时间内，将定量松弛测量集成到临床工作流程中。
URL
https://arxiv.org/abs/2512.17612
PDF
-
Multi-level distortion-aware deformable network for omnidirectional image super-resolution
2025-12-19 08:35:08Cuixin Yang, Rongkang Dong, Kin-Man Lam, Yuhang Zhang, Guoping QiuAbstract
As augmented reality and virtual reality applications gain popularity, image processing for OmniDirectional Images (ODIs) has attracted increasing attention. OmniDirectional Image Super-Resolution (ODISR) is a promising technique for enhancing the visual quality of ODIs. Before performing super-resolution, ODIs are typically projected from a spherical surface onto a plane using EquiRectangular Projection (ERP). This projection introduces latitude-dependent geometric distortion in ERP images: distortion is minimal near the equator but becomes severe toward the poles, where image content is stretched across a wider area. However, existing ODISR methods have limited sampling ranges and feature extraction capabilities, which hinder their ability to capture distorted patterns over large areas. To address this issue, we propose a novel Multi-level Distortion-aware Deformable Network (MDDN) for ODISR, designed to expand the sampling range and receptive field. Specifically, the feature extractor in MDDN comprises three parallel branches: a deformable attention mechanism (serving as the dilation=1 path) and two dilated deformable convolutions with dilation rates of 2 and 3. This architecture expands the sampling range to include more distorted patterns across wider areas, generating dense and comprehensive features that effectively capture geometric distortions in ERP images. The representations extracted from these deformable feature extractors are adaptively fused in a multi-level feature fusion module. Furthermore, to reduce computational cost, a low-rank decomposition strategy is applied to dilated deformable convolutions. Extensive experiments on publicly available datasets demonstrate that MDDN outperforms state-of-the-art methods, underscoring its effectiveness and superiority in ODISR.
Abstract (translated)
随着增强现实和虚拟现实应用的普及，全景图像（OmniDirectional Images, ODIs）的图像处理技术越来越受到关注。OmniDirectional Image Super-Resolution (ODISR) 是一种有前景的技术，用于提高 ODIs 的视觉质量。在进行超分辨率处理之前，通常会将 ODIs 从球面投影到平面上，采用的是等矩形投影（EquiRectangular Projection, ERP）。这种投影会在 ERP 图像中引入纬度相关的几何失真：在赤道附近失真较小，而在两极地区则显著增加，导致图像内容在更广泛的区域内拉伸。然而，现有的 ODISR 方法具有有限的采样范围和特征提取能力，这限制了它们捕捉大范围内失真模式的能力。为了解决这一问题，我们提出了一种新颖的多级扭曲感知变形网络（Multi-level Distortion-aware Deformable Network, MDDN），旨在扩大采样范围和感受野。具体而言，MDDN 中的特征提取器包括三个并行分支：一个可变形注意力机制（作为 dilation=1 的路径）以及两个具有扩张率为 2 和 3 的可变形膨胀卷积。这种架构扩展了采样范围，能够覆盖更广泛的失真模式，生成密集且全面的特征，有效捕捉 ERP 图像中的几何扭曲。从这些可变形特征提取器中抽取的表示在多级特征融合模块中自适应地进行融合。此外，为了减少计算成本，在扩张可变形卷积上应用了低秩分解策略。在公共数据集上的广泛实验表明，MDDN 在 ODISR 方面优于最先进的方法，证明了其有效性和优越性。
URL
https://arxiv.org/abs/2512.17343
PDF
-
CogSR: Semantic-Aware Speech Super-Resolution via Chain-of-Thought Guided Flow Matching
2025-12-18 08:46:21Jiajun Yuan, Xiaochen Wang, Yuhang Xiao, Yulin Wu, Chenhao Hu, Xueyang LvAbstract
Applying speech super-resolution (SR) to recordings with severely low sampling rates is a critical challenge in digital archiving and investigative audio recovery. In these scenarios, the input lacks essential acoustic cues. Consequently, existing generative models often fail; without sufficient context, they hallucinate phonetic content, guessing words based on probability rather than meaning. To address this, we propose CogSR, a framework designed specifically for high-precision, offline restoration. Our approach shifts the focus from simple signal mapping to cognitive reconstruction. By integrating a Large Audio-Language Model, we employ Chain-of-Thought reasoning to act as a semantic anchor, while explicit acoustic priors ensure the speaker's identity remains consistent. This guides a Rectified Flow backbone to synthesize high-frequency details that are not only realistic but linguistically accurate. Evaluations show that CogSR effectively eliminates ambiguity in severe degradation regimes, making it a robust solution for restoring high-value legacy and surveillance audio.
Abstract (translated)
将语音超分辨率（SR）技术应用于采样率极低的录音，在数字档案和调查音频恢复领域是一个关键挑战。在这种情况下，输入缺乏重要的声学线索。因此，现有的生成模型往往失败；由于缺少足够的上下文信息，它们会臆造出可能不符合实际意义的声音内容。为了解决这一问题，我们提出了一种名为CogSR的框架，该框架专门用于高精度、离线恢复任务。我们的方法将重点从简单的信号映射转移到认知重建上。通过整合大型音频-语言模型，我们利用链式思维推理充当语义锚点，同时显式的声学先验确保说话人的身份一致性。这引导了一个校正流主干网络合成高频细节，这些细节不仅现实而且在语音学上准确无误。 评估结果显示，CogSR能够有效消除严重退化情况下的模糊性，使其成为恢复高价值的历史和监控音频的可靠解决方案。
URL
https://arxiv.org/abs/2512.16304
PDF
-
Pixel Super-Resolved Fluorescence Lifetime Imaging Using Deep Learning
2025-12-18 07:28:10Paloma Casteleiro Costa, Parnian Ghapandar Kashani, Xuhui Liu, Alexander Chen, Ary Portes, Julien Bec, Laura Marcu, Aydogan OzcanAbstract
Fluorescence lifetime imaging microscopy (FLIM) is a powerful quantitative technique that provides metabolic and molecular contrast, offering strong translational potential for label-free, real-time diagnostics. However, its clinical adoption remains limited by long pixel dwell times and low signal-to-noise ratio (SNR), which impose a stricter resolution-speed trade-off than conventional optical imaging approaches. Here, we introduce FLIM_PSR_k, a deep learning-based multi-channel pixel super-resolution (PSR) framework that reconstructs high-resolution FLIM images from data acquired with up to a 5-fold increased pixel size. The model is trained using the conditional generative adversarial network (cGAN) framework, which, compared to diffusion model-based alternatives, delivers a more robust PSR reconstruction with substantially shorter inference times, a crucial advantage for practical deployment. FLIM_PSR_k not only enables faster image acquisition but can also alleviate SNR limitations in autofluorescence-based FLIM. Blind testing on held-out patient-derived tumor tissue samples demonstrates that FLIM_PSR_k reliably achieves a super-resolution factor of k = 5, resulting in a 25-fold increase in the space-bandwidth product of the output images and revealing fine architectural features lost in lower-resolution inputs, with statistically significant improvements across various image quality metrics. By increasing FLIM's effective spatial resolution, FLIM_PSR_k advances lifetime imaging toward faster, higher-resolution, and hardware-flexible implementations compatible with low-numerical-aperture and miniaturized platforms, better positioning FLIM for translational applications.
Abstract (translated)
荧光寿命成像显微镜（FLIM）是一种强大的定量技术，可提供代谢和分子对比度，并且具有重要的转化潜力，可用于无标签实时诊断。然而，由于像素停留时间长以及信噪比低，其临床应用受到了限制，这些因素对分辨率与速度的权衡要求比传统光学成像方法更为严格。在这里，我们引入了FLIM_PSR_k，这是一种基于深度学习的多通道像素超分辨率（PSR）框架，可以从使用高达5倍像素尺寸采集的数据中重建高分辨率的FLIM图像。该模型采用条件生成对抗网络（cGAN）架构进行训练，相比于基于扩散模型的方法，在提供更稳健的PSR重建的同时，还具有显著缩短推理时间的优势，这对于实际部署至关重要。 FLIM_PSR_k不仅加快了成像速度，还可以缓解自体荧光基FLIM中的信噪比限制。在未公开的患者来源肿瘤组织样本上的盲测结果表明，FLIM_PSR_k可靠地实现了k=5倍超分辨率，从而将输出图像的空间带宽产品提高了25倍，并揭示了低分辨率输入中丢失的精细结构特征，在各种图像质量指标上均有统计学意义上的显著改善。通过提高FLIM的有效空间分辨率，FLIM_PSR_k推进了寿命成像技术朝向更快、更高分辨率以及硬件灵活兼容的方向发展，尤其适用于低数值孔径和微型化平台，并且更好地将FLIM定位为转化应用的理想选择。
URL
https://arxiv.org/abs/2512.16266
PDF
-
MMMamba: A Versatile Cross-Modal In Context Fusion Framework for Pan-Sharpening and Zero-Shot Image Enhancement
2025-12-17 10:07:09Yingying Wang, Xuanhua He, Chen Wu, Jialing Huang, Suiyun Zhang, Rui Liu, Xinghao Ding, Haoxuan CheAbstract
Pan-sharpening aims to generate high-resolution multispectral (HRMS) images by integrating a high-resolution panchromatic (PAN) image with its corresponding low-resolution multispectral (MS) image. To achieve effective fusion, it is crucial to fully exploit the complementary information between the two modalities. Traditional CNN-based methods typically rely on channel-wise concatenation with fixed convolutional operators, which limits their adaptability to diverse spatial and spectral variations. While cross-attention mechanisms enable global interactions, they are computationally inefficient and may dilute fine-grained correspondences, making it difficult to capture complex semantic relationships. Recent advances in the Multimodal Diffusion Transformer (MMDiT) architecture have demonstrated impressive success in image generation and editing tasks. Unlike cross-attention, MMDiT employs in-context conditioning to facilitate more direct and efficient cross-modal information exchange. In this paper, we propose MMMamba, a cross-modal in-context fusion framework for pan-sharpening, with the flexibility to support image super-resolution in a zero-shot manner. Built upon the Mamba architecture, our design ensures linear computational complexity while maintaining strong cross-modal interaction capacity. Furthermore, we introduce a novel multimodal interleaved (MI) scanning mechanism that facilitates effective information exchange between the PAN and MS modalities. Extensive experiments demonstrate the superior performance of our method compared to existing state-of-the-art (SOTA) techniques across multiple tasks and benchmarks.
Abstract (translated)
光谱锐化（Pan-sharpening）的目标是通过整合高分辨率全色（PAN）图像和对应的低分辨率多光谱（MS）图像，生成高质量的高分辨多光谱（HRMS）图像。为了实现有效的融合，充分挖掘这两种模态之间的互补信息至关重要。传统的基于CNN的方法通常依赖于通道级串联以及固定的卷积操作符，这限制了它们适应各种空间和光谱变化的能力。虽然交叉注意力机制能够提供全局交互作用，但其计算效率较低，并且可能稀释精细对应关系，难以捕捉复杂的语义关系。 最近，在多模态扩散变压器（MMDiT）架构上的进展在图像生成和编辑任务中取得了令人印象深刻的成功表现。不同于交叉注意力机制，MMDiT采用上下文条件化以促进更加直接高效的跨模态信息交换。本文提出了一种新的光谱锐化框架——MMMamba，这是一种基于Mamba架构的跨模态上下文融合框架，并且能够支持零样本图像超分辨率任务。 我们的设计在确保线性计算复杂度的同时，仍然保持了强大的跨模态交互能力。此外，我们还引入了一个新颖的多模态交错（MI）扫描机制，以促进PAN和MS模态之间的有效信息交换。广泛的实验结果证明了我们的方法在多个任务和基准上的性能优于现有的最先进（SOTA）技术。
URL
https://arxiv.org/abs/2512.15261
PDF
-
MVGSR: Multi-View Consistent 3D Gaussian Super-Resolution via Epipolar Guidance
2025-12-17 03:23:12Kaizhe Zhang, Shinan Chen, Qian Zhao, Weizhan Zhang, Caixia Yan, Yudeng XinAbstract
Scenes reconstructed by 3D Gaussian Splatting (3DGS) trained on low-resolution (LR) images are unsuitable for high-resolution (HR) rendering. Consequently, a 3DGS super-resolution (SR) method is needed to bridge LR inputs and HR rendering. Early 3DGS SR methods rely on single-image SR networks, which lack cross-view consistency and fail to fuse complementary information across views. More recent video-based SR approaches attempt to address this limitation but require strictly sequential frames, limiting their applicability to unstructured multi-view datasets. In this work, we introduce Multi-View Consistent 3D Gaussian Splatting Super-Resolution (MVGSR), a framework that focuses on integrating multi-view information for 3DGS rendering with high-frequency details and enhanced consistency. We first propose an Auxiliary View Selection Method based on camera poses, making our method adaptable for arbitrarily organized multi-view datasets without the need of temporal continuity or data reordering. Furthermore, we introduce, for the first time, an epipolar-constrained multi-view attention mechanism into 3DGS SR, which serves as the core of our proposed multi-view SR network. This design enables the model to selectively aggregate consistent information from auxiliary views, enhancing the geometric consistency and detail fidelity of 3DGS representations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on both object-centric and scene-level 3DGS SR benchmarks.
Abstract (translated)
使用低分辨率（LR）图像训练的3D高斯点云重建方法（3D Gaussian Splatting，简称3DGS）所重构的场景对于高清渲染来说是不合适的。因此需要一种3DGS超分辨技术来连接低分辨率输入和高质量的高清渲染。早期的3DGS超分辨技术主要依赖于单图像超分辨网络，这些技术缺乏视图间的一致性，并且无法融合来自不同视角的互补信息。最近一些基于视频序列的技术试图解决这一问题，但要求严格的顺序帧输入，限制了其在非结构化多视图数据集中的应用。 本文中我们引入了一种新的方法：多视角一致3D高斯点云超分辨技术（MVGSR），该框架旨在整合多视角信息以提高基于3DGS渲染的高频细节和一致性。首先，我们提出一种基于摄像机姿态的辅助视图选择方法，使我们的方法能够适应任意组织化的多视角数据集，并且无需时间连续性或数据重新排序。 此外，我们首次在3DGS超分辨技术中引入了符合基线约束的多视角注意力机制，这是我们的多视角超分辨网络的核心设计。这种设计使得模型可以有选择地从辅助视图中聚合一致的信息，从而增强了3DGS表示的几何一致性与细节准确性。 大量的实验表明，在面向对象和场景级别的基于3DGS的超分辨率基准测试上，该方法达到了最先进的性能表现。
URL
https://arxiv.org/abs/2512.15048
PDF
-
TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration
2025-12-16 16:25:47Zhiwen Yang, Jiaju Zhang, Yang Yi, Jian Liang, Bingzheng Wei, Yan XuAbstract
Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at this https URL.
Abstract (translated)
医学图像恢复（MedIR）的目标是从低质量的医学图像中恢复出高质量的图像。最近，MedIR领域的发展集中在能够同时处理多种不同任务的全功能模型上。然而，由于不同的成像模态和退化类型之间存在显著差异，使用同一共享模型来应对这些多样化任务需要仔细考虑两个关键的任务间关系：即任务干扰（当不同任务在同一参数上的梯度更新方向发生冲突时产生）以及任务不平衡（由各个任务固有的学习难度不均造成的优化不平衡）。为了解决这些问题，我们提出了一种任务自适应的变换器（TAT），这是一种通过两种创新方法动态调整以应对不同任务的新框架。首先，引入了任务自适应权重生成策略来减轻任务干扰，该策略针对每个任务生成特定的任务权重参数，从而消除了共享权重参数上的潜在梯度冲突。其次，我们提出了一个任务自适应损失平衡策略，根据各个任务的具体学习难度动态调整损失权重，防止任务的主导或欠训练现象。 通过广泛的实验表明，在PET合成、CT去噪和MRI超分辨率这三种MedIR任务中，我们的TAT框架在特定任务设置以及全功能模型设置下均达到了最先进的性能水平。代码可在提供的链接处获取。
URL
https://arxiv.org/abs/2512.14550
PDF
-
Bridging Fidelity-Reality with Controllable One-Step Diffusion for Image Super-Resolution
2025-12-16 03:56:02Hao Chen, Junyang Chen, Jinshan Pan, Jiangxin DongAbstract
Recent diffusion-based one-step methods have shown remarkable progress in the field of image super-resolution, yet they remain constrained by three critical limitations: (1) inferior fidelity performance caused by the information loss from compression encoding of low-quality (LQ) inputs; (2) insufficient region-discriminative activation of generative priors; (3) misalignment between text prompts and their corresponding semantic regions. To address these limitations, we propose CODSR, a controllable one-step diffusion network for image super-resolution. First, we propose an LQ-guided feature modulation module that leverages original uncompressed information from LQ inputs to provide high-fidelity conditioning for the diffusion process. We then develop a region-adaptive generative prior activation method to effectively enhance perceptual richness without sacrificing local structural fidelity. Finally, we employ a text-matching guidance strategy to fully harness the conditioning potential of text prompts. Extensive experiments demonstrate that CODSR achieves superior perceptual quality and competitive fidelity compared with state-of-the-art methods with efficient one-step inference.
Abstract (translated)
最近基于扩散的一步法在图像超分辨率领域取得了显著进步，但仍受三个关键限制的制约：(1) 由于对低质量（LQ）输入进行压缩编码导致的信息丢失而产生的较低保真度性能；(2) 生成功能未能充分激活区域识别功能；(3) 文本提示与其相应语义区域之间的不匹配。为解决这些问题，我们提出了CODSR，一种用于图像超分辨率的可控一步扩散网络。首先，我们提出了一种LQ引导特征调节模块，该模块利用来自LQ输入的原始未压缩信息来为扩散过程提供高保真度条件设置。然后，我们开发了一种区域自适应生成功能激活方法，以有效增强感知丰富性而不牺牲局部结构保真度。最后，我们采用一种文本匹配指导策略，充分利用文本提示的调节潜力。广泛的实验表明，CODSR在视觉质量和保真度方面优于最先进的方法，并且其一步推理效率非常高。
URL
https://arxiv.org/abs/2512.14061
PDF
-
Coarse-to-Fine Hierarchical Alignment for UAV-based Human Detection using Diffusion Models
2025-12-15 19:57:36Wenda Li, Meng Wu, Sungmin Eum, Heesung Kwon, Qing QuAbstract
Training object detectors demands extensive, task-specific annotations, yet this requirement becomes impractical in UAV-based human detection due to constantly shifting target distributions and the scarcity of labeled images. As a remedy, synthetic simulators are adopted to generate annotated data, with a low annotation cost. However, the domain gap between synthetic and real images hinders the model from being effectively applied to the target domain. Accordingly, we introduce Coarse-to-Fine Hierarchical Alignment (CFHA), a three-stage diffusion-based framework designed to transform synthetic data for UAV-based human detection, narrowing the domain gap while preserving the original synthetic labels. CFHA explicitly decouples global style and local content domain discrepancies and bridges those gaps using three modules: (1) Global Style Transfer -- a diffusion model aligns color, illumination, and texture statistics of synthetic images to the realistic style, using only a small real reference set; (2) Local Refinement -- a super-resolution diffusion model is used to facilitate fine-grained and photorealistic details for the small objects, such as human instances, preserving shape and boundary integrity; (3) Hallucination Removal -- a module that filters out human instances whose visual attributes do not align with real-world data to make the human appearance closer to the target distribution. Extensive experiments on public UAV Sim2Real detection benchmarks demonstrate that our methods significantly improve the detection accuracy compared to the non-transformed baselines. Specifically, our method achieves up to $+14.1$ improvement of mAP50 on Semantic-Drone benchmark. Ablation studies confirm the complementary roles of the global and local stages and highlight the importance of hierarchical alignment. The code is released at \href{this https URL}{this url}.
Abstract (translated)
训练目标检测器需要大量的、特定任务的标注数据，但在基于无人机的人类检测中，由于目标分布不断变化以及标记图像稀少，这一要求变得不切实际。作为解决方案，合成模拟器被用来生成有注释的数据，这大大降低了标注成本。然而，合成与真实图像之间的领域差距阻碍了模型在目标领域的有效应用。为此，我们引入了一种名为“粗到细层次对齐”（Coarse-to-Fine Hierarchical Alignment, CFHA）的三阶段扩散框架，旨在将合成数据转换为适用于基于无人机的人类检测，并缩小域间差异，同时保持原始标签信息不变。 CFHA明确地解耦了全局风格与局部内容领域的不一致，并通过三个模块来弥补这些差距：（1）全局样式迁移——一个扩散模型仅使用少量的真实参考集，就能将合成图像的颜色、光照和纹理统计特性对齐到真实样式的水平；（2）局部细化——一种超分辨率扩散模型被用来促进小物体（如人类实例）的细粒度和逼真细节，同时保持形状和边界的完整性；（3）幻觉移除——一个过滤模块将视觉属性与现实世界数据不一致的人类实例排除出去，使人类外观更接近目标分布。 在公共无人机Sim2Real检测基准上的广泛实验表明，我们的方法相比于未转换的基线显著提高了检测精度。具体而言，在Semantic-Drone基准上，我们的方法实现了高达+14.1 mAP50的改进。消融研究表明全局和局部阶段之间的互补作用，并强调了层次对齐的重要性。代码可在[此处](this https URL)获取。
URL
https://arxiv.org/abs/2512.13869
PDF
-
Time-aware UNet and super-resolution deep residual networks for spatial downscaling
2025-12-15 08:19:28Mika Sipil\"a, Sabrina Maggio, Sandra De Iaco, Klaus Nordhausen, Monica Palma, Sara TaskinenAbstract
Satellite data of atmospheric pollutants are often available only at coarse spatial resolution, limiting their applicability in local-scale environmental analysis and decision-making. Spatial downscaling methods aim to transform the coarse satellite data into high-resolution fields. In this work, two widely used deep learning architectures, the super-resolution deep residual network (SRDRN) and the encoder-decoder-based UNet, are considered for spatial downscaling of tropospheric ozone. Both methods are extended with a lightweight temporal module, which encodes observation time using either sinusoidal or radial basis function (RBF) encoding, and fuses the temporal features with the spatial representations in the networks. The proposed time-aware extensions are evaluated against their baseline counterparts in a case study on ozone downscaling over Italy. The results suggest that, while only slightly increasing computational complexity, the temporal modules significantly improve downscaling performance and convergence speed.
Abstract (translated)
卫星大气污染物数据通常仅以较低的空间分辨率提供，这限制了它们在局部环境分析和决策中的应用。空间降尺度方法旨在将低分辨率的卫星数据转换为高分辨率的数据场。在这项工作中，考虑使用两种广泛使用的深度学习架构——超分辨率残差网络（SRDRN）和基于编码器-解码器的UNet，来进行对流层臭氧的空间降尺度处理。这两种方法都扩展了一个轻量级的时间模块，该模块通过正弦或径向基函数（RBF）编码来表示观测时间，并将这些时间特征与网络中的空间表示进行融合。在意大利臭氧降尺度案例研究中评估了所提出的具有时间感知功能的扩展方案及其基准模型的表现。结果表明，尽管计算复杂度仅略有增加，但时间模块显著提高了降尺度性能和收敛速度。
URL
https://arxiv.org/abs/2512.13753
PDF
