Super_Resolution
-
Conditional Diffusion for 3D CT Volume Reconstruction from 2D X-rays
2026-03-27 15:21:47Martin Rath, Morteza Ghahremani, Yitong Li, Ashkan Taghipour, Marcus Makowski, Christian WachingerAbstract
Computed tomography (CT) provides rich 3D anatomical details but is often constrained by high radiation exposure, substantial costs, and limited availability. While standard chest X-rays are cost-effective and widely accessible, they only provide 2D projections with limited pathological information. Reconstructing 3D CT volumes from 2D X-rays offers a transformative solution to increase diagnostic accessibility, yet existing methods predominantly rely on synthetic X-ray projections, limiting clinical generalization. In this work, we propose AXON, a multi-stage diffusion-based framework that reconstructs high-fidelity 3D CT volumes directly from real X-rays. AXON employs a coarse-to-fine strategy, with a Brownian Bridge diffusion model-based initial stage for global structural synthesis, followed by a ControlNet-based refinement stage for local intensity optimization. It also supports bi-planar X-ray input to mitigate depth ambiguities inherent in 2D-to-3D reconstruction. A super-resolution network is integrated to upscale the generated volumes to achieve diagnostic-grade resolution. Evaluations on both public and external datasets demonstrate that AXON significantly outperforms state-of-the-art baselines, achieving a 11.9% improvement in PSNR and a 11.0% increase in SSIM with robust generalizability across disparate clinical distributions. Our code is available at this https URL.
Abstract (translated)
计算机断层扫描（CT）可提供丰富的三维解剖细节，但常受限于高辐射暴露、高昂成本及可及性有限。而标准胸部X光片虽成本效益高且广泛可及，却仅能提供二维投影，病理信息有限。从二维X光重建三维CT体积为实现诊断可及性提升提供了变革性方案，但现有方法多依赖合成X光投影，限制了临床泛化能力。本研究提出AXON——一个基于多阶段扩散模型的框架，可直接从真实X光重建高保真三维CT体积。AXON采用由粗到细策略：首阶段基于布朗桥扩散模型进行全局结构合成，随后通过ControlNet进行局部强度优化。该框架同时支持双平面X光输入以缓解二维至三维重建固有的深度模糊问题，并集成超分辨率网络将生成体积上采样至诊断级分辨率。在公开及外部数据集上的评估表明，AXON显著优于现有最先进方法，在PSNR上提升11.9%，SSIM上提升11.0%，且在不同临床分布中展现出稳健的泛化能力。代码已公开于此https URL。
URL
https://arxiv.org/abs/2603.26509
PDF
-
InstaVSR: Taming Diffusion for Efficient and Temporally Consistent Video Super-Resolution
2026-03-27 07:33:13Jintong Hu, Bin Chen, Zhenyu Hu, Jiayue Liu, Guo Wang, Lu QiAbstract
Video super-resolution (VSR) seeks to reconstruct high-resolution frames from low-resolution inputs. While diffusion-based methods have substantially improved perceptual quality, extending them to video remains challenging for two reasons: strong generative priors can introduce temporal instability, and multi-frame diffusion pipelines are often too expensive for practical deployment. To address both challenges simultaneously, we propose InstaVSR, a lightweight diffusion framework for efficient video super-resolution. InstaVSR combines three ingredients: (1) a pruned one-step diffusion backbone that removes several costly components from conventional diffusion-based VSR pipelines, (2) recurrent training with flow-guided temporal regularization to improve frame-to-frame stability, and (3) dual-space adversarial learning in latent and pixel spaces to preserve perceptual quality after backbone simplification. On an NVIDIA RTX 4090, InstaVSR processes a 30-frame video at 2K$\times$2K resolution in under one minute with only 7 GB of memory usage, substantially reducing the computational cost compared to existing diffusion-based methods while maintaining favorable perceptual quality with significantly smoother temporal transitions.
Abstract (translated)
视频超分辨率（VSR）旨在从低分辨率输入重建高分辨率帧。尽管基于扩散的方法已显著提升了感知质量，但将其扩展到视频领域仍面临两大挑战：强大的生成先验可能引入时间不稳定性，而多帧扩散流程通常因计算成本过高难以实际部署。为同时应对这两项挑战，我们提出了InstaVSR——一个轻量级的高效视频超分辨率扩散框架。InstaVSR融合三项核心技术：（1）通过剪枝单步扩散主干网络，移除了传统基于扩散的VSR流程中多个高成本组件；（2）采用基于光流引导的时间正则化循环训练，以提升帧间稳定性；（3）在隐空间与像素空间实施双空间对抗学习，确保主干简化后仍能保持感知质量。在NVIDIA RTX 4090显卡上，InstaVSR处理30帧2K×2K分辨率视频耗时不足一分钟，内存占用仅7GB，相较于现有基于扩散的方法大幅降低计算成本，同时保持了更优的感知质量与显著更平滑的时间过渡效果。
URL
https://arxiv.org/abs/2603.26134
PDF
-
ScrollScape: Unlocking 32K Image Generation With Video Diffusion Priors
2026-03-25 13:03:02Haodong Yu, Yabo Zhang, Donglin Di, Ruyi Zhang, Wangmeng ZuoAbstract
While diffusion models excel at generating images with conventional dimensions, pushing them to synthesize ultra-high-resolution imagery at extreme aspect ratios (EAR) often triggers catastrophic structural failures, such as object repetition and spatial this http URL limitation fundamentally stems from a lack of robust spatial priors, as static text-to-image models are primarily trained on image distributions with conventional this http URL overcome this bottleneck, we present ScrollScape, a novel framework that reformulates EAR image synthesis into a continuous video generation process through two core this http URL mapping the spatial expansion of a massive canvas to the temporal evolution of video frames, ScrollScape leverages the inherent temporal consistency of video models as a powerful global constraint to ensure long-range structural this http URL, Scanning Positional Encoding (ScanPE) distributes global coordinates across frames to act as a flexible moving camera, while Scrolling Super-Resolution (ScrollSR) leverages video super-resolution priors to circumvent memory bottlenecks, efficiently scaling outputs to an unprecedented 32K resolution. Fine-tuned on a curated 3K multi-ratio image dataset, ScrollScape effectively aligns pre-trained video priors with the EAR generation task. Extensive evaluations demonstrate that it significantly outperforms existing image-diffusion baselines by eliminating severe localized artifacts. Consequently, our method overcomes inherent structural bottlenecks to ensure exceptional global coherence and visual fidelity across diverse domains at extreme scales.
Abstract (translated)
尽管扩散模型在生成常规尺寸图像方面表现出色，但将其推向极端纵横比（EAR）的超高分辨率图像合成时，常会引发灾难性的结构失效，例如物体重复和空间局限。这一根本限制源于缺乏稳健的空间先验，因为静态文本到图像模型主要基于常规尺寸图像分布进行训练。为突破这一瓶颈，我们提出了ScrollScape，一种新颖框架，通过两个核心机制将EAR图像合成重新构建为连续视频生成过程：通过将超大画布的空间扩展映射到视频帧的时间演变，ScrollScape利用视频模型固有的时间一致性作为强大的全局约束，以确保长程结构连贯性。扫描位置编码（ScanPE）将全局坐标分布到各帧中，充当灵活的移动摄像机；而滚动超分辨率（ScrollSR）则利用视频超分辨率先验来规避内存瓶颈，高效将输出扩展至前所未有的32K分辨率。经精心策划的3K多纵横比图像数据集微调后，ScrollScape能有效将预训练的视频先验与EAR生成任务对齐。广泛评估表明，该方法通过消除严重的局部伪影，显著优于现有图像扩散基线。因此，我们的方法克服了固有的结构瓶颈，确保了在极端尺度下跨不同领域的卓越全局连贯性和视觉保真度。
URL
https://arxiv.org/abs/2603.24270
PDF
-
InstanceRSR: Real-World Super-Resolution via Instance-Aware Representation Alignment
2026-03-25 12:27:58Zixin Guo, Kai Zhao, Luyan ZhangAbstract
Existing real-world super-resolution (RSR) methods based on generative priors have achieved remarkable progress in producing high-quality and globally consistent reconstructions. However, they often struggle to recover fine-grained details of diverse object instances in complex real-world scenes. This limitation primarily arises because commonly adopted denoising losses (e.g., MSE) inherently favor global consistency while neglecting instance-level perception and restoration. To address this issue, we propose InstanceRSR, a novel RSR framework that jointly models semantic information and introduces instance-level feature alignment. Specifically, we employ low-resolution (LR) images as global consistency guidance while jointly modeling image data and semantic segmentation maps to enforce semantic relevance during sampling. Moreover, we design an instance representation learning module to align the diffusion latent space with the instance latent space, enabling instance-aware feature alignment, and further incorporate a scale alignment mechanism to enhance fine-grained perception and detail recovery. Benefiting from these designs, our approach not only generates photorealistic details but also preserves semantic consistency at the instance level. Extensive experiments on multiple real-world benchmarks demonstrate that InstanceRSR significantly outperforms existing methods in both quantitative metrics and visual quality, achieving new state-of-the-art (SOTA) performance.
Abstract (translated)
现有基于生成先验的真实世界超分辨率（RSR）方法在生成高质量、全局一致的图像重建方面取得了显著进展。然而，它们在复杂真实场景中恢复多样目标实例的细粒度细节方面往往存在不足。这一局限性主要源于常用的去噪损失函数（如MSE）本质上倾向于全局一致性，而忽略了实例级的感知与修复。针对该问题，我们提出了InstanceRSR，一种新型RSR框架，通过联合建模语义信息并引入实例级特征对齐来解决问题。具体而言，我们采用低分辨率（LR）图像作为全局一致性指导，同时联合建模图像数据与语义分割图，以在采样过程中强化语义相关性。此外，我们设计了实例表征学习模块，将扩散隐空间与实例隐空间进行对齐，实现实例感知的特征对齐，并进一步引入尺度对齐机制以增强细粒度感知与细节恢复能力。得益于上述设计，我们的方法不仅能生成照片般真实的细节，还能在实例层面保持语义一致性。在多个真实场景基准数据集上的大量实验表明，InstanceRSR在定量指标与视觉质量上均显著优于现有方法，达到了新的最先进（SOTA）性能。
URL
https://arxiv.org/abs/2603.24240
PDF
-
RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution
2026-03-25 11:20:51Yushuai Song, Weize Quan, Weining Wang, Jiahui Sun, Jing Liu, Meng Li, Pengbin Yu, Zhentao Chen, Wei Shen, Lunxi Yuan, Dong-ming YanAbstract
Recent advances in generative super-resolution (SR) have greatly improved visual realism, yet existing evaluation and optimization frameworks remain misaligned with human perception. Full-Reference and No-Reference metrics often fail to reflect perceptual preference, either penalizing semantically plausible details due to pixel misalignment or favoring visually sharp but inconsistent artifacts. Moreover, most SR methods rely on ground-truth (GT)-dependent distribution matching, which does not necessarily correspond to human judgments. In this work, we propose RefReward-SR, a low-resolution (LR) reference-aware reward model for preference-aligned SR. Instead of relying on GT supervision or NR evaluation, RefReward-SR assesses high-resolution (HR) reconstructions conditioned on their LR inputs, treating the LR image as a semantic anchor. Leveraging the visual-linguistic priors of a Multimodal Large Language Models (MLLM), it evaluates semantic consistency and plausibility in a reasoning-aware manner. To support this paradigm, we construct RefSR-18K, the first large-scale LR-conditioned preference dataset for SR, providing pairwise rankings based on LR-HR consistency and HR naturalness. We fine-tune the MLLM with Group Relative Policy Optimization (GRPO) using LR-conditioned ranking rewards, and further integrate GRPO into SR model training with RefReward-SR as the core reward signal for preference-aligned generation. Extensive experiments show that our framework achieves substantially better alignment with human judgments, producing reconstructions that preserve semantic consistency while enhancing perceptual plausibility and visual naturalness. Code, models, and datasets will be released upon paper acceptance.
Abstract (translated)
生成式超分辨率（SR）的最新进展显著提升了视觉真实感，但现有的评估与优化框架仍与人类感知存在偏差。全参考与无参考指标常因像素错位而惩罚语义合理的细节，或偏好视觉锐利但不一致的伪影，难以反映感知偏好。此外，多数SR方法依赖真值（GT）监督的分布匹配，这与人类判断并不必然一致。本研究提出RefReward-SR——一种基于低分辨率（LR）参考的感知对齐奖励模型。该方法不依赖GT监督或无参考评估，而是以LR图像作为语义锚点，评估其条件生成的高分辨率（HR）重建结果。通过利用多模态大语言模型（MLLM）的视觉-语言先验，RefReward-SR以具备推理能力的方式评估语义一致性与合理性。为支持该范式，我们构建了首个大规模LR条件偏好数据集RefSR-18K，提供基于LR-HR一致性与HR自然度的成对排序。我们采用组相对策略优化（GRPO）对MLLM进行微调，使用LR条件排序奖励，并将GRPO进一步集成到SR模型训练中，以RefReward-SR作为偏好对齐生成的核心奖励信号。大量实验表明，本框架与人类判断的契合度显著提升，所生成的重建结果在增强感知合理性与视觉自然度的同时，保持了语义一致性。论文接受后，代码、模型与数据集将公开释放。
URL
https://arxiv.org/abs/2603.24198
PDF
-
VoDaSuRe: A Large-Scale Dataset Revealing Domain Shift in Volumetric Super-Resolution
2026-03-24 12:54:05August Leander H{\o}eg, Sophia Wiinberg Bardenfleth, Hans Martin Kjer, Tim Bj{\o}rn Dyrby, Vedrana Andersen Dahl, Anders Bjorholm DahlAbstract
Recent advances in volumetric super-resolution (SR) have demonstrated strong performance in medical and scientific imaging, with transformer- and CNN-based approaches achieving impressive results even at extreme scaling factors. In this work, we show that much of this performance stems from training on downsampled data rather than real low-resolution scans. This reliance on downsampling is partly driven by the scarcity of paired high- and low-resolution 3D datasets. To address this, we introduce VoDaSuRe, a large-scale volumetric dataset containing paired high- and low-resolution scans. When training models on VoDaSuRe, we reveal a significant discrepancy: SR models trained on downsampled data produce substantially sharper predictions than those trained on real low-resolution scans, which smooth fine structures. Conversely, applying models trained on downsampled data to real scans preserves more structure but is inaccurate. Our findings suggest that current SR methods are overstated - when applied to real data, they do not recover structures lost in low-resolution scans and instead predict a smoothed average. We argue that progress in deep learning-based volumetric SR requires datasets with paired real scans of high complexity, such as VoDaSuRe. Our dataset and code are publicly available through: this https URL
Abstract (translated)
近期在体积超分辨率（SR）领域的研究进展已在医学与科学影像中展现出卓越性能，基于Transformer和CNN的方法甚至在极端缩放因子下仍能取得令人瞩目的成果。本研究指出，此类性能在很大程度上源于模型在降采样数据而非真实低分辨率扫描数据上的训练。这种对降采样的依赖部分源于成对高-低分辨率3D数据集的匮乏。为此，我们引入了VoDaSuRe——一个包含成对高-低分辨率扫描的大规模体积数据集。在该数据集上训练模型时，我们发现显著差异：在降采样数据上训练的SR模型能生成更锐利的预测结果，而在真实低分辨率扫描数据上训练的模型则会使精细结构平滑化。反之，将在降采样数据上训练的模型应用于真实扫描时，虽能保留更多结构但准确性不足。我们的研究表明，当前SR方法存在高估现象——当应用于真实数据时，它们无法恢复低分辨率扫描中丢失的结构，反而输出平滑化的平均预测。我们认为，基于深度学习的体积超分辨率研究需要像VoDaSuRe这样包含高复杂度真实成对扫描的数据集来推动进展。本数据集及代码已公开，可通过此链接获取。
URL
https://arxiv.org/abs/2603.23153
PDF
-
From Pixels to Semantics: A Multi-Stage AI Framework for Structural Damage Detection in Satellite Imagery
2026-03-24 03:50:41Bijay Shakya, Catherine Hoier, Khandaker Mamun AhmedAbstract
Rapid and accurate structural damage assessment following natural disasters is critical for effective emergency response and recovery. However, remote sensing imagery often suffers from low spatial resolution, contextual ambiguity, and limited semantic interpretability, reducing the reliability of traditional detection pipelines. In this work, we propose a novel hybrid framework that integrates AI-based super-resolution, deep learning object detection, and Vision-Language Models (VLMs) for comprehensive post-disaster building damage assessment. First, we enhance pre- and post-disaster satellite imagery using a Video Restoration Transformer (VRT) to upscale images from 1024x1024 to 4096x4096 resolution, improving structural detail visibility. Next, a YOLOv11-based detector localizes buildings in pre-disaster imagery, and cropped building regions are analyzed using VLMs to semantically assess structural damage across four severity levels. To ensure robust evaluation in the absence of ground-truth captions, we employ CLIPScore for reference-free semantic alignment and introduce a multi-model VLM-as-a-Jury strategy to reduce individual model bias in safety-critical decision making. Experiments on subsets of the xBD dataset, including the Moore Tornado and Hurricane Matthew events, demonstrate that the proposed framework enhances the semantic interpretation of damaged buildings. In addition, our framework provides helpful recommendations to first responders for recovery based on damage analysis.
Abstract (translated)
自然灾害后快速准确的结构损伤评估对于有效的应急响应和恢复至关重要。然而，遥感影像常受限于空间分辨率低、上下文模糊以及语义可解释性不足，这降低了传统检测管线的可靠性。本研究提出一种新颖的混合框架，该框架整合了基于AI的超分辨率技术、深度学习目标检测以及视觉语言模型（VLMs），用于综合灾后建筑损伤评估。首先，我们采用视频恢复Transformer（VRT）对灾前灾后卫星影像进行增强，将图像从1024×1024分辨率提升至4096×4096，从而改善结构细节的可视性。随后，基于YOLOv11的检测器在灾前影像中定位建筑物，并对裁剪后的建筑区域使用VLMs进行语义分析，按四个严重程度等级评估结构损伤。为确保在缺乏真实标注描述的情况下进行稳健评估，我们采用CLIPScore进行无参考语义对齐，并引入多模型VLM-as-a-Jury策略以减少安全关键决策中的个体模型偏差。在xBD数据集子集（包括摩尔龙卷风和飓风马修事件）上的实验表明，所提框架增强了受损建筑的语义解释能力。此外，我们的框架还能基于损伤分析为救援人员提供恢复建议。
URL
https://arxiv.org/abs/2603.22768
PDF
-
Single-Subject Multi-View MRI Super-Resolution via Implicit Neural Representations
2026-03-23 23:00:18Heejong Kim, Abhishek Thanki, Roel van Herten, Daniel Margolis, Mert R SabuncuAbstract
Clinical MRI frequently acquires anisotropic volumes with high in-plane resolution and low through-plane resolution to reduce acquisition time. Multiple orientations are therefore acquired to provide complementary anatomical information. Conventional integration of these views relies on registration followed by interpolation, which can degrade fine structural details. Recent deep learning-based super-resolution (SR) approaches have demonstrated strong performance in enhancing single-view images. However, their clinical reliability is often limited by the need for large-scale training datasets, resulting in increased dependence on cohort-level priors. Self-supervised strategies offer an alternative by learning directly from the target scans. Prior work either neglects the existence of multi-view information or assumes that in-plane information can supervise through-plane reconstruction under the assumption of pre-alignment between images. However, this assumption is rarely satisfied in clinical settings. In this work, we introduce Single-Subject Implicit Multi-View Super-Resolution for MRI (SIMS-MRI), a framework that operates solely on anisotropic multi-view scans from a single patient without requiring pre- or post-processing. Our method combines a multi-resolution hash-encoded implicit representation with learned inter-view alignment to generate a spatially consistent isotropic reconstruction. We validate the SIMS-MRI pipeline on both simulated brain and clinical prostate MRI datasets. Code will be made publicly available for reproducibility: this https URL
Abstract (translated)
临床磁共振成像（MRI）常通过采用高平面分辨率、低层间分辨率的各向异性扫描来缩短采集时间，因此需采集多角度视图以提供互补的解剖信息。传统多视图融合方法依赖图像配准与插值，可能损害精细结构细节。近期基于深度学习的超分辨率（SR）方法在单视图图像增强上表现优异，但其临床可靠性常受限于大规模训练数据集的需求，导致对队列级先验的依赖增强。自监督策略提供了一种替代方案，可直接从目标扫描数据中学习。既往工作要么忽略多视图信息的存在，要么假设图像预对齐后平面信息可监督层间重建，然而该假设在临床场景中极少成立。本研究提出单被试隐式多视图MRI超分辨率（SIMS-MRI）框架，该框架仅基于单患者各向异性多视图扫描运行，无需预处理或后处理。我们的方法结合多分辨率哈希编码隐式表示与学习的视图间对齐，生成空间一致的各向同性重建。我们在模拟脑部与临床前列腺MRI数据集上验证了SIMS-MRI流程。代码将公开以确保可复现性：此 https URL
URL
https://arxiv.org/abs/2603.22627
PDF
-
DUO-VSR: Dual-Stream Distillation for One-Step Video Super-Resolution
2026-03-23 17:56:17Zhengyao Lv, Menghan Xia, Xintao Wang, Kwan-Yee K. WongAbstract
Diffusion-based video super-resolution (VSR) has recently achieved remarkable fidelity but still suffers from prohibitive sampling costs. While distribution matching distillation (DMD) can accelerate diffusion models toward one-step generation, directly applying it to VSR often results in training instability alongside degraded and insufficient supervision. To address these issues, we propose DUO-VSR, a three-stage framework built upon a Dual-Stream Distillation strategy that unifies distribution matching and adversarial supervision for one-step VSR. Firstly, a Progressive Guided Distillation Initialization is employed to stabilize subsequent training through trajectory-preserving distillation. Next, the Dual-Stream Distillation jointly optimizes the DMD and Real-Fake Score Feature GAN (RFS-GAN) streams, with the latter providing complementary adversarial supervision leveraging discriminative features from both real and fake score models. Finally, a Preference-Guided Refinement stage further aligns the student with perceptual quality preferences. Extensive experiments demonstrate that DUO-VSR achieves superior visual quality and efficiency over previous one-step VSR approaches.
Abstract (translated)
基于扩散模型的视频超分辨率（VSR）技术近期在保真度上取得了显著进展，但仍受制于极高的采样成本。尽管分布匹配蒸馏（DMD）能够加速扩散模型实现一步生成，但直接将其应用于VSR常导致训练不稳定，并伴随监督信号退化与不足。针对这些问题，我们提出了DUO-VSR——一个基于双流蒸馏策略构建的三阶段框架，该策略统一了分布匹配与对抗式监督以实现一步VSR。首先，采用渐进式引导蒸馏初始化，通过轨迹保持蒸馏稳定后续训练。其次，双流蒸馏联合优化DMD流与真伪分数特征GAN（RFS-GAN）流，后者利用真实与伪造分数模型的判别特征提供互补的对抗监督。最后，通过偏好引导精炼阶段进一步使学生模型与感知质量偏好对齐。大量实验表明，DUO-VSR在视觉质量和效率上均优于此前的一步VSR方法。
URL
https://arxiv.org/abs/2603.22271
PDF
-
Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model
2026-03-23 13:49:06SII-GAIR, Sand. ai, , Ethan Chern, Hansi Teng, Hanwen Sun, Hao Wang, Hong Pan, Hongyu Jia, Jiadi Su, Jin Li, Junjie Yu, Lijie Liu, Lingzhi Li, Lyumanshan Ye, Min Hu, Qiangang Wang, Quanwei Qi, Steffi Chern, Tao Bu, Taoran Wang, Teren Xu, Tianning Zhang, Tiantian Mi, Weixian Xu, Wenqiang Zhang, Wentai Zhang, Xianping Yi, Xiaojie Cai, Xiaoyang Kang, Yan Ma, Yixiu Liu, Yunbo Zhang, Yunpeng Huang, Yutong Lin, Zewei Tao, Zhaoliang Liu, Zheng Zhang, Zhiyao Cen, Zhixuan Yu, Zhongshu Wang, Zhulin Hu, Zijin Zhou, Zinan Guo, Yue Cao, Pengfei LiuAbstract
We present daVinci-MagiHuman, an open-source audio-video generative foundation model for human-centric generation. daVinci-MagiHuman jointly generates synchronized video and audio using a single-stream Transformer that processes text, video, and audio within a unified token sequence via self-attention only. This single-stream design avoids the complexity of multi-stream or cross-attention architectures while remaining easy to optimize with standard training and inference infrastructure. The model is particularly strong in human-centric scenarios, producing expressive facial performance, natural speech-expression coordination, realistic body motion, and precise audio-video synchronization. It supports multilingual spoken generation across Chinese (Mandarin and Cantonese), English, Japanese, Korean, German, and French. For efficient inference, we combine the single-stream backbone with model distillation, latent-space super-resolution, and a Turbo VAE decoder, enabling generation of a 5-second 256p video in 2 seconds on a single H100 GPU. In automatic evaluation, daVinci-MagiHuman achieves the highest visual quality and text alignment among leading open models, along with the lowest word error rate (14.60%) for speech intelligibility. In pairwise human evaluation, it achieves win rates of 80.0% against Ovi 1.1 and 60.9% against LTX 2.3 over 2000 comparisons. We open-source the complete model stack, including the base model, the distilled model, the super-resolution model, and the inference codebase.
Abstract (translated)
我们推出daVinci-MagiHuman，这是一个面向人类中心生成的开源音视频生成基础模型。daVinci-MagiHuman采用仅通过自注意力机制在统一标记序列中处理文本、视频和音频的单流Transformer架构，联合生成同步的视频与音频。这种单流设计避免了多流或交叉注意力架构的复杂性，同时易于使用标准训练和推理基础设施进行优化。该模型在人类中心场景中表现尤为突出，能够生成富有表现力的面部表演、自然的语音-表情协调、逼真的身体动作以及精确的音视频同步。它支持中文（普通话和粤语）、英语、日语、韩语、德语和法语的多语言口语生成。为实现高效推理，我们将单流主干网络与模型蒸馏、潜在空间超分辨率以及Turbo VAE解码器相结合，可在单张H100 GPU上2秒内生成5秒256p视频。在自动评估中，daVinci-MagiHuman在视觉质量和文本对齐方面领先于其他主流开源模型，并以14.60%的最低词错误率实现语音可懂度最优。在成对人工评估中，基于2000次对比，该模型对Ovi 1.1的胜率达80.0%，对LTX 2.3的胜率达60.9%。我们开源了完整的模型技术栈，包括基础模型、蒸馏模型、超分辨率模型及推理代码库。
URL
https://arxiv.org/abs/2603.21986
PDF
-
OmniFM: Toward Modality-Robust and Task-Agnostic Federated Learning for Heterogeneous Medical Imaging
2026-03-23 07:38:08Meilin Liu, Jiaying Wang, Jing ShanAbstract
Federated learning (FL) has become a promising paradigm for collaborative medical image analysis, yet existing frameworks remain tightly coupled to task-specific backbones and are fragile under heterogeneous imaging modalities. Such constraints hinder real-world deployment, where institutions vary widely in modality distributions and must support diverse downstream tasks. To address this limitation, we propose OmniFM, a modality- and task-agnostic FL framework that unifies training across classification, segmentation, super-resolution, visual question answering, and multimodal fusion without re-engineering the optimization pipeline. OmniFM builds on a key frequency-domain insight: low-frequency spectral components exhibit strong cross-modality consistency and encode modality-invariant anatomical structures. Accordingly, OmniFM integrates (i) Global Spectral Knowledge Retrieval to inject global frequency priors, (ii) Embedding-wise Cross-Attention Fusion to align representations, and (iii) Prefix-Suffix Spectral Prompting to jointly condition global and personalized cues, together regularized by a Spectral-Proximal Alignment objective that stabilizes aggregation. Experiments on real-world datasets show that OmniFM consistently surpasses state-of-the-art FL baselines across intra- and cross-modality heterogeneity, achieving superior results under both fine-tuning and training-from-scratch setups.
Abstract (translated)
联邦学习（FL）已成为协作式医学图像分析的一种有前景范式，但现有框架仍与特定任务架构紧密耦合，且在异构成像模态条件下表现脆弱。此类限制阻碍了真实场景部署——不同机构的模态分布差异显著，且需支持多样化的下游任务。为突破此局限，我们提出OmniFM，一个模态与任务无关的联邦学习框架，可在无需重构优化流程的前提下，统一实现分类、分割、超分辨率、视觉问答及多模态融合等任务的训练。OmniFM基于一个关键频域洞察：低频谱分量呈现强跨模态一致性，并编码模态不变解剖结构。据此，OmniFM整合了（i）全局频谱知识检索以注入全局频率先验，（ii）逐嵌入跨注意力融合以对齐表征，以及（iii）前缀-后缀频谱提示以联合约束全局与个性化线索，并通过频谱近端对齐目标进行正则化以稳定聚合。在真实数据集上的实验表明，OmniFM在模态内与跨模态异构场景下持续超越最先进联邦学习基线，在微调与从头训练设置中均取得更优结果。
URL
https://arxiv.org/abs/2603.21660
PDF
-
Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability
2026-03-23 02:55:16Jiahui Song, Sagar Shrestha, Xiao FuAbstract
This paper addresses the fusion of a pair of spatially unregistered hyperspectral image (HSI) and multispectral image (MSI) covering roughly overlapping regions. HSIs offer high spectral but low spatial resolution, while MSIs provide the opposite. The goal is to integrate their complementary information to enhance both HSI spatial resolution and MSI spectral resolution. While hyperspectral-multispectral fusion (HMF) has been widely studied, the unregistered setting remains challenging. Many existing methods focus solely on MSI super-resolution, leaving HSI unchanged. Supervised deep learning approaches were proposed for HSI super-resolution, but rely on accurate training data, which is often unavailable. Moreover, theoretical analyses largely address the co-registered case, leaving unregistered HMF poorly understood. In this work, an unsupervised framework is proposed to simultaneously super-resolve both MSI and HSI. The method integrates coupled spectral unmixing for MSI super-resolution with latent-space adversarial learning for HSI super-resolution. Theoretical guarantees on the recoverability of the super-resolution MSI and HSI are established under reasonable generative models -- providing, to our best knowledge, the first such insights for unregistered HMF. The approach is validated on semi-real and real HSI-MSI pairs across diverse conditions.
Abstract (translated)
本文针对空间未配准的高光谱图像（HSI）与多光谱图像（MSI）融合问题展开研究，两者覆盖区域大致重叠。高光谱图像具有高光谱分辨率但空间分辨率低，而多光谱图像则相反。研究目标在于整合二者互补信息，以同时提升高光谱图像的空间分辨率与多光谱图像的光谱分辨率。尽管高光谱-多光谱融合（HMF）已得到广泛研究，但未配准场景仍具挑战性。现有方法多仅关注多光谱图像超分辨率，未改变高光谱图像质量。虽有监督深度学习方法被提出用于高光谱超分辨率，但其依赖难以获取的精确训练数据。此外，现有理论分析多针对配准场景，未配准条件下的融合问题尚缺乏深入理解。 本研究提出一种无监督框架，可同时对多光谱与高光谱图像进行超分辨率重建。该方法将耦合光谱解混用于多光谱超分辨率，并结合潜在空间对抗学习实现高光谱超分辨率。在合理的生成模型下，本文建立了超分辨率重建多光谱与高光谱图像可恢复性的理论保证——据我们所知，这是首个针对未配准高光谱-多光谱融合的理论分析。通过在多样条件下对半真实与真实HSI-MSI数据对进行验证，证实了该方法的有效性。
URL
https://arxiv.org/abs/2603.21510
PDF
-
LPNSR: Prior-Enhanced Diffusion Image Super-Resolution via LR-Guided Noise Prediction
2026-03-22 03:52:38Shuwei Huang, Shizhuo Liu, Zijun WeiAbstract
Diffusion-based image super-resolution (SR), which aims to reconstruct high-resolution (HR) images from corresponding low-resolution (LR) observations, faces a fundamental trade-off between inference efficiency and reconstruction quality. The state-of-the-art residual-shifting diffusion framework achieves efficient 4-step inference, yet suffers from severe performance degradation in compact sampling trajectories. This is mainly attributed to two core limitations: the inherent suboptimality of unconstrained random Gaussian noise in intermediate steps, which leads to error accumulation and insufficient LR prior guidance, and the initialization bias caused by naive bicubic upsampling. In this paper, we propose LPNSR, a prior-enhanced efficient diffusion framework to address these issues. We first mathematically derive the closed-form analytical solution of the optimal intermediate noise for the residual-shifting diffusion paradigm, and accordingly design an LR-guided multi-input-aware noise predictor to replace random Gaussian noise, embedding LR structural priors into the reverse process while fully preserving the framework's core efficient residual-shifting mechanism. We further mitigate initial bias with a high-quality pre-upsampling network to optimize the diffusion starting point. With a compact 4-step trajectory, LPNSR can be optimized in an end-to-end manner. Extensive experiments demonstrate that LPNSR achieves state-of-the-art perceptual performance on both synthetic and real-world datasets, without relying on any large-scale text-to-image priors. The source code of our method can be found at this https URL.
Abstract (translated)
基于扩散模型的图像超分辨率（SR）旨在从相应的低分辨率（LR）观测图像中重建高分辨率（HR）图像，其面临推理效率与重建质量之间的根本性权衡。最先进的残差移位扩散框架实现了高效的4步推理，但在紧凑采样轨迹中性能严重下降。这主要归因于两个核心局限：中间步骤中无约束随机高斯噪声的内在次优性导致误差累积和LR先验指导不足，以及朴素双三次上采样引起的初始化偏差。本文提出LPNSR，一种先验增强的高效扩散框架以解决这些问题。我们首先为残差移位扩散范式数学推导出最优中间噪声的闭式解析解，并据此设计LR引导的多输入感知噪声预测器以替代随机高斯噪声，将LR结构先验嵌入逆向过程，同时充分保留框架核心的高效残差移位机制。我们进一步通过高质量预上采样网络缓解初始偏差以优化扩散起点。在紧凑的4步轨迹下，LPNSR可实现端到端优化。大量实验表明，LPNSR在合成和真实世界数据集上均达到最先进的感知性能，且不依赖任何大规模文本到图像先验。我们方法的源代码可在此https URL找到。
URL
https://arxiv.org/abs/2603.21045
PDF
-
MFSR: MeanFlow Distillation for One Step Real-World Image Super Resolution
2026-03-21 07:18:01Ruiqing Wang, Kai Zhang, Yuanzhi Zhu, Hanshu Yan, Shilin Lu, Jian YangAbstract
Diffusion- and flow-based models have advanced Real-world Image Super-Resolution (Real-ISR), but their multi-step sampling makes inference slow and hard to deploy. One-step distillation alleviates the cost, yet often degrades restoration quality and removes the option to refine with more steps. We present Mean Flows for Super-Resolution (MFSR), a new distillation framework that produces photorealistic results in a single step while still allowing an optional few-step path for further improvement. Our approach uses MeanFlow as the learning target, enabling the student to approximate the average velocity between arbitrary states of the Probability Flow ODE (PF-ODE) and effectively capture the teacher's dynamics without explicit rollouts. To better leverage pretrained generative priors, we additionally improve original MeanFlow's Classifier-Free Guidance (CFG) formulation with teacher CFG distillation strategy, which enhances restoration capability and preserves fine details. Experiments on both synthetic and real-world benchmarks demonstrate that MFSR achieves efficient, flexible, and high-quality super-resolution, delivering results on par with or even better than multi-step teachers while requiring much lower computational cost.
Abstract (translated)
扩散模型与流模型推动了真实世界图像超分辨率（Real-ISR）的发展，但其多步采样过程导致推理速度缓慢且难以部署。一步蒸馏虽能降低计算成本，却常损害修复质量并丧失多步精细调整的可能性。本文提出超分辨率平均流（MFSR）这一新型蒸馏框架，既能单步生成照片级真实结果，又保留可选的多步优化路径以实现进一步提升。该方法以平均流作为学习目标，使学生模型能够逼近概率流常微分方程（PF-ODE）任意状态间的平均速度，无需显式展开即可有效捕捉教师模型动态。为更好利用预训练生成先验，我们进一步改进原始平均流的无分类器引导（CFG）公式，通过教师CFG蒸馏策略增强修复能力并保留精细细节。在合成与真实世界基准测试中，MFSR均展现出高效、灵活且高质量的超分辨率性能，其效果与多步教师模型相当甚至更优，而计算成本显著降低。
URL
https://arxiv.org/abs/2603.20690
PDF
-
Interpretable Operator Learning for Inverse Problems via Adaptive Spectral Filtering: Convergence and Discretization Invariance
2026-03-21 02:19:19Hang-Cheng Dong, Pengcheng Cheng, Shuhuan LiAbstract
Solving ill-posed inverse problems necessitates effective regularization strategies to stabilize the inversion process against measurement noise. While classical methods like Tikhonov regularization require heuristic parameter tuning, and standard deep learning approaches often lack interpretability and generalization across resolutions, we propose SC-Net (Spectral Correction Network), a novel operator learning framework. SC-Net operates in the spectral domain of the forward operator, learning a pointwise adaptive filter function that reweights spectral coefficients based on the signal-to-noise ratio. We provide a theoretical analysis showing that SC-Net approximates the continuous inverse operator, guaranteeing discretization invariance. Numerical experiments on 1D integral equations demonstrate that SC-Net: (1) achieves the theoretical minimax optimal convergence rate ($O(\delta^{0.5})$ for $s=p=1.5$), matching theoretical lower bounds; (2) learns interpretable sharp-cutoff filters that outperform Oracle Tikhonov regularization; and (3) exhibits zero-shot super-resolution, maintaining stable reconstruction errors ($\approx 0.23$) when trained on coarse grids ($N=256$) and tested on significantly finer grids (up to $N=2048$). The proposed method bridges the gap between rigorous regularization theory and data-driven operator learning.
Abstract (translated)
求解病态反问题需要有效的正则化策略，以稳定反演过程对测量噪声的鲁棒性。尽管经典方法（如吉洪诺夫正则化）依赖启发式参数调优，而标准深度学习方法常缺乏可解释性及跨分辨率泛化能力，我们提出了SC-Net（谱校正网络），一种新颖的算子学习框架。SC-Net在前向算子的谱域中运行，学习逐点自适应滤波函数，依据信噪比重新加权谱系数。我们提供理论分析表明SC-Net逼近连续反演算子，保证离散化不变性。在一维积分方程上的数值实验证明SC-Net：（1）达到理论极小极大最优收敛速率（当s=p=1.5时为O(δ^0.5)），匹配理论下界；（2）学习可解释的锐截止滤波器，性能超越最优吉洪诺夫正则化；（3）展现零样本超分辨率能力，在粗网格（N=256）上训练后，于显著更细网格（最高N=2048）测试时仍保持稳定重建误差（约0.23）。该方法弥合了严格正则化理论与数据驱动算子学习之间的差距。
URL
https://arxiv.org/abs/2603.20602
PDF
-
Generalizable NGP-SR: Generalizable Neural Radiance Fields Super-Resolution via Neural Graph Primitives
2026-03-20 16:50:16Wanqi Yuan, Omkar Sharad Mayekar, Connor Pennington, Nianyi LiAbstract
Neural Radiance Fields (NeRF) achieve photorealistic novel view synthesis but become costly when high-resolution (HR) rendering is required, as HR outputs demand dense sampling and higher-capacity models. Moreover, naively super-resolving per-view renderings in 2D often breaks multi-view consistency. We propose Generalizable NGP-SR, a 3D-aware super-resolution framework that reconstructs an HR radiance field directly from low-resolution (LR) posed images. Built on Neural Graphics Primitives (NGP), NGP-SR conditions radiance prediction on 3D coordinates and learned local texture tokens, enabling recovery of high-frequency details within the radiance field and producing view-consistent HR novel views without external HR references or post-hoc 2D upsampling. Importantly, our model is generalizable: once trained, it can be applied to unseen scenes and rendered from novel viewpoints without per-scene optimization. Experiments on multiple datasets show that NGP-SR consistently improves both reconstruction quality and runtime efficiency over prior NeRF-based super-resolution methods, offering a practical solution for scalable high-resolution novel view synthesis.
Abstract (translated)
神经辐射场（NeRF）可实现照片级真实感的新视角合成，但当需要高分辨率（HR）渲染时，其计算成本高昂，因为高分辨率输出需要密集采样和更高容量的模型。此外，简单地以2D方式对单视角渲染结果进行超分辨率处理，往往会破坏多视角一致性。我们提出了通用型NGP-SR，这是一个三维感知的超分辨率框架，可直接从低分辨率（LR）的位姿图像中重建高分辨率辐射场。基于神经图形基元（NGP），NGP-SR将辐射预测条件化于三维坐标与学习得到的局部纹理标记，从而在辐射场内恢复高频细节，并生成视角一致的高分辨率新视图，而无需外部高分辨率参考或事后的2D上采样。重要的是，我们的模型具有通用性：训练完成后，可应用于未见场景并从新视角进行渲染，无需针对每个场景进行优化。在多个数据集上的实验表明，NGP-SR在重建质量和运行效率上均持续优于先前的基于NeRF的超分辨率方法，为可扩展的高分辨率新视角合成提供了实用解决方案。
URL
https://arxiv.org/abs/2603.20128
PDF
-
CAFlow: Adaptive-Depth Single-Step Flow Matching for Efficient Histopathology Super-Resolution
2026-03-19 05:45:58Elad Yoshai, Ariel D. Yoshai, Natan T. ShakedAbstract
In digital pathology, whole-slide images routinely exceed gigapixel resolution, making computationally intensive generative super-resolution (SR) impractical for routine deployment. We introduce CAFlow, an adaptive-depth single-step flow-matching framework that routes each image tile to the shallowest network exit that preserves reconstruction quality. CAFlow performs flow matching in pixel-unshuffled rearranged space, reducing spatial computation by 16x while enabling direct inference. We show that dedicating half of training to exact t=0 samples is essential for single-step quality (-1.5 dB without it). The backbone, FlowResNet (1.90M parameters), mixes convolution and window self-attention blocks across four early exits spanning 3.1 to 13.3 GFLOPs. A lightweight exit classifier (~6K parameters) achieves 33% compute savings at only 0.12 dB cost. On multi-organ histopathology x4 SR, adaptive routing achieves 31.72 dB PSNR versus 31.84 dB at full depth, while the shallowest exit exceeds bicubic by +1.9 dB at 2.8x less compute than SwinIR-light. The method generalizes to held-out colon tissue with minimal quality loss (-0.02 dB), and at x8 upscaling it outperforms all comparable-compute baselines while remaining competitive with the much larger SwinIR-Medium model. Downstream nuclei segmentation confirms preservation of clinically relevant structure. The model trains in under 5 hours on a single GPU, and adaptive routing can reduce whole-slide inference from minutes to seconds.
Abstract (translated)
URL
https://arxiv.org/abs/2603.18513
PDF
-
Unrolled Reconstruction with Integrated Super-Resolution for Accelerated 3D LGE MRI
2026-03-18 21:48:48Md Hasibul Husain Hisham, Shireen Elhabian, Ganesh Adluru, Jason Mendes, Andrew Arai, Eugene Kholmovski, Ravi Ranjan, Edward DiBellaAbstract
Accelerated 3D late gadolinium enhancement (LGE) MRI requires robust reconstruction methods to recover thin atrial structures from undersampled k-space data. While unrolled model-based networks effectively integrate physics-driven data consistency with learned priors, they operate at the acquired resolution and may fail to fully recover high-frequency detail. We propose a hybrid unrolled reconstruction framework in which an Enhanced Deep Super-Resolution (EDSR) network replaces the proximal operator within each iteration of the optimization loop, enabling joint super-resolution enhancement and data consistency enforcement. The model is trained end-to-end on retrospectively undersampled preclinical 3D LGE datasets and compared against compressed sensing, Model-Based Deep Learning (MoDL), and self-guided Deep Image Prior (DIP) baselines. Across acceleration factors, the proposed method consistently improves PSNR and SSIM over standard unrolled reconstruction and better preserves fine cardiac structures, leading to improved LA (left atrium) segmentation performance. These results demonstrate that integrating super-resolution priors directly within model-based reconstruction provides measurable gains in accelerated 3D LGE MRI.
Abstract (translated)
URL
https://arxiv.org/abs/2603.18309
PDF
-
ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation
2026-03-18 15:04:57Dmitriy Rivkin, Parker Ewen, Lili Gao, Julian Ost, Stefanie Walz, Rasika Kangutkar, Mario Bijelic, Felix HeideAbstract
Recent video diffusion models achieve high-quality generation through recurrent frame processing where each frame generation depends on previous frames. However, this recurrent mechanism means that training such models in the pixel domain incurs prohibitive memory costs, as activations accumulate across the entire video sequence. This fundamental limitation also makes fine-tuning these models with pixel-wise losses computationally intractable for long or high-resolution videos. This paper introduces ChopGrad, a truncated backpropagation scheme for video decoding, limiting gradient computation to local frame windows while maintaining global consistency. We provide a theoretical analysis of this approximation and show that it enables efficient fine-tuning with frame-wise losses. ChopGrad reduces training memory from scaling linearly with the number of video frames (full backpropagation) to constant memory, and compares favorably to existing state-of-the-art video diffusion models across a suite of conditional video generation tasks with pixel-wise losses, including video super-resolution, video inpainting, video enhancement of neural-rendered scenes, and controlled driving video generation.
Abstract (translated)
URL
https://arxiv.org/abs/2603.17812
PDF
-
Joint Degradation-Aware Arbitrary-Scale Super-Resolution for Variable-Rate Extreme Image Compression
2026-03-18 06:35:53Xinning Chai, Zhengxue Cheng, Xin Li, Rong Xie, Li SongAbstract
Recent diffusion-based extreme image compression methods have demonstrated remarkable performance at ultra-low bitrates. However, most approaches require training separate diffusion models for each target bitrate, resulting in substantial computational overhead and hindering practical deployment. Meanwhile, recent studies have shown that joint super-resolution can serve as an effective approach for enhancing low-bitrate reconstruction. However, when moving toward ultra-low bitrate regimes, these methods struggle due to severe information loss, and their reliance on fixed super-resolution scales prevents flexible adaptation across diverse bitrates. To address these limitations, we propose ASSR-EIC, a novel image compression framework that leverages arbitrary-scale super-resolution (ASSR) to support variable-rate extreme image compression (EIC). An arbitrary-scale downsampling module is introduced at the encoder side to provide controllable rate reduction, while a diffusion-based, joint degradation-aware ASSR decoder enables rate-adaptive reconstruction within a single model. We exploit the compression- and rescaling-aware diffusion prior to guide the reconstruction, yielding high fidelity and high realism restoration across diverse compression and rescaling settings. Specifically, we design a global compression-rescaling adaptor that offers holistic guidance for rate adaptation, and a local compression-rescaling modulator that dynamically balances generative and fidelity-oriented behaviors to achieve fine-grained, bitrate-adaptive detail restoration. To further enhance reconstruction quality, we introduce a dual semantic-enhanced design. Extensive experiments demonstrate that ASSR-EIC delivers state-of-the-art performance in extreme image compression while simultaneously supporting flexible bitrate control and adaptive rate-dependent reconstruction.
Abstract (translated)
URL
https://arxiv.org/abs/2603.17408
PDF
