Abstract
Multimodal medical image fusion facilitates comprehensive diagnosis by aggregating complementary structural and functional information, but its effectiveness is limited by resolution degradation and modality discrepancies. Existing approaches typically perform image fusion and super-resolution (SR) in separate stages, leading to artifacts and degraded perceptual quality. These limitations are further amplified in tri-modal settings that combine anatomical modalities (e.g., MRI, CT) with functional scans (e.g., PET, SPECT) due to pronounced frequency domain imbalances. We propose TriFusionSR, a wavelet-guided conditional diffusion framework for joint tri-modal fusion and SR. The framework explicitly decomposes multimodal features into frequency bands using the 2D Discrete Wavelet Transform, enabling frequency-aware crossmodal interaction. We further introduce a Rectified Wavelet Features (RWF) strategy for latent coefficient calibration, followed by an Adaptive Spatial-Frequency Fusion (ASFF) module with gated channel-spatial attention to enable structure-driven multimodal refinement. Extensive experiments demonstrate state-of-the-art performance, achieving 4.8-12.4% PSNR improvement and substantial reductions in RMSE and LPIPS across multiple upsampling scales.
Abstract (translated)
多模态医学图像融合通过聚合互补的结构和功能信息，促进了全面诊断。然而，其有效性受到分辨率退化和模式差异的限制。现有方法通常在单独阶段执行图像融合和超分辨率(SR)，这导致了伪影和感知质量下降。在结合解剖学模式（如MRI、CT）与功能性扫描（如PET、SPECT）的三模态设置中，这些局限性由于频域不平衡而进一步加剧。 我们提出了TriFusionSR，这是一种基于小波指导的条件扩散框架，用于联合执行三模态融合和超分辨率。该框架明确地使用二维离散小波变换将多模式特征分解为频率带，从而实现频率感知的跨模态交互。我们还引入了一种修正小波特征（RWF）策略，用于潜在系数校准，并随后采用了一个带有门控通道空间注意机制的自适应空域频域融合（ASFF）模块，以实现结构驱动的多模式细化。 广泛的实验展示了该方法在多个上采样尺度下的优越性能，实现了PSNR 4.8-12.4%的提升，并大幅减少了RMSE和LPIPS误差。
URL
https://arxiv.org/abs/2603.09702
