Abstract
Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at this https URL.
Abstract (translated)
医学图像恢复（MedIR）的目标是从低质量的医学图像中恢复出高质量的图像。最近，MedIR领域的发展集中在能够同时处理多种不同任务的全功能模型上。然而，由于不同的成像模态和退化类型之间存在显著差异，使用同一共享模型来应对这些多样化任务需要仔细考虑两个关键的任务间关系：即任务干扰（当不同任务在同一参数上的梯度更新方向发生冲突时产生）以及任务不平衡（由各个任务固有的学习难度不均造成的优化不平衡）。为了解决这些问题，我们提出了一种任务自适应的变换器（TAT），这是一种通过两种创新方法动态调整以应对不同任务的新框架。首先，引入了任务自适应权重生成策略来减轻任务干扰，该策略针对每个任务生成特定的任务权重参数，从而消除了共享权重参数上的潜在梯度冲突。其次，我们提出了一个任务自适应损失平衡策略，根据各个任务的具体学习难度动态调整损失权重，防止任务的主导或欠训练现象。 通过广泛的实验表明，在PET合成、CT去噪和MRI超分辨率这三种MedIR任务中，我们的TAT框架在特定任务设置以及全功能模型设置下均达到了最先进的性能水平。代码可在提供的链接处获取。
URL
https://arxiv.org/abs/2512.14550
