Abstract
Image reconstruction and image synthesis are important for handling incomplete multimodal imaging data, but existing methods require various task-specific models, complicating training and deployment workflows. We introduce Any2all, a unified framework that addresses this limitation by formulating these disparate tasks as a single virtual inpainting problem. We train a single, unconditional diffusion model on the complete multimodal data stack. This model is then adapted at inference time to ``inpaint'' all target modalities from any combination of inputs of available clean images or noisy measurements. We validated Any2all on a PET/MR/CT brain dataset. Our results show that Any2all can achieve excellent performance on both multimodal reconstruction and synthesis tasks, consistently yielding images with competitive distortion-based performance and superior perceptual quality over specialized methods.
Abstract (translated)
图像重建和图像合成对于处理不完整的多模态成像数据非常重要，但现有的方法需要各种特定任务的模型，从而增加了训练和部署流程的复杂性。我们介绍了Any2all，这是一个统一框架，通过将这些不同的任务表述为单一虚拟修复问题来解决这一局限性。我们在完整的多模态数据堆栈上训练一个无条件扩散模型。在推理时，这个模型被调整以从任何组合的可用清晰图像或噪声测量值中“修补”所有目标模式。我们使用PET/MR/CT脑部数据集验证了Any2all的有效性。我们的结果显示，Any2all在多模态重建和合成任务上均能取得出色性能，在基于失真的性能方面与专业方法相当，并且感知质量更优。
URL
https://arxiv.org/abs/2602.08249
