Abstract
Diffusion models have achieved remarkable success in image synthesis. However, addressing artifacts and unrealistic regions remains a critical challenge. We propose self-refining diffusion, a novel framework that enhances image generation quality by detecting these flaws. The framework employs an explainable artificial intelligence (XAI)-based flaw highlighter to produce flaw activation maps (FAMs) that identify artifacts and unrealistic regions. These FAMs improve reconstruction quality by amplifying noise in flawed regions during the forward process and by focusing on these regions during the reverse process. The proposed approach achieves up to a 27.3% improvement in Fréchet inception distance across various diffusion-based models, demonstrating consistently strong performance on diverse datasets. It also shows robust effectiveness across different tasks, including image generation, text-to-image generation, and inpainting. These results demonstrate that explainable AI techniques can extend beyond interpretability to actively contribute to image refinement. The proposed framework offers a versatile and effective approach applicable to various diffusion models and tasks, significantly advancing the field of image synthesis.
Abstract (translated)
扩散模型在图像生成方面取得了显著的成功，但解决伪影和不真实区域的问题仍然是一个关键挑战。我们提出了一种名为“自我完善扩散”的新型框架，该框架通过检测这些缺陷来提升图像生成的质量。此框架使用基于可解释人工智能（XAI）的瑕疵高亮器来生成瑕疵激活图（FAMs），以识别伪影和不真实区域。在正向过程中，这些FAMs通过放大有缺陷区域中的噪声来改进重建质量；而在反向过程中，则通过聚焦于这些特定区域来进行优化。 所提出的方法在各种基于扩散的模型上实现了高达27.3%的Fréchet inception距离（FID）改善，并且在不同的数据集上均表现出强大的性能。此外，这种方法还显示出对不同任务的强大效果，包括图像生成、文本到图像生成以及修复画笔工具应用等。 这些结果表明，可解释的人工智能技术不仅可以用于提高模型的透明度和理解性，还可以主动参与图像精炼过程。所提出的框架提供了一种适用于各种扩散模型和任务的灵活有效的方法，从而在图像合成领域取得了显著进展。
URL
https://arxiv.org/abs/2512.08774
