Abstract
Existing inpainting methods often require extensive retraining or fine-tuning to integrate new content seamlessly, yet they struggle to maintain coherence in both structure and style between inpainted regions and the surrounding background. Motivated by these limitations, we introduce HarmonPaint, a training-free inpainting framework that seamlessly integrates with the attention mechanisms of diffusion models to achieve high-quality, harmonized image inpainting without any form of training. By leveraging masking strategies within self-attention, HarmonPaint ensures structural fidelity without model retraining or fine-tuning. Additionally, we exploit intrinsic diffusion model properties to transfer style information from unmasked to masked regions, achieving a harmonious integration of styles. Extensive experiments demonstrate the effectiveness of HarmonPaint across diverse scenes and styles, validating its versatility and performance.
Abstract (translated)
现有的图像修复方法通常需要大量的重新训练或微调，才能将新的内容无缝地整合进来，但它们在保持修补区域与周围背景之间结构和风格的一致性方面存在困难。鉴于这些限制，我们引入了HarmonPaint，这是一个无需训练的图像修复框架，它通过利用扩散模型中的注意力机制来实现高质量且一致的图像修复。HarmonPaint通过在自注意力中使用掩码策略，在不进行任何重新训练或微调的情况下保证结构的一致性。此外，该方法还利用扩散模型的内在属性将风格信息从未被遮罩的部分传递到被遮罩的部分，从而实现了风格的一致融合。广泛的实验验证了HarmonPaint在各种场景和风格中的有效性和通用性。
URL
https://arxiv.org/abs/2507.16732
