Abstract
Image harmonization, which involves adjusting the foreground of a composite image to attain a unified visual consistency with the background, can be conceptualized as an image-to-image translation task. Diffusion models have recently promoted the rapid development of image-to-image translation tasks . However, training diffusion models from scratch is computationally intensive. Fine-tuning pre-trained latent diffusion models entails dealing with the reconstruction error induced by the image compression autoencoder, making it unsuitable for image generation tasks that involve pixel-level evaluation metrics. To deal with these issues, in this paper, we first adapt a pre-trained latent diffusion model to the image harmonization task to generate the harmonious but potentially blurry initial images. Then we implement two strategies: utilizing higher-resolution images during inference and incorporating an additional refinement stage, to further enhance the clarity of the initially harmonized images. Extensive experiments on iHarmony4 datasets demonstrate the superiority of our proposed method. The code and model will be made publicly available at this https URL .
Abstract (translated)
图像和谐，涉及调整合成图像的前景以实现与背景的统一视觉一致性，可以概念化为一个图像到图像的映射任务。最近，扩散模型推动了许多图像到图像映射任务的快速发展。然而，从零开始训练扩散模型计算量很大。对预训练的 latent 扩散模型的微调涉及处理图像压缩自动加密器引起的图像重建误差，这使得它不适合用于需要像素级评估指标的图像生成任务。为解决这些问题，本文首先将预训练的 latent 扩散模型适应到图像和谐任务中，生成和谐但可能有点模糊的初始图像。然后我们实现了两种策略：在推理过程中使用高分辨率图像，并包含一个额外的细化阶段，以进一步增强最初和谐图像的清晰度。在 iHarmony4 数据集上进行的大量实验证明了我们提出的方法的优越性。代码和模型将公开发布在本文的链接 URL 上。
URL
https://arxiv.org/abs/2404.06139
