Abstract
In Virtual Product Placement (VPP) applications, the discrete integration of specific brand products into images or videos has emerged as a challenging yet important task. This paper introduces a novel three-stage fully automated VPP system. In the first stage, a language-guided image segmentation model identifies optimal regions within images for product inpainting. In the second stage, Stable Diffusion (SD), fine-tuned with a few example product images, is used to inpaint the product into the previously identified candidate regions. The final stage introduces an "Alignment Module", which is designed to effectively sieve out low-quality images. Comprehensive experiments demonstrate that the Alignment Module ensures the presence of the intended product in every generated image and enhances the average quality of images by 35%. The results presented in this paper demonstrate the effectiveness of the proposed VPP system, which holds significant potential for transforming the landscape of virtual advertising and marketing strategies.
Abstract (translated)
在虚拟产品置入（VPP）应用中，将特定品牌产品离散地集成到图像或视频中已成为一个具有挑战性但重要的问题。本文介绍了一种新颖的三阶段完全自动VPP系统。在第一阶段，受语言指导的图像分割模型在图像中确定产品修复的最佳区域。在第二阶段，使用经过几例产品图像微调的Stable Diffusion（SD）对产品进行修复，将产品修复到之前确定的候选区域中。最后阶段引入了一个“对齐模块”，旨在有效地筛选出低质量的图像。全面的实验结果表明，对齐模块确保了每个生成的图像中都含有意图产品，并提高了图像的平均质量35%。本文所呈现的结果证明了所提出的VPP系统的有效性，该系统在改变虚拟广告和营销策略的地图方面具有巨大的潜力。
URL
https://arxiv.org/abs/2405.01130
