Abstract
Diffusion Models (DMs) have evolved into advanced image generation tools, especially for few-shot generation where a pretrained model is fine-tuned on a small set of images to capture a specific style or object. Despite their success, concerns exist about potential copyright violations stemming from the use of unauthorized data in this process. In response, we present Contrasting Gradient Inversion for Diffusion Models (CGI-DM), a novel method featuring vivid visual representations for digital copyright authentication. Our approach involves removing partial information of an image and recovering missing details by exploiting conceptual differences between the pretrained and fine-tuned models. We formulate the differences as KL divergence between latent variables of the two models when given the same input image, which can be maximized through Monte Carlo sampling and Projected Gradient Descent (PGD). The similarity between original and recovered images serves as a strong indicator of potential infringements. Extensive experiments on the WikiArt and Dreambooth datasets demonstrate the high accuracy of CGI-DM in digital copyright authentication, surpassing alternative validation techniques. Code implementation is available at this https URL.
Abstract (translated)
扩散模型（DMs）已经发展成为先进的图像生成工具，尤其是在几 shot 生成中，使用预训练模型在少量图像上微调以捕捉特定风格或对象。尽管取得了成功，但担心在这个过程中使用未经授权的数据可能引发版权侵犯。为了应对这个问题，我们提出了对比梯度反演方法（CGI-DM），一种新型的方法，为数字版权认证提供了生动的视觉表示。我们的方法涉及移除图像的部分信息并通过利用预训练模型和微调模型之间的概念差异来恢复缺失细节。我们将差异表示为给定相同输入图像的两个模型的潜在变量之间的 KL 散度，这可以通过蒙特卡洛采样和拟合梯度下降（PGD）来最大化。原始和恢复图像之间的相似性作为潜在侵权的强有力指标。在维基艺术和 Dreambooth 数据集上的大量实验证明，CGI-DM 在数字版权认证方面的准确性非常高，超过了其他验证技术。代码实现可通过此链接https://www.researchgate.net/publication/327513661_Contrasting_Gradient_Inversion_for_Diffusion_Models_CGI-DM_a_ novel_method_for_digital_copyright_authentication_with_ vivid_visual_representations_for_copyright_verification feel free to use.
URL
https://arxiv.org/abs/2403.11162
