Abstract
The success of diffusion models has enabled effortless, high-quality image modifications that precisely align with users' intentions, thereby raising concerns about their potential misuse by malicious actors. Previous studies have attempted to mitigate such misuse through adversarial attacks. However, these approaches heavily rely on image-level inconsistencies, which pose fundamental limitations in addressing the influence of textual prompts. In this paper, we propose PromptFlare, a novel adversarial protection method designed to protect images from malicious modifications facilitated by diffusion-based inpainting models. Our approach leverages the cross-attention mechanism to exploit the intrinsic properties of prompt embeddings. Specifically, we identify and target shared token of prompts that is invariant and semantically uninformative, injecting adversarial noise to suppress the sampling process. The injected noise acts as a cross-attention decoy, diverting the model's focus away from meaningful prompt-image alignments and thereby neutralizing the effect of prompt. Extensive experiments on the EditBench dataset demonstrate that our method achieves state-of-the-art performance across various metrics while significantly reducing computational overhead and GPU memory usage. These findings highlight PromptFlare as a robust and efficient protection against unauthorized image manipulations. The code is available at this https URL.
Abstract (translated)
扩散模型的成功使得高质量的图像修改变得轻松且与用户意图高度一致，这引发了对这些技术可能被恶意行为者滥用的关注。以往的研究试图通过对抗攻击来缓解此类问题。然而，这些方法严重依赖于基于图像级别的不一致性，从而在解决文本提示影响方面存在根本性的局限性。为此，在本文中我们提出了 PromptFlare，这是一种新型的对抗防护方法，旨在保护图像免受扩散模型驱动的修补程序带来的恶意修改。 我们的方法利用了跨注意力机制来挖掘提示嵌入的基本属性。具体来说，我们将识别并针对那些不变且语义上无信息的共享提示词注入对抗性噪声，以抑制采样过程。这种注入的噪音作为跨注意力诱饵，使模型的关注点偏离有意义的提示-图像对齐，并从而削弱了提示的作用。 在 EditBench 数据集上的广泛实验表明，我们的方法在各种指标下取得了最先进的性能，同时大幅减少了计算开销和 GPU 内存使用量。这些发现突显了 PromptFlare 是一种针对未经授权的图像操纵的强大且高效的保护措施。代码可在此网址获得：[此URL链接]
URL
https://arxiv.org/abs/2508.16217
