Abstract
The rapid advancement of diffusion models has enhanced their image inpainting and editing capabilities but also introduced significant societal risks. Adversaries can exploit user images from social media to generate misleading or harmful content. While adversarial perturbations can disrupt inpainting, global perturbation-based methods fail in mask-guided editing tasks due to spatial constraints. To address these challenges, we propose Structure Disruption Attack (SDA), a powerful protection framework for safeguarding sensitive image regions against inpainting-based editing. Building upon the contour-focused nature of self-attention mechanisms of diffusion models, SDA optimizes perturbations by disrupting queries in self-attention during the initial denoising step to destroy the contour generation process. This targeted interference directly disrupts the structural generation capability of diffusion models, effectively preventing them from producing coherent images. We validate our motivation through visualization techniques and extensive experiments on public datasets, demonstrating that SDA achieves state-of-the-art (SOTA) protection performance while maintaining strong robustness.
Abstract (translated)
扩散模型的迅速发展增强了其图像修复和编辑能力，但也带来了重要的社会风险。对手可以利用社交媒体上的用户图片生成误导性或有害内容。虽然对抗性扰动能够干扰图像修复过程，但基于全局扰动的方法在引导式编辑任务中由于空间约束而失效。为了解决这些挑战，我们提出了结构破坏攻击（SDA），这是一种强大的保护框架，用于防御扩散模型对敏感图像区域进行修复式编辑的威胁。基于扩散模型中的自注意力机制以轮廓为重点的特点，SDA通过在初始去噪步骤中干扰自注意力中的查询来优化扰动，从而破坏轮廓生成过程。这种有针对性的干扰直接中断了扩散模型的结构生成能力，有效防止其产生连贯图像。我们通过可视化技术和在公开数据集上的广泛实验验证了我们的动机，证明SDA在保持强大鲁棒性的同时达到了最先进的（SOTA）保护性能。
URL
https://arxiv.org/abs/2505.19425
