Abstract
Transformation-based adversarial attacks (TAAs) demonstrate strong transferability when deceiving classification models. However, existing TAAs often perform unsatisfactorily or even fail when applied to structured tasks such as semantic segmentation and object detection. Encouragingly, recent studies that categorize transformations into non-spatial and spatial transformations inspire us to address this challenge. We find that for non-structured tasks, labels are spatially non-structured, and thus TAAs are not required to adjust labels when applying spatial transformations. In contrast, for structured tasks, labels are spatially structured, and failing to transform labels synchronously with inputs can cause spatial misalignment and yield erroneous gradients. To address these issues, we propose a novel unified Spatial Alignment Framework (SAF) for highly transferable TAAs on spatially structured tasks, where the TAAs spatially transform labels synchronously with the input using the proposed Spatial Alignment (SA) algorithm. Extensive experiments demonstrate the crucial role of our SAF for TAAs on structured tasks. Specifically, in non-targeted attacks, our SAF degrades the average mIoU on Cityscapes from 24.50 to 11.34, and on Kvasir-SEG from 49.91 to 31.80, while reducing the average mAP of COCO from 17.89 to 5.25.
Abstract (translated)
基于变换的对抗攻击（TAAs）在欺骗分类模型时展现出强迁移性。然而，当应用于语义分割和目标检测等结构化任务时，现有TAAs往往表现不佳甚至失效。令人鼓舞的是，近期研究将变换分为非空间变换与空间变换的分类方式启发了我们应对这一挑战。我们发现：对于非结构化任务，其标签在空间上呈非结构化分布，因此TAAs在应用空间变换时无需调整标签；而对于结构化任务，标签具有空间结构性，若不同步变换标签与输入会导致空间错位并产生错误梯度。针对这些问题，我们提出了一种新颖的统一空间对齐框架（SAF），用于在空间结构化任务中实现高迁移性TAAs，其中TAAs通过所提出的空间对齐（SA）算法同步变换标签与输入。大量实验验证了SAF对结构化任务中TAAs的关键作用。具体而言，在非目标攻击中，SAF使Cityscapes数据集上的平均mIoU从24.50降至11.34，Kvasir-SEG数据集上从49.91降至31.80，同时将COCO数据集的平均mAP从17.89降至5.25。
URL
https://arxiv.org/abs/2603.25230
