Abstract
Deep learning models have shown remarkable success in dermatological image analysis, offering potential for automated skin disease diagnosis. Previously, convolutional neural network(CNN) based architectures have achieved immense popularity and success in computer vision (CV) based task like skin image recognition, generation and video analysis. But with the emergence of transformer based models, CV tasks are now are nowadays carrying out using these models. Vision Transformers (ViTs) is such a transformer-based models that have shown success in computer vision. It uses self-attention mechanisms to achieve state-of-the-art performance across various tasks. However, their reliance on global attention mechanisms makes them susceptible to adversarial perturbations. This paper aims to investigate the susceptibility of ViTs for medical images to adversarial watermarking-a method that adds so-called imperceptible perturbations in order to fool models. By generating adversarial watermarks through Projected Gradient Descent (PGD), we examine the transferability of such attacks to CNNs and analyze the performance defense mechanism -- adversarial training. Results indicate that while performance is not compromised for clean images, ViTs certainly become much more vulnerable to adversarial attacks: an accuracy drop of as low as 27.6%. Nevertheless, adversarial training raises it up to 90.0%.
Abstract (translated)
深度学习模型在皮肤病图像分析中表现出显著的成功，为自动化皮肤疾病诊断提供了潜在可能。此前，基于卷积神经网络（CNN）的架构在计算机视觉（CV）任务如皮肤图像识别、生成和视频分析方面取得了巨大的流行度和成功。然而，随着基于变换器（Transformer）模型的出现，如今许多CV任务正在使用这些模型来完成。Vision Transformers (ViTs) 是一种基于变换器的模型，在计算机视觉领域表现出色。它通过自注意力机制在各种任务中达到了最先进的性能。但是，它们依赖于全局注意机制使得其容易受到对抗性扰动的影响。 本文旨在研究医学图像中的ViT对对抗性水印（即添加所谓的不显眼干扰以误导模型的方法）的敏感性。通过使用Projected Gradient Descent (PGD) 生成对抗性水印，我们考察了此类攻击在CNN上的转移性和分析防御机制——对抗训练的表现。 结果表明，在处理干净图像时性能并未受损，但ViT确实对对抗性攻击更加脆弱：准确率下降至低至27.6%。然而，通过对抗训练可以将其提升到90.0%。
URL
https://arxiv.org/abs/2506.06389
