Abstract
Due to great success of transformers in many applications such as natural language processing and computer vision, transformers have been successfully applied in automatic modulation classification. We have shown that transformer-based radio signal classification is vulnerable to imperceptible and carefully crafted attacks called adversarial examples. Therefore, we propose a defense system against adversarial examples in transformer-based modulation classifications. Considering the need for computationally efficient architecture particularly for Internet of Things (IoT)-based applications or operation of devices in environment where power supply is limited, we propose a compact transformer for modulation classification. The advantages of robust training such as adversarial training in transformers may not be attainable in compact transformers. By demonstrating this, we propose a novel compact transformer that can enhance robustness in the presence of adversarial attacks. The new method is aimed at transferring the adversarial attention map from the robustly trained large transformer to a compact transformer. The proposed method outperforms the state-of-the-art techniques for the considered white-box scenarios including fast gradient method and projected gradient descent attacks. We have provided reasoning of the underlying working mechanisms and investigated the transferability of the adversarial examples between different architectures. The proposed method has the potential to protect the transformer from the transferability of adversarial examples.
Abstract (translated)
由于Transformer在自然语言处理和计算机视觉等众多应用中的巨大成功，它们已经被成功应用于自动调制分类中。我们已经证明了基于Transformer的无线电信号分类对精心设计且难以察觉的对抗性样本攻击非常脆弱。因此，我们提出了一种针对基于Transformer的调制分类中对抗性示例的防御系统。考虑到计算效率架构的需求，特别是在物联网（IoT）应用或在电力供应有限的环境中操作设备的情况下，我们提出了一个用于调制分类的小型化变压器。然而，在小型化变压器中可能无法实现增强鲁棒性的方法，例如基于Transformer中的对抗训练。通过证明这一点，我们提出了一种新的紧凑型变压器，可以在存在对抗性攻击时提高其健壮性。新方法旨在将来自稳健大型Transformer的对抗注意力图转移到紧凑型Transformer中。在考虑的白盒场景（包括快速梯度法和投影梯度下降攻击）下，所提方法优于现有的最先进技术。我们提供了该方法背后的工作机制的理由，并探讨了不同架构之间对抗样本的可迁移性。所提出的方法有望保护变压器免受对抗样本传输的影响。
URL
https://arxiv.org/abs/2506.11892
