Abstract
Current machine learning models achieve super-human performance in many real-world applications. Still, they are susceptible against imperceptible adversarial perturbations. The most effective solution for this problem is adversarial training that trains the model with adversarially perturbed samples instead of original ones. Various methods have been developed over recent years to improve adversarial training such as data augmentation or modifying training attacks. In this work, we examine the same problem from a new data-centric perspective. For this purpose, we first demonstrate that the existing model-based methods can be equivalent to applying smaller perturbation or optimization weights to the hard training examples. By using this finding, we propose detecting and removing these hard samples directly from the training procedure rather than applying complicated algorithms to mitigate their effects. For detection, we use maximum softmax probability as an effective method in out-of-distribution detection since we can consider the hard samples as the out-of-distribution samples for the whole data distribution. Our results on SVHN and CIFAR-10 datasets show the effectiveness of this method in improving the adversarial training without adding too much computational cost.
Abstract (translated)
当前机器学习模型在许多实际应用程序中实现了人类般的表现。然而,它们仍然容易被微小的dversarial干扰影响。解决这个问题的最有效方法是dversarial训练,使用dversarially扰动样本而不是原始样本来训练模型。近年来,开发了许多方法来改进dversarial训练,例如数据增强或修改训练攻击。在本研究中,我们从一个数据为中心的新视角审视了相同的问题。为此,我们首先证明了现有基于模型的方法可以与较小的扰动或优化权重应用于困难的训练示例相当。通过利用这一发现,我们提议直接在训练过程中检测并删除这些困难的样本,而不是应用复杂的算法来减轻它们的影响。为了防止检测,我们使用softmax最大概率作为非分布检测的有效方法,因为我们可以将其视为整个数据分布的非分布样本。我们在SVO和CIFAR-10数据集上的结果表明,这种方法可以提高dversarial训练,而不必增加过多的计算成本。
URL
https://arxiv.org/abs/2301.10454
