Abstract
Adversarial training suffers from the issue of robust overfitting, which seriously impairs its generalization performance. Data augmentation, which is effective at preventing overfitting in standard training, has been observed by many previous works to be ineffective in mitigating overfitting in adversarial training. This work proves that, contrary to previous findings, data augmentation alone can significantly boost accuracy and robustness in adversarial training. We find that the hardness and the diversity of data augmentation are important factors in combating robust overfitting. In general, diversity can improve both accuracy and robustness, while hardness can boost robustness at the cost of accuracy within a certain limit and degrade them both over that limit. To mitigate robust overfitting, we first propose a new crop transformation, Cropshift, which has improved diversity compared to the conventional one (Padcrop). We then propose a new data augmentation scheme, based on Cropshift, with much improved diversity and well-balanced hardness. Empirically, our augmentation method achieves the state-of-the-art accuracy and robustness for data augmentations in adversarial training. Furthermore, when combined with weight averaging it matches, or even exceeds, the performance of the best contemporary regularization methods for alleviating robust overfitting. Code is available at: this https URL.
Abstract (translated)
对抗性训练面临着稳健过度拟合的问题,这严重限制了其泛化性能。数据增强在标准训练中可以有效地防止过度拟合,但在许多过去的研究中,在对抗性训练中却未能减轻过度拟合。这项工作证明了,与以前的发现相反,数据增强单独可以显著增强对抗性训练中的准确性和稳健性。我们发现,数据增强的硬度和多样性是对抗稳健过度拟合的重要因素。一般来说,多样性可以提高准确性和稳健性,而硬度可以在准确性的代价下提高稳健性,并在超过一定极限时降低两者的性能。为了减轻稳健过度拟合,我们首先提出了一种新的作物变换,即“Cropshift”,它与传统的作物变换相比,可以提高多样性(例如,Padcrop)。然后我们提出了基于Cropshift的新数据增强方案,其多样性得到了显著提高,硬度也得到了平衡。经验证,我们的增强方法在对抗性训练中实现了最先进的数据增强准确性和稳健性。此外,与权重平均相结合,它甚至可以与最好的 contemporary 正则化方法的性能相媲美,以减轻稳健过度拟合。代码可在以下 https URL 获得。
URL
https://arxiv.org/abs/2301.09879
