Abstract
Deep Neural Network (DNN) trained by the gradient descent method is known to be vulnerable to maliciously perturbed adversarial input, aka. adversarial attack. As one of the countermeasures against adversarial attack, increasing the model capacity for DNN robustness enhancement was discussed and reported as an effective approach by many recent works. In this work, we show that shrinking the model size through proper weight pruning can even be helpful to improve the DNN robustness under adversarial attack. For obtaining a simultaneously robust and compact DNN model, we propose a multi-objective training method called Robust Sparse Regularization (RSR), through the fusion of various regularization techniques, including channel-wise noise injection, lasso weight penalty, and adversarial training. We conduct extensive experiments across popular ResNet-20, ResNet-18 and VGG-16 DNN architectures to demonstrate the effectiveness of RSR against popular white-box (i.e., PGD and FGSM) and black-box attacks. Thanks to RSR, 85% weight connections of ResNet-18 can be pruned while still achieving 0.68% and 8.72% improvement in clean- and perturbed-data accuracy respectively on CIFAR-10 dataset, in comparison to its PGD adversarial training baseline.
Abstract (translated)
采用梯度下降法训练的深部神经网络（DNN）容易受到恶意干扰的敌方输入的攻击。对抗性攻击。作为对抗敌方攻击的对策之一，提高DNN模型的鲁棒性是近年来研究的一种有效方法。在这项工作中，我们发现通过适当的权值修剪来缩小模型尺寸，甚至有助于提高在对抗攻击下的DNN鲁棒性。为了获得同时具有鲁棒性和紧凑性的DNN模型，我们提出了一种多目标训练方法，称为鲁棒稀疏正则化（RSR），通过融合各种正则化技术，包括信道噪声注入、套索重量惩罚和对抗训练。我们对流行的resnet-20、resnet-18和vgg-16 dnn体系结构进行了广泛的实验，以证明rsr对流行的白盒（即pgd和fgsm）和黑盒攻击的有效性。由于RSR，与PGD对抗训练基线相比，在CIFAR-10数据集上，85%重量的resnet-18连接可以被修剪，同时仍然可以分别提高0.68%和8.72%的干净和扰动数据精度。
URL
https://arxiv.org/abs/1905.13074
