Abstract
In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods that prune channels only once per layer in a layer-by-layer fashion, our new progressive framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration. In the attempting step, we attempt to prune a pre-defined number of channels from one layer by using any existing channel pruning methods and estimate the accuracy drop for this layer based on the labelled samples in the validation set. In the selecting step, based on the estimated accuracy drops for all layers, we propose a greedy strategy to automatically select a set of layers that will lead to less overall accuracy drop after pruning these layers. In the pruning step, we prune a small number of channels from these selected layers. We further extend our PCP framework to prune channels for the deep transfer learning methods like Domain Adversarial Neural Network (DANN), in which we effectively reduce the data distribution mismatch in the channel pruning process by using both labelled samples from the source domain and pseudo-labelled samples from the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches under both supervised learning and transfer learning settings.
Abstract (translated)
在这项工作中，我们提出了一种简单但有效的通道剪枝框架，称为渐进式通道剪枝（PCP），用于加速卷积神经网络（CNN）。与现有的逐层一次性修剪一层中所有通道的通道剪枝方法不同，我们的新渐进式框架通过迭代地从选定的几层中剪除少量通道来工作。每个迭代包括尝试、选择和剪枝三个步骤的管道。 在尝试阶段，我们使用任何现有通道剪枝方法对一个预定义数量的通道进行逐层修剪，并基于验证集中的标记样本估计该层的精度损失。 在选择阶段，根据所有层上的估算精度下降值，我们提出了一种贪婪策略以自动选择一组经过这些操作后整体精度下降最小的层级。 在剪枝阶段，从选定的各层中实际剪除少量通道。 此外，我们将PCP框架扩展到了深层迁移学习方法（例如域对抗神经网络DANN）中的通道剪枝。在这个过程中，我们通过使用源领域和目标领域的标记样本以及伪标签样本来有效减少数据分布不匹配问题，在通道修剪流程中提升了性能表现。 我们在两个基准数据集上的全面实验表明，我们的PCP框架在监督学习和迁移学习设置下均优于现有的通道剪枝方法。
URL
https://arxiv.org/abs/2507.04792
