Abstract
The remarkable performance of deep Convolutional neural networks (CNNs) is generally attributed to their deeper and wider architectures, which can come with significant computational costs. Pruning neural networks has thus gained interest since it effectively lowers storage and computational costs. In contrast to weight pruning, which results in unstructured models, structured pruning provides the benefit of realistic acceleration by producing models that are friendly to hardware implementation. The special requirements of structured pruning have led to the discovery of numerous new challenges and the development of innovative solutions. This article surveys the recent progress towards structured pruning of deep CNNs. We summarize and compare the state-of-the-art structured pruning techniques with respect to filter ranking methods, regularization methods, dynamic execution, neural architecture search, the lottery ticket hypothesis, and the applications of pruning. While discussing structured pruning algorithms, we briefly introduce the unstructured pruning counterpart to emphasize their differences. Furthermore, we provide insights into potential research opportunities in the field of structured pruning. A curated list of neural network pruning papers can be found at this https URL
Abstract (translated)
深度学习卷积神经网络(CNN)的出色表现通常归咎于其更深层和更广阔的架构,这可能会导致高昂的计算成本。因此,神经网络修剪变得越来越受欢迎,因为它有效地降低了存储和计算成本。与重量修剪(导致无结构模型)相比,结构修剪产生实际加速的效果,通过产生对硬件实现友好的模型。结构修剪的特殊要求导致了发现许多新挑战和创新解决方案。本文综述了近年来深度学习CNN结构修剪的进展。我们总结和比较了最先进的结构修剪技术,与筛选排名方法、 Regularization方法、动态执行、神经网络架构搜索、彩票假设和修剪应用等。在讨论结构修剪算法时,我们简要介绍了无结构修剪的对应版本,以强调它们之间的差异。此外,我们提供了在结构修剪领域的潜在研究机会的洞察力。一个精选的神经网络修剪论文列表可以在这个 https URL 中找到。
URL
https://arxiv.org/abs/2303.00566
