Abstract
This paper introduces Growing Networks with Autonomous Pruning (GNAP) for image classification. Unlike traditional convolutional neural networks, GNAP change their size, as well as the number of parameters they are using, during training, in order to best fit the data while trying to use as few parameters as possible. This is achieved through two complementary mechanisms: growth and pruning. GNAP start with few parameters, but their size is expanded periodically during training to add more expressive power each time the network has converged to a saturation point. Between these growing phases, model parameters are trained for classification and pruned simultaneously, with complete autonomy by gradient descent. Growing phases allow GNAP to improve their classification performance, while autonomous pruning allows them to keep as few parameters as possible. Experimental results on several image classification benchmarks show that our approach can train extremely sparse neural networks with high accuracy. For example, on MNIST, we achieved 99.44% accuracy with as few as 6.2k parameters, while on CIFAR10, we achieved 92.2\ accuracy with 157.8k parameters.
Abstract (translated)
本文介绍了用于图像分类的自主修剪生长网络（GNAP）。与传统卷积神经网络不同，GNAP在训练过程中会动态调整其规模及所用参数数量，以在尽可能减少参数的同时最佳适配数据。这通过两个互补机制实现：生长与修剪。GNAP初始参数较少，但在每次网络收敛至饱和点后，会定期扩展规模以增强表达能力。在生长阶段之间，模型参数在分类训练中同步进行自主修剪，完全由梯度下降驱动。生长阶段提升分类性能，而自主修剪则确保参数精简。在多个图像分类基准测试中，该方法能训练出高精度且极度稀疏的神经网络。例如，在MNIST数据集上，仅用6.2k参数即达到99.44%准确率；在CIFAR10数据集上，以157.8k参数实现92.2%准确率。
URL
https://arxiv.org/abs/2603.19759
