Abstract
Large-scale deep neural networks (DNNs) have achieved remarkable success in many application scenarios. However, high computational complexity and energy costs of modern DNNs make their deployment on edge devices challenging. Model quantization is a common approach to deal with deployment constraints, but searching for optimized bit-widths can be challenging. In this work, we present Adaptive Bit-Width Quantization Aware Training (AdaQAT), a learning-based method that automatically optimizes weight and activation signal bit-widths during training for more efficient DNN inference. We use relaxed real-valued bit-widths that are updated using a gradient descent rule, but are otherwise discretized for all quantization operations. The result is a simple and flexible QAT approach for mixed-precision uniform quantization problems. Compared to other methods that are generally designed to be run on a pretrained network, AdaQAT works well in both training from scratch and fine-tuning scenarios.Initial results on the CIFAR-10 and ImageNet datasets using ResNet20 and ResNet18 models, respectively, indicate that our method is competitive with other state-of-the-art mixed-precision quantization approaches.
Abstract (translated)
大规模的深度神经网络（DNNs）在许多应用场景中取得了显著的成功。然而，现代DNN的高计算复杂度和能源成本使得将它们部署到边缘设备上具有挑战性。模型量化是一种常见的应对部署限制的方法，但是寻找最优的比特宽度可能具有挑战性。在本文中，我们提出了自适应比特宽度量化感知训练（AdaQAT），一种基于学习的优化方法，用于在训练过程中自动优化权重和激活信号的比特宽度，以实现更高效的DNN推理。我们使用通过梯度下降规则更新的松弛实值比特宽度，但其他量化操作则全部离散化。结果是一种简单而灵活的QAT方法，用于混合精度统一量化问题。与通常为预训练网络设计的其他方法相比，AdaQAT在从零开始训练和微调场景上都表现良好。使用ResNet20和ResNet18模型的CIFAR-10和ImageNet数据集的初步结果表明，我们的方法与最先进的混合精度量化方法相当。
URL
https://arxiv.org/abs/2404.16876
