Abstract
Deep neural networks have been proven effective in a wide range of tasks. However, their high computational and memory costs make them impractical to deploy on resource-constrained devices. To address this issue, quantization schemes have been proposed to reduce the memory footprint and improve inference speed. While numerous quantization methods have been proposed, they lack systematic analysis for their effectiveness. To bridge this gap, we collect and improve existing quantization methods and propose a gold guideline for post-training quantization. We evaluate the effectiveness of our proposed method with two popular models, ResNet50 and MobileNetV2, on the ImageNet dataset. By following our guidelines, no accuracy degradation occurs even after directly quantizing the model to 8-bits without additional training. A quantization-aware training based on the guidelines can further improve the accuracy in lower-bits quantization. Moreover, we have integrated a multi-stage fine-tuning strategy that works harmoniously with existing pruning techniques to reduce costs even further. Remarkably, our results reveal that a quantized MobileNetV2 with 30\% sparsity actually surpasses the performance of the equivalent full-precision model, underscoring the effectiveness and resilience of our proposed scheme.
Abstract (translated)
深度神经网络在多种任务中已被证明有效。然而,它们的高计算和内存成本使其在资源受限的设备上部署不可行。为了解决这一问题,提出了量化方案以减少内存占用并提高推理速度。虽然已经提出了许多量化方法,但它们缺乏对它们的系统性分析。为了填补这一差距,我们收集并改进了现有的量化方法,并提出了在训练后量化的最优指南。我们使用ResNet50和MobileNetV2等流行的模型,在ImageNet数据集上评估了我们提出的方法和两个模型的效果。通过遵循我们的指南,即使在直接量化模型到8位二进制数的情况下,也没有发生精度下降。基于指南的量化训练可以进一步改进较低位量化的准确性。此外,我们整合了一个多阶段微调策略,它与现有的修剪技术和谐共处,进一步减少了成本。令人惊讶地,我们的结果表明,一个量化MobileNetV2的 sparsity 为30\%的模型实际上超过了等价的全精度模型的性能,强调了我们提出的方案的有效性和韧性。
URL
https://arxiv.org/abs/2303.07080
