Abstract
Network binarization emerges as one of the most promising compression approaches offering extraordinary computation and memory savings by minimizing the bit-width. However, recent research has shown that applying existing binarization algorithms to diverse tasks, architectures, and hardware in realistic scenarios is still not straightforward. Common challenges of binarization, such as accuracy degradation and efficiency limitation, suggest that its attributes are not fully understood. To close this gap, we present BiBench, a rigorously designed benchmark with in-depth analysis for network binarization. We first carefully scrutinize the requirements of binarization in the actual production and define evaluation tracks and metrics for a comprehensive and fair investigation. Then, we evaluate and analyze a series of milestone binarization algorithms that function at the operator level and with extensive influence. Our benchmark reveals that 1) the binarized operator has a crucial impact on the performance and deployability of binarized networks; 2) the accuracy of binarization varies significantly across different learning tasks and neural architectures; 3) binarization has demonstrated promising efficiency potential on edge devices despite the limited hardware support. The results and analysis also lead to a promising paradigm for accurate and efficient binarization. We believe that BiBench will contribute to the broader adoption of binarization and serve as a foundation for future research.
Abstract (translated)
网络配分 emerging as one of the most promising压缩方法,通过最小化比特宽度提供非凡的计算和内存节省。然而,最近的研究表明,将现有配分算法应用于实际场景中的各种任务、架构和硬件仍然不太简单。配分的常见挑战,如精度下降和效率限制,表明其属性并未完全理解。为了填补这一差距,我们提出了比Bench(BiBench),一个严格设计的标准参考框架,以深入研究网络配分。我们首先仔细审查了实际生产中的配分要求,并定义了全面公正的调查评估轨道和指标。然后,我们对一系列里程碑配分算法进行了评估和分析,它们在操作级别上具有广泛影响力。我们的基准表明,1)配分操作对配分网络的性能和部署具有关键影响;2)配分精度在不同学习任务和神经网络架构中差异显著;3)尽管硬件支持有限,配分在边缘设备上表现出令人鼓舞的效率潜力。结果和分析还引出了准确和高效的配分的良好范式。我们相信比bench将促进更广泛的采用配分,并成为未来研究的 foundation。
URL
https://arxiv.org/abs/2301.11233
