Abstract
Today, artificial neural networks are the state of the art for solving a variety of complex tasks, especially in image classification. Such architectures consist of a sequence of stacked layers with the aim of extracting useful information and having it processed by a classifier to make accurate predictions. However, intermediate information within such models is often left unused. In other cases, such as in edge computing contexts, these architectures are divided into multiple partitions that are made functional by including early exits, i.e. intermediate classifiers, with the goal of reducing the computational and temporal load without extremely compromising the accuracy of the classifications. In this paper, we present Anticipate, Ensemble and Prune (AEP), a new training technique based on weighted ensembles of early exits, which aims at exploiting the information in the structure of networks to maximise their performance. Through a comprehensive set of experiments, we show how the use of this approach can yield average accuracy improvements of up to 15% over traditional training. In its hybrid-weighted configuration, AEP's internal pruning operation also allows reducing the number of parameters by up to 41%, lowering the number of multiplications and additions by 18% and the latency time to make inference by 16%. By using AEP, it is also possible to learn weights that allow early exits to achieve better accuracy values than those obtained from single-output reference models.
Abstract (translated)
当今,神经网络是解决各种复杂任务的最新技术,特别是在图像分类方面。这些架构由一系列叠加的层组成,旨在提取有用的信息,并通过分类器进行处理以进行准确的预测。然而,在这些模型中,常常存在中间信息,而这些信息往往未被充分利用。在其他情况下,如在边缘计算上下文中,这些架构被分为多个分区,通过包括早期退出(即中间分类器)来实现功能,以减轻计算和时间负载,而又不想极大地影响分类的准确性。在本文中,我们介绍了预测、集成和修剪(AEP),一种新的训练技术,基于早期退出的加权集合,旨在利用网络结构中的信息,最大限度地提高其性能。通过全面的实验组合,我们展示了如何使用这种方法可以平均地提高传统训练的准确率多达15%。在混合加权配置中,AEP的内部修剪操作还可以允许减少参数数量多达41%,降低乘法和加法的数量多达18%,缩短推理的延迟时间多达16%。通过使用AEP,也可以尝试学习让早期退出实现比单输出参考模型更好的准确性值的权重。
URL
https://arxiv.org/abs/2301.12168
