Abstract
Early-exit networks are effective solutions for reducing the overall energy consumption and latency of deep learning models by adjusting computation based on the complexity of input data. By incorporating intermediate exit branches into the architecture, they provide less computation for simpler samples, which is particularly beneficial for resource-constrained devices where energy consumption is crucial. However, designing early-exit networks is a challenging and time-consuming process due to the need to balance efficiency and performance. Recent works have utilized Neural Architecture Search (NAS) to design more efficient early-exit networks, aiming to reduce average latency while improving model accuracy by determining the best positions and number of exit branches in the architecture. Another important factor affecting the efficiency and accuracy of early-exit networks is the depth and types of layers in the exit branches. In this paper, we use hardware-aware NAS to strengthen exit branches, considering both accuracy and efficiency during optimization. Our performance evaluation on the CIFAR-10, CIFAR-100, and SVHN datasets demonstrates that our proposed framework, which considers varying depths and layers for exit branches along with adaptive threshold tuning, designs early-exit networks that achieve higher accuracy with the same or lower average number of MACs compared to the state-of-the-art approaches.
Abstract (translated)
早期退出网络通过根据输入数据的复杂性调整计算量，是减少深度学习模型整体能耗和延迟的有效解决方案。通过在架构中引入中间退出分支，这些网络为简单样本提供较少的计算资源，在能源消耗至关重要的资源受限设备上特别有益。然而，设计早期退出网络是一个既具挑战又耗时的过程，因为它需要平衡效率与性能。最近的研究利用神经架构搜索（NAS）来设计更高效的早期退出网络，通过确定最佳的退出分支位置和数量，以减少平均延迟并提高模型精度。 影响早期退出网络效率和准确性的另一个重要因素是退出分支中的深度及层类型。在本文中，我们采用硬件感知型NAS技术来增强退出分支，在优化过程中同时考虑准确性与效率。我们在CIFAR-10、CIFAR-100和SVHN数据集上的性能评估表明，我们的框架通过综合考虑退出分支的多变深度和层类型以及自适应阈值调整，设计出的早期退出网络达到了比当前最先进方法相同或更低的平均MAC数（乘法累加操作），同时实现了更高的准确率。
URL
https://arxiv.org/abs/2512.10671
