Abstract
Advancements in high-performance computing and cloud technologies have enabled the development of increasingly sophisticated Deep Learning (DL) models. However, the growing demand for embedded intelligence at the edge imposes stringent computational and energy constraints, challenging the deployment of these large-scale models. Early Exiting Neural Networks (EENN) have emerged as a promising solution, allowing dynamic termination of inference based on input complexity to enhance efficiency. Despite their potential, EENN performance is highly influenced by the heterogeneity of edge accelerators and the constraints imposed by quantization, affecting accuracy, energy efficiency, and latency. Yet, research on the automatic optimization of EENN design for edge hardware remains limited. To bridge this gap, we propose a hardware-aware Neural Architecture Search (NAS) framework that systematically integrates the effects of quantization and hardware resource allocation to optimize the placement of early exit points within a network backbone. Experimental results on the CIFAR-10 dataset demonstrate that our NAS framework can discover architectures that achieve over a 50\% reduction in computational costs compared to conventional static networks, making them more suitable for deployment in resource-constrained edge environments.
Abstract (translated)
高性能计算和云技术的进步推动了深度学习（DL）模型的快速发展。然而，随着对边缘设备嵌入式智能的需求日益增长，严格的计算和能源限制给大型模型的应用部署带来了挑战。早期退出神经网络（EENN）作为一种有潜力的解决方案应运而生，它通过根据输入复杂性动态终止推理过程来提高效率。尽管如此，EENN的表现受到边缘加速器异构性和量化约束的影响，这些因素影响了准确性、能源效率和延迟。但是，针对边缘硬件自动优化EENN设计的研究仍然有限。 为了解决这一问题，我们提出了一种硬件感知的神经架构搜索（NAS）框架，该框架系统地整合了量化的效应以及硬件资源分配对早期退出点在网络骨干中定位的影响。在CIFAR-10数据集上的实验结果表明，我们的NAS框架能够发现比传统静态网络计算成本减少超过50%的架构，使其更适合部署于资源受限的边缘环境中。
URL
https://arxiv.org/abs/2512.04705
