Abstract
Thanks to the evolving network depth, convolutional neural networks (CNNs) have achieved remarkable success across various embedded scenarios, paving the way for ubiquitous embedded intelligence. Despite its promise, the evolving network depth comes at the cost of degraded hardware efficiency. In contrast to deep networks, shallow networks can deliver superior hardware efficiency but often suffer from inferior accuracy. To address this dilemma, we propose Double-Win NAS, a novel deep-to-shallow transformable neural architecture search (NAS) paradigm tailored for resource-constrained intelligent embedded systems. Specifically, Double-Win NAS strives to automatically explore deep networks to first win strong accuracy, which are then equivalently transformed into their shallow counterparts to further win strong hardware efficiency. In addition to search, we also propose two enhanced training techniques, including hybrid transformable training towards better training accuracy and arbitrary-resolution elastic training towards enabling natural network elasticity across arbitrary input resolutions. Extensive experimental results on two popular intelligent embedded systems (i.e., NVIDIA Jetson AGX Xavier and NVIDIA Jetson Nano) and two representative large-scale datasets (i.e., ImageNet and ImageNet-100) clearly demonstrate the superiority of Double-Win NAS over previous state-of-the-art NAS approaches.
Abstract (translated)
得益于网络深度的演进，卷积神经网络（CNN）在各种嵌入式场景中取得了显著的成功，为无处不在的嵌入式智能铺平了道路。尽管其前景广阔，但随着网络深度的增加，硬件效率也随之下降。与深层网络相比，浅层网络可以提供更优的硬件效率，但却常常因为精度较低而逊色。为了应对这一困境，我们提出了Double-Win NAS，这是一种新颖的从深到浅可变换神经架构搜索（NAS）范式，专为资源受限的智能嵌入式系统设计。具体而言，Double-Win NAS致力于自动探索深层网络以首先获得强大的准确性，然后将其等效地转换为其浅层版本，从而进一步提高硬件效率。除了搜索之外，我们还提出了两种增强训练技术，包括混合可变换训练（旨在提高训练精度）和任意分辨率弹性训练（用于实现跨任意输入分辨率的自然网络弹性）。在两个流行的智能嵌入式系统（即NVIDIA Jetson AGX Xavier和NVIDIA Jetson Nano）以及两个代表性的大规模数据集（即ImageNet和ImageNet-100）上的大量实验结果清楚地表明，Double-Win NAS优于之前的最先进的NAS方法。
URL
https://arxiv.org/abs/2512.19731
