Abstract
We introduce Universal Neural Architecture Space (UniNAS), a generic search space for neural architecture search (NAS) which unifies convolutional networks, transformers, and their hybrid architectures under a single, flexible framework. Our approach enables discovery of novel architectures as well as analyzing existing architectures in a common framework. We also propose a new search algorithm that allows traversing the proposed search space, and demonstrate that the space contains interesting architectures, which, when using identical training setup, outperform state-of-the-art hand-crafted architectures. Finally, a unified toolkit including a standardized training and evaluation protocol is introduced to foster reproducibility and enable fair comparison in NAS research. Overall, this work opens a pathway towards systematically exploring the full spectrum of neural architectures with a unified graph-based NAS perspective.
Abstract (translated)
我们介绍了一种通用神经网络架构空间（UniNAS），这是一种用于神经架构搜索（NAS）的通用搜索空间，它将卷积网络、变压器及其混合架构统一在一个单一且灵活的框架下。我们的方法能够发现新型架构并使用同一框架分析现有的架构。此外，我们还提出了一种新的搜索算法，该算法允许在提议的搜索空间中进行遍历，并展示了此空间包含一些有趣的架构，在相同的训练设置下，这些架构的表现优于当前最先进的手工设计架构。最后，介绍了一个统一工具包，包括标准化的训练和评估协议，以促进研究结果的可重复性并支持NAS研究中的公平比较。总的来说，这项工作开辟了一条系统探索基于统一图论视角下的神经网络架构全谱系的道路。
URL
https://arxiv.org/abs/2510.06035
