Abstract
The integration of Large Language Models (LLMs) with Neural Architecture Search (NAS) has introduced new possibilities for automating the design of neural architectures. However, most existing methods face critical limitations, including architectural invalidity, computational inefficiency, and inferior performance compared to traditional NAS. In this work, we present Collaborative LLM-based NAS (CoLLM-NAS), a two-stage NAS framework with knowledge-guided search driven by two complementary LLMs. Specifically, we propose a Navigator LLM to guide search direction and a Generator LLM to synthesize high-quality candidates, with a dedicated Coordinator module to manage their interaction. CoLLM-NAS efficiently guides the search process by combining LLMs' inherent knowledge of structured neural architectures with progressive knowledge from iterative feedback and historical trajectory. Experimental results on ImageNet and NAS-Bench-201 show that CoLLM-NAS surpasses existing NAS methods and conventional search algorithms, achieving new state-of-the-art results. Furthermore, CoLLM-NAS consistently enhances the performance and efficiency of various two-stage NAS methods (e.g., OFA, SPOS, and AutoFormer) across diverse search spaces (e.g., MobileNet, ShuffleNet, and AutoFormer), demonstrating its excellent generalization.
Abstract (translated)
大型语言模型（LLMs）与神经架构搜索（NAS）的结合为自动设计神经网络结构带来了新的可能性。然而，大多数现有方法面临着诸如架构无效性、计算效率低下以及相较于传统NAS性能较差的关键限制。在此项研究中，我们提出了基于协作大语言模型的神经架构搜索框架（CoLLM-NAS），这是一个两阶段的NAS框架，由两个互补的大语言模型驱动的知识引导搜索，并配备了一个专用的协调模块来管理它们之间的互动。 具体来说，我们提出了一种导航器大语言模型来指导搜索方向，并且还设计了一个生成器大语言模型来合成高质量候选结构。此外，还有一个专有的协调模块负责管理和促进这两种大语言模型之间的协作过程。CoLLM-NAS通过结合大语言模型对结构化神经架构的固有知识以及从迭代反馈和历史轨迹中获得的进步性知识，有效地引导搜索流程。 在ImageNet和NAS-Bench-201数据集上的实验结果表明，CoLLM-NAS超越了现有的NAS方法和传统的搜索算法，并取得了新的最先进的性能。此外，在不同的搜索空间（例如MobileNet、ShuffleNet以及AutoFormer）上对各种两阶段NAS方法（如OFA、SPOS和AutoFormer）进行评估时，CoLLM-NAS持续提高了这些方法的性能和效率，展示了其卓越的泛化能力。
URL
https://arxiv.org/abs/2509.26037
