Abstract
Gradient-based one-shot neural architecture search (NAS) has significantly reduced the cost of exploring architectural spaces with discrete design choices, such as selecting operations within a model. However, the field faces two major challenges. First, evaluations of gradient-based NAS methods heavily rely on the DARTS benchmark, despite the existence of other available benchmarks. This overreliance has led to saturation, with reported improvements often falling within the margin of noise. Second, implementations of gradient-based one-shot NAS methods are fragmented across disparate repositories, complicating fair and reproducible comparisons and further development. In this paper, we introduce Configurable Optimizer (confopt), an extensible library designed to streamline the development and evaluation of gradient-based one-shot NAS methods. Confopt provides a minimal API that makes it easy for users to integrate new search spaces, while also supporting the decomposition of NAS optimizers into their core components. We use this framework to create a suite of new DARTS-based benchmarks, and combine them with a novel evaluation protocol to reveal a critical flaw in how gradient-based one-shot NAS methods are currently assessed. The code can be found at this https URL.
Abstract (translated)
基于梯度的一次性神经架构搜索（NAS）显著降低了探索包含离散设计选择的架构空间的成本，例如在模型中选择操作。然而，该领域面临着两个主要挑战。首先，基于梯度的NAS方法的评估严重依赖于DARTS基准，尽管存在其他可用的基准。这种过度依赖导致了饱和状态，报告的改进往往处于噪声范围内。其次，基于梯度的一次性NAS方法的实现分散在不同的仓库中，这使得公平和可重复的比较及进一步开发变得复杂。 在这篇论文中，我们介绍了Configurable Optimizer (confopt)，这是一个可扩展的库，旨在简化基于梯度的一次性NAS方法的发展和评估。Confopt提供了一个最小化的API，使用户能够轻松集成新的搜索空间，并支持将NAS优化器分解为其核心组件。利用这一框架，我们创建了一套全新的DARTS基准测试集，并结合一种新颖的评价协议揭示了当前对基于梯度的一次性NAS方法评估的关键缺陷。 代码可在以下网址找到：[此URL]（请使用原文中的实际链接）。
URL
https://arxiv.org/abs/2507.16533
