Abstract
Neural architecture search (NAS) is a promising approach for automatically designing neural network architectures. However, the architecture estimation of NAS is computationally expensive and time-consuming because of training multiple architectures from scratch. Although existing zero-shot NAS methods use training-free proxies to accelerate the architecture estimation, their effectiveness, stability, and generality are still lacking. We present a novel training-free estimation proxy called weighted response correlation (WRCor). WRCor utilizes correlation coefficient matrices of responses across different input samples to calculate the proxy scores of estimated architectures, which can measure their expressivity and generalizability. Experimental results on proxy evaluation demonstrate that WRCor and its voting proxies are more efficient estimation strategies than existing proxies. We also apply them with different search strategies in architecture search. Experimental results on architecture search show that our zero-shot NAS algorithm outperforms most existing NAS algorithms in different search spaces. Our NAS algorithm can discover an architecture with a 22.1% test error on the ImageNet-1k dataset within 4 GPU hours. All codes are publicly available at this https URL.
Abstract (translated)
神经架构搜索（NAS）是一种有前景的自动设计神经网络架构的方法。然而，由于需要从头开始训练多个架构，NAS中的架构评估计算成本高且耗时。尽管现有的零样本NAS方法使用无训练代理来加速架构估计，但它们的有效性、稳定性和通用性仍然不足。我们提出了一种名为加权响应相关（WRCor）的新颖无训练估算代理。WRCor利用不同输入样本之间的响应的相关系数矩阵来计算所估架构的代理分数，以衡量其表达能力和泛化能力。在代理评估实验中表明，与现有代理相比，WRCor及其投票代理是更高效的估计策略。我们还将其与不同的搜索策略结合应用于架构搜索中。在架构搜索方面的实验证明，在各种搜索空间中，我们的零样本NAS算法优于大多数现有的NAS算法。我们的NAS算法能够在4个GPU小时内发现一个在ImageNet-1k数据集上具有22.1%测试错误率的架构。所有代码都公开发布在此网址：[此URL]（请使用原文中的链接）。
URL
https://arxiv.org/abs/2507.08841
