Abstract
Determining the performance of a Deep Neural Network during Neural Architecture Search processes is essential for identifying optimal architectures and hyperparameters. Traditionally, this process requires training and evaluation of each network, which is time-consuming and resource-intensive. Zero-cost proxies estimate performance without training, serving as an alternative to traditional training. However, recent proxies often lack generalization across diverse scenarios and provide only relative rankings rather than predicted accuracies. To address these limitations, we propose GreenFactory, an ensemble of zero-cost proxies that leverages a random forest regressor to combine multiple predictors' strengths and directly predict model test accuracy. We evaluate GreenFactory on NATS-Bench, achieving robust results across multiple datasets. Specifically, GreenFactory achieves high Kendall correlations on NATS-Bench-SSS, indicating substantial agreement between its predicted scores and actual performance: 0.907 for CIFAR-10, 0.945 for CIFAR-100, and 0.920 for ImageNet-16-120. Similarly, on NATS-Bench-TSS, we achieve correlations of 0.921 for CIFAR-10, 0.929 for CIFAR-100, and 0.908 for ImageNet-16-120, showcasing its reliability in both search spaces.
Abstract (translated)
在神经架构搜索过程中，确定深层神经网络的性能对于识别最优架构和超参数至关重要。传统的方法需要对每个网络进行训练和评估，这既耗时又耗费资源。零成本代理（zero-cost proxies）通过不需训练就能估计性能来作为替代方案，但最近的一些代理方法在不同场景下缺乏泛化能力，并且只能提供相对排名而非预测准确率。为了解决这些问题，我们提出了GreenFactory，这是一个结合了多种零成本代理的集合体，利用随机森林回归器将多个预测器的优势整合起来，并直接预测模型的测试准确性。 我们在NATS-Bench上对GreenFactory进行了评估，在多数据集上取得了稳健的结果。具体来说，在NATS-Bench-SSS中，GreenFactory在Kendall等级相关系数方面表现优异：CIFAR-10为0.907，CIFAR-100为0.945，ImageNet-16-120为0.920。同样地，在NATS-Bench-TSS中，我们达到了以下相关性结果：CIFAR-10为0.921，CIFAR-100为0.929，ImageNet-16-120为0.908，这表明GreenFactory在不同的搜索空间内都具有可靠的预测性能。
URL
https://arxiv.org/abs/2505.09344
