Abstract
Evolutionary Neural Architecture Search (ENAS) has gained attention for automatically designing neural network architectures. Recent studies use a neural predictor to guide the process, but the high computational costs of gathering training data -- since each label requires fully training an architecture -- make achieving a high-precision predictor with { limited compute budget (i.e., a capped number of fully trained architecture-label pairs)} crucial for ENAS success. This paper introduces ENAS with Dual Contrastive Learning (DCL-ENAS), a novel method that employs two stages of contrastive learning to train the neural predictor. In the first stage, contrastive self-supervised learning is used to learn meaningful representations from neural architectures without requiring labels. In the second stage, fine-tuning with contrastive learning is performed to accurately predict the relative performance of different architectures rather than their absolute performance, which is sufficient to guide the evolutionary search. Across NASBench-101 and NASBench-201, DCL-ENAS achieves the highest validation accuracy, surpassing the strongest published baselines by 0.05\% (ImageNet16-120) to 0.39\% (NASBench-101). On a real-world ECG arrhythmia classification task, DCL-ENAS improves performance by approximately 2.5 percentage points over a manually designed, non-NAS model obtained via random search, while requiring only 7.7 GPU-days.
Abstract (translated)
进化神经架构搜索（ENAS）因其能够自动设计神经网络结构而受到关注。近期的研究采用了神经预测器来指导这一过程，但由于收集训练数据所需的高昂计算成本——因为每个标签都需要对一个完整的架构进行完全训练——因此，在有限的计算预算内（即一定数量的完全训练过的架构-标签配对），实现高精度的预测对于ENAS的成功至关重要。 本文提出了一种名为具有双对比学习的ENAS (DCL-ENAS) 的新方法，该方法采用两个阶段的对比学习来训练神经预测器。在第一阶段，使用对比自监督学习从神经架构中学习有意义的表示，无需标签。在第二阶段，则通过对比学习进行微调，以准确预测不同架构之间的相对性能，而非绝对性能——这一点足以引导进化搜索过程。 在NASBench-101和NASBench-201两个基准测试上，DCL-ENAS实现了最高的验证精度，在ImageNet16-120和NASBench-101上的表现分别超越了最强的已发布基线模型0.05% 和 0.39%。在一项真实世界的心电图心律失常分类任务中，DCL-ENAS相较于手动设计且非NAS方法通过随机搜索获得的模型，在仅使用7.7个GPU天的情况下，提高了约2.5个百分点的性能。 此研究展示了对比学习技术如何优化神经架构搜索过程，并可能为未来的自动机器学习开发提供新的方向。
URL
https://arxiv.org/abs/2512.20112
