Abstract
Performance predictors have emerged as a promising method to accelerate the evaluation stage of neural architecture search (NAS). These predictors estimate the performance of unseen architectures by learning from the correlation between a small set of trained architectures and their performance. However, most existing predictors ignore the inherent distribution shift between limited training samples and diverse test samples. Hence, they tend to learn spurious correlations as shortcuts to predictions, leading to poor generalization. To address this, we propose a Causality-guided Architecture Representation Learning (CARL) method aiming to separate critical (causal) and redundant (non-causal) features of architectures for generalizable architecture performance prediction. Specifically, we employ a substructure extractor to split the input architecture into critical and redundant substructures in the latent space. Then, we generate multiple interventional samples by pairing critical representations with diverse redundant representations to prioritize critical features. Extensive experiments on five NAS search spaces demonstrate the state-of-the-art accuracy and superior interpretability of CARL. For instance, CARL achieves 97.67% top-1 accuracy on CIFAR-10 using DARTS.
Abstract (translated)
性能预测器已经作为加速神经架构搜索（NAS）评估阶段的一种有前景的方法出现。这些预测器通过从一组有限的训练架构及其性能之间的相关性学习，来估算未见过架构的性能。然而，大多数现有的预测方法忽略了在有限训练样本和多样测试样本之间固有的分布差异。因此，它们往往倾向于学习虚假的相关性作为预测的捷径，从而导致较差的泛化能力。 为了解决这个问题，我们提出了一种基于因果关系的架构表示学习（CARL）方法，旨在区分架构中的关键（因果）特征与冗余（非因果）特征，以实现通用化的架构性能预测。具体来说，我们使用了一个子结构提取器，将输入的架构在潜在空间中拆分为关键和冗余的子结构。然后，通过将关键表示与多样化的冗余表示配对来生成多个干预样本，从而优先考虑关键特征。 广泛的实验结果表明，在五个NAS搜索空间上，CARL方法达到了最先进的准确性，并且具有优越的可解释性。例如，使用DARTS在CIFAR-10数据集上，CARL实现了97.67%的第一名准确率。
URL
https://arxiv.org/abs/2506.04001
