Abstract
Evolutionary computation (EC)-based neural architecture search (NAS) has achieved remarkable performance in the automatic design of neural architectures. However, the high computational cost associated with evaluating searched architectures poses a challenge for these methods, and a fixed form of learning rate (LR) schedule means greater information loss on diverse searched architectures. This paper introduces an efficient EC-based NAS method to solve these problems via an innovative meta-learning framework. Specifically, a meta-learning-rate (Meta-LR) scheme is used through pretraining to obtain a suitable LR schedule, which guides the training process with lower information loss when evaluating each individual. An adaptive surrogate model is designed through an adaptive threshold to select the potential architectures in a few epochs and then evaluate the potential architectures with complete epochs. Additionally, a periodic mutation operator is proposed to increase the diversity of the population, which enhances the generalizability and robustness. Experiments on CIFAR-10, CIFAR-100, and ImageNet1K datasets demonstrate that the proposed method achieves high performance comparable to that of many state-of-the-art peer methods, with lower computational cost and greater robustness.
Abstract (translated)
基于进化计算（EC）的神经架构搜索（NAS）在自动设计神经网络结构方面取得了显著成果。然而，评估所搜寻架构时的高计算成本对这些方法构成了一定挑战，并且固定的学习率（LR）调度方式会导致针对多样化架构的信息损失增加。本文提出了一种创新性的元学习框架，以一种高效的基于进化计算的NAS方法来解决这些问题。具体来说，通过预训练使用了一个元学习率（Meta-LR）方案，该方案可以获取一个合适的LR调度，并在评估每个个体时减少信息损失。另外设计了一种自适应替代模型，利用自适应阈值选择潜在架构并在少量轮次中筛选这些架构，在之后的完整轮次进行充分评估。此外，还提出了一种周期性变异算子以增加群体多样性，从而增强泛化能力和鲁棒性。 在CIFAR-10、CIFAR-100和ImageNet1K数据集上的实验表明，所提方法在计算成本更低的情况下达到了与许多最先进同类方法相当的高性能，并且具有更高的鲁棒性。
URL
https://arxiv.org/abs/2504.21545
