Abstract
Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search. However, these approaches often report lower accuracy in evaluating the searched architecture or transferring it to another dataset. This is arguably due to the large gap between the architecture depths in search and evaluation scenarios. In this paper, we present an efficient algorithm which allows the depth of searched architectures to grow gradually during the training procedure. This brings two issues, namely, heavier computational overheads and weaker search stability, which we solve using search space approximation and regularization, respectively. With a significantly reduced search time (~7 hours on a single GPU), our approach achieves state-of-the-art performance on both the proxy dataset (CIFAR10 or CIFAR100) and the target dataset (ImageNet). Code is available at https://github.com/chenxin061/pdarts.
Abstract (translated)
近年来，可微搜索方法在降低神经结构搜索的计算成本方面取得了重大进展。然而，这些方法在评估所搜索的体系结构或将其传输到另一个数据集时通常报告的准确性较低。这可以说是由于搜索和评估场景中的体系结构深度之间存在很大的差距。在本文中，我们提出了一种有效的算法，使得在训练过程中搜索架构的深度逐渐增加。这就带来了两个问题，即计算开销大，搜索稳定性差，分别用搜索空间近似法和正则化法求解。通过显著缩短搜索时间（在单个GPU上约7小时），我们的方法在代理数据集（cifar10或cifar100）和目标数据集（imagenet）上实现了最先进的性能。代码可从https://github.com/chenxin061/pdarts获取。
URL
https://arxiv.org/abs/1904.12760
