Abstract
For network architecture search (NAS), it is crucial but challenging to simultaneously guarantee both effectiveness and efficiency. Towards achieving this goal, we develop a differentiable NAS solution, where the search space includes arbitrary feed-forward network consisting of the predefined number of connections. Benefiting from a proposed ensemble Gumbel-Softmax estimator, our method optimizes both the architecture of a deep network and its parameters in the same round of backward propagation, yielding an end-to-end mechanism of searching network architectures. Extensive experiments on a variety of popular datasets strongly evidence that our method is capable of discovering high-performance architectures, while guaranteeing the requisite efficiency during searching.
Abstract (translated)
对于网络体系结构搜索（NAS），同时保证有效性和效率是至关重要的，但也是具有挑战性的。为了实现这一目标，我们开发了一个可区别的NAS解决方案，其中搜索空间包括由预先定义的连接数组成的任意前馈网络。该方法利用所提出的合集Gumbel-Softmax估计量，在同一轮反向传播中对深网结构及其参数进行了优化，得到了一种搜索网络结构的端到端机制。对各种流行的数据集进行了大量的实验，有力地证明了我们的方法能够发现高性能的体系结构，同时保证了在搜索过程中所需的效率。
URL
https://arxiv.org/abs/1905.01786
