Abstract
Monte-Carlo Tree Search (MCTS) is a powerful tool for many non-differentiable search related problems such as adversarial games. However, the performance of such approach highly depends on the order of the nodes that are considered at each branching of the tree. If the first branches cannot distinguish between promising and deceiving configurations for the final task, the efficiency of the search is exponentially reduced. In Neural Architecture Search (NAS), as only the final architecture matters, the visiting order of the branching can be optimized to improve learning. In this paper, we study the application of MCTS to NAS for image classification. We analyze several sampling methods and branching alternatives for MCTS and propose to learn the branching by hierarchical clustering of architectures based on their similarity. The similarity is measured by the pairwise distance of output vectors of architectures. Extensive experiments on two challenging benchmarks on CIFAR10 and ImageNet show that MCTS, if provided with a good branching hierarchy, can yield promising solutions more efficiently than other approaches for NAS problems.
Abstract (translated)
蒙特卡洛树搜索（MCTS）是一种强大的工具，适用于许多与非微分相关的问题，如对抗性游戏。然而，这种方法的性能很大程度上取决于在每棵树分支时考虑节点的顺序。如果最初的几个分支无法区分对最终任务有前景和误导性的配置，那么搜索效率将呈指数级下降。在神经架构搜索（NAS）中，只有最后的架构才重要，因此可以通过优化分枝访问顺序来改进学习过程。在这篇论文中，我们研究了MCTS在图像分类领域应用于NAS的应用。我们分析了几种采样方法和MCTS分支选择的替代方案，并提出通过基于架构相似性的层次聚类来学习分支选择。相似性是通过架构输出向量之间的成对距离进行度量的。在两个具有挑战性的CIFAR10和ImageNet基准测试上的广泛实验表明，如果为MCTS提供一个良好的分枝层级结构，它能够比其他NAS方法更高效地生成有前景的解决方案。
URL
https://arxiv.org/abs/2503.21061
