Abstract
We address the challenge of training a large supernet for the object detection task, using a relatively small amount of training data. Specifically, we propose an efficient supernet-based neural architecture search (NAS) method that uses transfer learning and search space pruning. First, the supernet is pre-trained on a classification task, for which large datasets are available. Second, the search space defined by the supernet is pruned by removing candidate models that are predicted to perform poorly. To effectively remove the candidates over a wide range of resource constraints, we particularly design a performance predictor, called path filter, which can accurately predict the relative performance of the models that satisfy similar resource constraints. Hence, supernet training is more focused on the best-performing candidates. Our path filter handles prediction for paths with different resource budgets. Compared to once-for-all, our proposed method reduces the computational cost of the optimal network architecture by 30% and 63%, while yielding better accuracy-floating point operations Pareto front (0.85 and 0.45 points of improvement on average precision for Pascal VOC and COCO, respectively).
Abstract (translated)
我们解决了训练大型超网络用于目标检测任务的挑战，使用了相对较小的训练数据。具体而言，我们提出了一种高效的超网络神经网络架构搜索方法(NAS)，该方法使用迁移学习和搜索空间剪枝。首先，超网络在分类任务上进行了预训练，有大量数据可用。其次，超网络定义的搜索空间通过删除预测表现较差的候选模型进行修剪。为了有效地去除在各种资源限制下的候选模型，我们特别设计了性能预测器，称为路径滤波，它能够准确地预测满足类似资源限制的模型的性能相对表现。因此，超网络训练更关注表现最好的候选模型。我们的路径滤波处理不同资源预算下的预测路径。与一次性搜索相比，我们提出的方法降低了最优网络架构的计算成本，下降了30%和63%，同时提供了更好的浮点操作精度 Pareto 前端(分别提高0.85点和0.45点)。
URL
https://arxiv.org/abs/2303.13121
