Abstract
Multi-task neural architecture search (NAS) enables transferring architectural knowledge among different tasks. However, ranking disorder between the source task and the target task degrades the architecture performance on the downstream task. We propose KTNAS, an evolutionary cross-task NAS algorithm, to enhance transfer efficiency. Our data-agnostic method converts neural architectures into graphs and uses architecture embedding vectors for the subsequent architecture performance prediction. The concept of transfer rank, an instance-based classifier, is introduced into KTNAS to address the performance degradation issue. We verify the search efficiency on NASBench-201 and transferability to various vision tasks on Micro TransNAS-Bench-101. The scalability of our method is demonstrated on DARTs search space including CIFAR-10/100, MNIST/Fashion-MNIST, MedMNIST. Experimental results show that KTNAS outperforms peer multi-task NAS algorithms in search efficiency and downstream task performance. Ablation studies demonstrate the vital importance of transfer rank for transfer performance.
Abstract (translated)
多任务神经架构搜索（NAS）使不同任务之间的网络结构知识得以转移。然而，源任务与目标任务间的排名顺序混乱会降低在下游任务上构建的架构性能。我们提出了KTNAS，一种进化型跨任务NAS算法，以提高迁移效率。我们的数据无关方法将神经网络结构转换为图，并使用架构嵌入向量来进行后续的架构性能预测。为了应对性能下降问题，我们在KTNAS中引入了基于实例分类器的概念“转移秩”。我们已经在NASBench-201上验证了搜索效率，在Micro TransNAS-Bench-101上的各种视觉任务上验证了可迁移性，并在包括CIFAR-10/100、MNIST/Fashion-MNIST和MedMNIST在内的DARTS搜索空间中展示了方法的扩展能力。实验结果显示，KTNAS在搜索效率和下游任务性能方面优于同行中的多任务NAS算法。消融研究表明，转移秩对于迁移性能至关重要。
URL
https://arxiv.org/abs/2504.00772
