Abstract
Hardware-aware Neural Architecture Search approaches (HW-NAS) automate the design of deep learning architectures, tailored specifically to a given target hardware platform. Yet, these techniques demand substantial computational resources, primarily due to the expensive process of assessing the performance of identified architectures. To alleviate this problem, a recent direction in the literature has employed representation similarity metric for efficiently evaluating architecture performance. Nonetheless, since it is inherently a single objective method, it requires multiple runs to identify the optimal architecture set satisfying the diverse hardware cost constraints, thereby increasing the search cost. Furthermore, simply converting the single objective into a multi-objective approach results in an under-explored architectural search space. In this study, we propose a Multi-Objective method to address the HW-NAS problem, called MO-HDNAS, to identify the trade-off set of architectures in a single run with low computational cost. This is achieved by optimizing three objectives: maximizing the representation similarity metric, minimizing hardware cost, and maximizing the hardware cost diversity. The third objective, i.e. hardware cost diversity, is used to facilitate a better exploration of the architecture search space. Experimental results demonstrate the effectiveness of our proposed method in efficiently addressing the HW-NAS problem across six edge devices for the image classification task.
Abstract (translated)
硬件感知的神经架构搜索方法（HW-NAS）自动设计适用于特定目标硬件平台的深度学习架构。然而，这些技术需要大量的计算资源，主要原因是确定架构性能的过程代价昂贵。为解决这个问题，文献中最近的一个方向采用表示相似性度量来高效评估架构性能。然而，由于它本质上是一个单目标方法，因此需要多次运行来找到满足多样硬件成本约束的最优架构集合，从而增加搜索成本。此外，将单目标转换为多目标方法导致了一个未被探索的建筑搜索空间。在本研究中，我们提出了一种多目标方法来解决HW-NAS问题，称为MO-HDNAS，以在低计算成本的单次运行中识别架构的权衡集。这是通过优化三个目标来实现的：最大化表示相似性度量，最小化硬件成本，最大化硬件成本多样性。第三个目标，即硬件成本多样性，用于促进更好地探索架构搜索空间。实验结果表明，我们提出的方法在有效地解决图像分类任务的六个边缘设备上的HW-NAS问题方面非常有效。
URL
https://arxiv.org/abs/2404.12403
