Abstract
Neural architecture search (NAS) automates the discovery of neural networks that meet specified criteria, yet its evaluation procedures are often hardcoded, limiting the ability to introduce new metrics. This issue is especially pronounced in hardware-aware NAS, where objectives depend on target devices such as edge hardware. To address this limitation, we propose SEval-NAS, a metric-evaluation mechanism that converts architectures to strings, embeds them as vectors, and predicts performance metrics. Using NATS-Bench and HW-NAS-Bench, we evaluated accuracy, latency, and memory. Kendall's $\tau$ correlations showed stronger latency and memory predictions than accuracy, indicating the suitability of SEval-NAS as a hardware cost predictor. We further integrated SEval-NAS into FreeREA to evaluate metrics not originally included. The method successfully ranked FreeREA-generated architectures, maintained search time, and required minimal algorithmic changes. Our implementation is available at: this https URL
Abstract (translated)
神经架构搜索（NAS）自动发现了符合特定标准的神经网络，但其评估过程通常是硬编码的，限制了引入新度量指标的能力。在硬件感知型NAS中，目标设备如边缘计算硬件的具体需求使得这一问题更加突出。 为了解决这个问题，我们提出了SEval-NAS，这是一种将架构转换为字符串、嵌入向量并预测性能指标的度量评估机制。使用NATS-Bench和HW-NAS-Bench进行了准确性、延迟和内存的评估。根据肯德尔的$\tau$相关性分析，对于延迟和内存的预测结果比准确性的预测更强，这表明SEval-NAS可以作为硬件成本预测的有效工具。 我们进一步将SEval-NAS集成到了FreeREA中，以评估原本未包含在内的度量指标。该方法成功地对FreeREA生成的架构进行了排序，并保持了搜索时间的同时只需要进行少量算法修改。 我们的实现代码可在以下链接获取：this https URL
URL
https://arxiv.org/abs/2603.00099
