Abstract
This paper presents a neural architecture search method based on Transformer architecture, searching cross multihead attention computation ways for different number of encoder and decoder combinations. In order to search for neural network structures with better translation results, we considered perplexity as an auxiliary evaluation metric for the algorithm in addition to BLEU scores and iteratively improved each individual neural network within the population by a multi-objective genetic algorithm. Experimental results show that the neural network structures searched by the algorithm outperform all the baseline models, and that the introduction of the auxiliary evaluation metric can find better models than considering only the BLEU score as an evaluation metric.
Abstract (translated)
本文提出了一种基于Transformer架构的神经架构搜索方法，该方法探索了不同编码器和解码器组合数量下的多头注意力计算方式。为了寻找翻译效果更好的神经网络结构，在算法中除了使用BLEU分数之外，还考虑了困惑度（perplexity）作为辅助评估指标，并通过一个多目标遗传算法迭代优化种群中的每个个体神经网络。实验结果表明，该算法搜索到的神经网络结构优于所有基线模型，而且引入辅助评估指标能够找到比仅依据BLEU分数评估更好的模型。
URL
https://arxiv.org/abs/2505.01314
