Abstract
Neural Architecture Search (NAS) has gained widespread attention for its transformative potential in deep learning model design. However, the vast and complex search space of NAS leads to significant computational and time costs. Neural Architecture Generation (NAG) addresses this by reframing NAS as a generation problem, enabling the precise generation of optimal architectures for specific tasks. Despite its promise, mainstream methods like diffusion models face limitations in global search capabilities and are still hindered by high computational and time demands. To overcome these challenges, we propose Evolutionary Diffusion-based Neural Architecture Generation (EDNAG), a novel approach that achieves efficient and training-free architecture generation. EDNAG leverages evolutionary algorithms to simulate the denoising process in diffusion models, using fitness to guide the transition from random Gaussian distributions to optimal architecture distributions. This approach combines the strengths of evolutionary strategies and diffusion models, enabling rapid and effective architecture generation. Extensive experiments demonstrate that EDNAG achieves state-of-the-art (SOTA) performance in architecture optimization, with an improvement in accuracy of up to 10.45%. Furthermore, it eliminates the need for time-consuming training and boosts inference speed by an average of 50 times, showcasing its exceptional efficiency and effectiveness.
Abstract (translated)
神经架构搜索（NAS）因其在深度学习模型设计方面的变革潜力而备受关注。然而，NAS的巨大且复杂的搜索空间导致了显著的计算和时间成本问题。神经架构生成（NAG）通过将NAS重新定义为一个生成问题来解决这一难题，从而能够精确地为特定任务生成最优架构。尽管NAG展现出巨大前景，但主流方法如扩散模型仍然面临着全局搜索能力有限及高计算与时间需求的问题。 为了克服这些挑战，我们提出了进化式扩散神经架构生成（EDNAG），这是一种新型的方法，实现了高效且无需训练的架构生成。EDNAG利用进化算法模拟了扩散模型中的去噪过程，并通过适应度来引导随机高斯分布向最优架构分布转化。该方法结合了进化策略和扩散模型的优势，能够实现快速有效的架构生成。 广泛的实验表明，EDNAG在架构优化方面达到了最先进的性能（SOTA），其准确率提高了高达10.45%。此外，它消除了耗时的训练需求，并将推理速度平均提升了50倍，展示了其卓越的效率和有效性。
URL
https://arxiv.org/abs/2504.17827
