Abstract
Neural Architecture Search (NAS) has emerged as a powerful approach for automating neural network design. However, existing NAS methods face critical limitations in real-world deployments: architectures lack adaptability across scenarios, each deployment context requires costly separate searches, and performance consistency across diverse platforms remains challenging. We propose DANCE (Dynamic Architectures with Neural Continuous Evolution), which reformulates architecture search as a continuous evolution problem through learning distributions over architectural components. DANCE introduces three key innovations: a continuous architecture distribution enabling smooth adaptation, a unified architecture space with learned selection gates for efficient sampling, and a multi-stage training strategy for effective deployment optimization. Extensive experiments across five datasets demonstrate DANCE's effectiveness. Our method consistently outperforms state-of-the-art NAS approaches in terms of accuracy while significantly reducing search costs. Under varying computational constraints, DANCE maintains robust performance while smoothly adapting architectures to different hardware requirements. The code and appendix can be found at this https URL.
Abstract (translated)
神经架构搜索（NAS）已经成为自动化设计神经网络的一种强有力的方法。然而，现有的NAS方法在实际部署中面临着关键的限制：架构缺乏跨场景的适应性，每个部署环境都需要进行代价高昂的独特搜索，并且在不同平台上保持性能一致性仍然具有挑战性。我们提出了DANCE（基于神经连续演化的动态架构），通过学习架构组件上的分布来将架构搜索重新表述为一个连续演进问题。DANCE引入了三个关键创新：一种能够实现平滑适应的连续架构分布，一种采用学习选择门进行高效采样的统一架构空间，以及一种多阶段训练策略以用于有效的部署优化。在五个数据集上进行了广泛的实验，证明了DANCE的有效性。我们的方法在精度方面持续超越最先进的NAS方法，并且大幅降低了搜索成本。在不同的计算约束条件下，DANCE能够维持稳健的性能，并平滑地调整架构以适应各种硬件需求。代码和附录可在以下链接找到：[此 URL]（请将“URL”替换为实际提供的链接）。
URL
https://arxiv.org/abs/2507.04671
