Abstract
Modern computer vision requires balancing predictive accuracy with real-time efficiency, yet the high inference cost of large vision models (LVMs) limits deployment on resource-constrained edge devices. Although Evolutionary Neural Architecture Search (ENAS) is well suited for multi-objective optimization, its practical use is hindered by two issues: expensive candidate evaluation and ranking inconsistency among subnetworks. To address them, we propose EvoNAS, an efficient distributed framework for multi-objective evolutionary architecture search. We build a hybrid supernet that integrates Vision State Space and Vision Transformer (VSS-ViT) modules, and optimize it with a Cross-Architecture Dual-Domain Knowledge Distillation (CA-DDKD) strategy. By coupling the computational efficiency of VSS blocks with the semantic expressiveness of ViT modules, CA-DDKD improves the representational capacity of the shared supernet and enhances ranking consistency, enabling reliable fitness estimation during evolution without extra fine-tuning. To reduce the cost of large-scale validation, we further introduce a Distributed Multi-Model Parallel Evaluation (DMMPE) framework based on GPU resource pooling and asynchronous scheduling. Compared with conventional data-parallel evaluation, DMMPE improves efficiency by over 70% through concurrent multi-GPU, multi-model execution. Experiments on COCO, ADE20K, KITTI, and NYU-Depth v2 show that the searched architectures, termed EvoNets, consistently achieve Pareto-optimal trade-offs between accuracy and efficiency. Compared with representative CNN-, ViT-, and Mamba-based models, EvoNets deliver lower inference latency and higher throughput under strict computational budgets while maintaining strong generalization on downstream tasks such as novel view synthesis. Code is available at this https URL
Abstract (translated)
现代计算机视觉需要在预测精度与实时效率之间取得平衡，但大型视觉模型（LVMs）高昂的推理成本限制了其在资源受限的边缘设备上的部署。尽管进化神经架构搜索（ENAS）非常适合多目标优化，但其实际应用受限于两个问题：候选架构评估成本高昂，以及子网络间排名不一致。为解决这些问题，我们提出了EvoNAS，一个高效的多目标进化架构搜索分布式框架。我们构建了一个融合视觉状态空间与视觉Transformer（VSS-ViT）模块的混合超网络，并采用跨架构双域知识蒸馏（CA-DDKD）策略对其进行优化。通过耦合VSS模块的计算效率与ViT模块的语义表达能力，CA-DDKD提升了共享超网络的表征能力并增强了排名一致性，从而在无需额外微调的情况下实现进化过程中可靠的适应度估计。为降低大规模验证成本，我们进一步引入了基于GPU资源池与异步调度的分布式多模型并行评估（DMMPE）框架。相较于传统的数据并行评估，DMMPE通过多GPU、多模型并发执行将效率提升超过70%。在COCO、ADE20K、KITTI和NYU-Depth v2上的实验表明，所搜索的架构（称为EvoNets）持续实现了精度与效率之间的帕累托最优权衡。与代表性的CNN、ViT和Mamba基础模型相比，EvoNets在严格的计算预算下实现了更低的推理延迟和更高的吞吐量，同时在新视角合成等下游任务中保持了强大的泛化能力。代码已公开于此https链接。
URL
https://arxiv.org/abs/2603.19563
