Abstract
Differentiable Neural Architecture Search (NAS) provides efficient, gradient-based methods for automatically designing neural networks, yet its adoption remains limited in practice. We present MIDAS, a novel approach that modernizes DARTS by replacing static architecture parameters with dynamic, input-specific parameters computed via self-attention. To improve robustness, MIDAS (i) localizes the architecture selection by computing it separately for each spatial patch of the activation map, and (ii) introduces a parameter-free, topology-aware search space that models node connectivity and simplifies selecting the two incoming edges per node. We evaluate MIDAS on the DARTS, NAS-Bench-201, and RDARTS search spaces. In DARTS, it reaches 97.42% top-1 on CIFAR-10 and 83.38% on CIFAR-100. In NAS-Bench-201, it consistently finds globally optimal architectures. In RDARTS, it sets the state of the art on two of four search spaces on CIFAR-10. We further analyze why MIDAS works, showing that patchwise attention improves discrimination among candidate operations, and the resulting input-specific parameter distributions are class-aware and predominantly unimodal, providing reliable guidance for decoding.
Abstract (translated)
可微神经架构搜索（NAS）提供了一种自动设计神经网络的高效、基于梯度的方法，但其在实践中应用仍然有限。我们提出了MIDAS，这是一种通过用自注意力计算出的动态且特定于输入的参数来替换静态架构参数，从而对DARTS进行了现代化改进的新方法。为了提高鲁棒性，MIDAS（i）通过对激活图中的每个空间补丁分别进行架构选择来进行局部化，并（ii）引入了一种无参数、拓扑感知的搜索空间，该空间建模节点连接并简化了每个节点选择两个输入边的过程。 我们在DARTS、NAS-Bench-201和RDARTS搜索空间上评估了MIDAS。在DARTS中，它在CIFAR-10数据集上的顶级准确率为97.42%，在CIFAR-100数据集上的准确率为83.38%。在NAS-Bench-201中，它始终能找到全局最优架构。在RDARTS上，它在四个搜索空间中的两个中为CIFAR-10设定了最新的技术水平。 我们进一步分析了MIDAS为何有效：补丁级注意力提高了对候选操作的区分能力，并且生成的特定于输入的参数分布是类别感知的并且主要是单峰的，这为解码提供了可靠的指导。
URL
https://arxiv.org/abs/2602.17700
