Abstract
In order to address the scalability challenge within Neural Architecture Search (NAS), we speed up NAS training via dynamic hard example mining within a curriculum learning framework. By utilizing an autoencoder that enforces an image similarity embedding in latent space, we construct an efficient kd-tree structure to order images by furthest neighbour dissimilarity in a low-dimensional embedding. From a given query image from our subsample dataset, we can identify the most dissimilar image within the global dataset in logarithmic time. Via curriculum learning, we then dynamically re-formulate an unbiased subsample dataset for NAS optimisation, upon which the current NAS solution architecture performs poorly. We show that our DDS-NAS framework speeds up gradient-based NAS strategies by up to 27x without loss in performance. By maximising the contribution of each image sample during training, we reduce the duration of a NAS training cycle and the number of iterations required for convergence.
Abstract (translated)
为了应对神经架构搜索（NAS）中的可扩展性挑战，我们通过在课程学习框架内进行动态难例挖掘来加快NAS训练速度。利用一个自编码器强制执行图像相似性的嵌入表示，我们在潜在空间中构建了一个高效的kd树结构，以便按照低维嵌入中的最远邻域差异对图像进行排序。从我们的子样本数据集中给定的查询图像出发，我们可以在对数时间内识别全局数据集中与之最为不同的图像。通过课程学习，我们动态地重新定义一个无偏的子样本数据集供NAS优化使用，在这个数据集上当前的NAS解决方案架构表现不佳。我们展示了DDS-NAS框架能够将基于梯度的NAS策略的速度提高多达27倍，并且不会影响性能。通过最大化每个图像样本在训练过程中的贡献，我们将NAS训练周期的时间和收敛所需的迭代次数都减少了。
URL
https://arxiv.org/abs/2506.14667
