Abstract
Over the past several years progress in designing better neural network architectures for visual recognition has been substantial. To help sustain this rate of progress, in this work we propose to reexamine the methodology for comparing network architectures. In particular, we introduce a new comparison paradigm of distribution estimates, in which network design spaces are compared by applying statistical techniques to populations of sampled models, while controlling for confounding factors like network complexity. Compared to current methodologies of comparing point and curve estimates of model families, distribution estimates paint a more complete picture of the entire design landscape. As a case study, we examine design spaces used in neural architecture search (NAS). We find significant statistical differences between recent NAS design space variants that have been largely overlooked. Furthermore, our analysis reveals that the design spaces for standard model families like ResNeXt can be comparable to the more complex ones used in recent NAS work. We hope these insights into distribution analysis will enable more robust progress toward discovering better networks for visual recognition.
Abstract (translated)
在过去的几年中，为视觉识别设计更好的神经网络结构已经取得了实质性进展。为了帮助保持这一进展速度，在这项工作中，我们建议重新检查比较网络架构的方法。特别地，我们引入了一种新的分布估计比较范式，通过将统计技术应用于抽样模型的总体来比较网络设计空间，同时控制诸如网络复杂性之类的混杂因素。与当前比较模型族的点和曲线估计的方法相比，分布估计描绘了整个设计景观的更完整的画面。作为一个案例研究，我们研究了神经架构搜索（NAS）中使用的设计空间。我们发现，最近被忽视的NAS设计空间变体之间存在显著的统计差异。此外，我们的分析表明，像resnext这样的标准模型族的设计空间可以与最近NAS工作中使用的更复杂的设计空间相比较。我们希望这些对分布分析的洞察能使我们在发现更好的视觉识别网络方面取得更为强劲的进展。
URL
https://arxiv.org/abs/1905.13214
