Abstract
Architecture design and optimization are challenging problems in the field of artificial neural networks. Working in this context, we here present SPARCS (SPectral ARchiteCture Search), a novel architecture search protocol which exploits the spectral attributes of the inter-layer transfer matrices. SPARCS allows one to explore the space of possible architectures by spanning continuous and differentiable manifolds, thus enabling for gradient-based optimization algorithms to be eventually employed. With reference to simple benchmark models, we show that the newly proposed method yields a self-emerging architecture with a minimal degree of expressivity to handle the task under investigation and with a reduced parameter count as compared to other viable alternatives.
Abstract (translated)
在人工神经网络领域，架构设计和优化是极具挑战性的问题。在此背景下，我们提出了SPARCS（SPectral ARchiteCture Search），这是一种新颖的架构搜索协议，利用了层间转移矩阵的谱属性。通过探索连续且可微流形的空间，SPARCS使得基于梯度的优化算法能够最终被采用。参考简单的基准模型，我们展示了新提出的方法可以生成一种具有处理任务所需最小表达力和参数量较少的自出现架构，相较于其他可行选项而言更为优越。
URL
https://arxiv.org/abs/2504.00885
