Abstract
Deep Neural Networks (DNNs) can be represented as graphs whose links and vertices iteratively process data and solve tasks sub-optimally. Complex Network Theory (CNT), merging statistical physics with graph theory, provides a method for interpreting neural networks by analysing their weights and neuron structures. However, classic works adapt CNT metrics that only permit a topological analysis as they do not account for the effect of the input data. In addition, CNT metrics have been applied to a limited range of architectures, mainly including Fully Connected neural networks. In this work, we extend the existing CNT metrics with measures that sample from the DNNs' training distribution, shifting from a purely topological analysis to one that connects with the interpretability of deep learning. For the novel metrics, in addition to the existing ones, we provide a mathematical formalisation for Fully Connected, AutoEncoder, Convolutional and Recurrent neural networks, of which we vary the activation functions and the number of hidden layers. We show that these metrics differentiate DNNs based on the architecture, the number of hidden layers, and the activation function. Our contribution provides a method rooted in physics for interpreting DNNs that offers insights beyond the traditional input-output relationship and the CNT topological analysis.
Abstract (translated)
深度神经网络（DNNs）可以表示为具有边和顶点递归处理数据和解决子优化问题的图。复杂网络理论（CNT）通过将统计物理学与图论相结合，提供了一种解释神经网络的方法，通过分析它们的权重和神经元结构。然而，经典的网络理论仅允许进行拓扑分析，因为它们没有考虑输入数据的影响。此外，CNT metrics 已应用于广泛的架构，主要包括完全连接神经网络。在这篇工作中，我们通过采样来自 DNNs 的训练分布来扩展现有的 CNT metrics，从纯粹的拓扑分析转变为一个与深度学习的可解释性相结合的分析。对于新 metrics，除了现有的 ones，我们为全连接神经网络、自编码器、卷积神经网络和循环神经网络提供了数学公式，其中我们改变激活函数和隐藏层数。我们证明了这些 metrics 根据架构、隐藏层数和激活函数区分 DNNs。我们的工作为基于物理的解释 DNNs 提供了一种方法，该方法不仅限于传统的输入-输出关系和 CNT topological analysis。
URL
https://arxiv.org/abs/2404.11172
