Abstract
We propose a novel, brain-inspired deep neural network model known as the Deep Oscillatory Neural Network (DONN). Deep neural networks like the Recurrent Neural Networks indeed possess sequence processing capabilities but the internal states of the network are not designed to exhibit brain-like oscillatory activity. With this motivation, the DONN is designed to have oscillatory internal dynamics. Neurons of the DONN are either nonlinear neural oscillators or traditional neurons with sigmoidal or ReLU activation. The neural oscillator used in the model is the Hopf oscillator, with the dynamics described in the complex domain. Input can be presented to the neural oscillator in three possible modes. The sigmoid and ReLU neurons also use complex-valued extensions. All the weight stages are also complex-valued. Training follows the general principle of weight change by minimizing the output error and therefore has an overall resemblance to complex backpropagation. A generalization of DONN to convolutional networks known as the Oscillatory Convolutional Neural Network is also proposed. The two proposed oscillatory networks are applied to a variety of benchmark problems in signal and image/video processing. The performance of the proposed models is either comparable or superior to published results on the same data sets.
Abstract (translated)
我们提出了一个新型的基于脑的深度神经网络模型，称为深度 oscillatory 神经网络（DONN）。与递归神经网络这样的深度神经网络确实具有序列处理能力，但是网络内部状态并没有被设计成具有类似脑的周期性活动。为了实现这一点，DONN 被设计具有周期性的内部动态。DONN 的神经元可以是非线性神经元周期器或具有sigmoidal 或 ReLU 激活的传统神经元。在模型中使用的神经元是 Hopf oscillator，其动力学在复数域中描述。输入可以通过三种可能的模式呈现给神经元振荡器。sigmoid 和 ReLU 神经元也使用了复数值扩展。所有权重阶段也是复数值。训练通过最小化输出误差来遵循一般原则，因此具有与复杂反向传播相似的整体外观。也提出了将 DONN 扩展到卷积网络的振荡卷积神经网络的一般化模型。两个提出的振荡网络被应用于各种信号和图像/视频处理基准问题。所提出模型的性能与相同数据集中的已发布结果相比要么相当，要么更优。
URL
https://arxiv.org/abs/2405.03725
