Abstract
The goal of this work is to investigate what music source separation approaches based on neural networks learn from the data. We examine the mapping functions of neural networks that are based on the denoising autoencoder (DAE) model, and conditioned on the mixture magnitude spectra. For approximating the mapping functions, we propose an algorithm that is inspired by the knowledge distillation and is denoted as the neural couplings algorithm (NCA). The NCA yields a matrix that expresses the mapping of the mixture to the target source magnitude information. Using the NCA we examine the mapping functions of three fundamental DAE models in music source separation; one with single layer encoder and decoder, one with multi-layer encoder and single layer decoder, and one using the skip-filtering connections (SF) with a single encoding and decoding layer. We first train these models with realistic data to estimate the singing voice magnitude spectra from the corresponding mixture. We then use the optimized models and test spectral data as input to the NCA. Our experimental findings show that approaches based on the DAE model learn scalar filtering operators, exhibiting a predominant diagonal structure in their corresponding mapping functions, limiting the exploitation of inter-frequency structure of music data. In contrast, skip-filtering connections are shown to assist the DAE model in learning filtering operators that exploit richer inter-frequency structure.
Abstract (translated)
本文的目的是研究基于神经网络的音乐源分离方法从数据中学习到什么。我们研究了基于去噪自编码模型的神经网络的映射函数，并以混合幅度谱为条件。为了逼近映射函数，提出了一种基于知识蒸馏的神经耦合算法（NCA）。NCA生成一个矩阵，表示混合到目标震源震级信息的映射。利用NCA，我们研究了音乐源分离中三种基本的DAE模型的映射功能：一种采用单层编码器和解码器，一种采用多层编码器和单层解码器，另一种采用单层编码和解码层的跳过过滤连接（SF）。我们首先用实际数据训练这些模型，从相应的混合物中估计出声音的幅度谱。然后，我们使用优化模型和测试光谱数据作为NCA的输入。我们的实验结果表明，基于DAE模型的方法学习标量滤波算子，在相应的映射函数中显示出主要的对角结构，限制了音乐数据的频率间结构的利用。相比之下，跳过过滤连接显示有助于dae模型学习利用更丰富的频率间结构的过滤操作员。
URL
https://arxiv.org/abs/1904.06157
