Abstract
We propose the use of Non-Negative Autoencoders (NAEs) for sound deconstruction and user-guided manipulation of sounds for creative purposes. NAEs offer a versatile and scalable extension of traditional Non-Negative Matrix Factorization (NMF)-based approaches for interpretable audio decomposition. By enforcing non-negativity constraints through projected gradient descent, we obtain decompositions where internal weights and activations can be directly interpreted as spectral shapes and temporal envelopes, and where components can themselves be listened to as individual sound events. In particular, multi-layer Deep NAE architectures enable hierarchical representations with an adjustable level of granularity, allowing sounds to be deconstructed at multiple levels of abstraction: from high-level note envelopes down to fine-grained spectral details. This framework enables a wide new range of expressive, controllable, and randomized sound transformations. We introduce novel manipulation operations including cross-component and cross-layer synthesis, hierarchical deconstructions, and several randomization strategies that control timbre and event density. Through visualizations and resynthesis of practical examples, we demonstrate how NAEs can serve as flexible and interpretable tools for object-based sound editing.
Abstract (translated)
我们提出使用非负自动编码器（NAE）来进行声音分解，并允许用户引导式的音效操控，以满足创意需求。NAEs为传统的基于非负矩阵因式分解（NMF）的方法提供了一种灵活且可扩展的扩展，使得音频分解具有解释性。通过投影梯度下降方法实施非负约束，我们可以获得一种分解形式，在这种形式下内部权重和激活可以直接解释为频谱形状和时域包络，并且这些组件本身可以作为单独的声音事件被听取。特别是，多层深层NAE架构能够提供可调节颗粒度的分层表示，允许声音在多个抽象级别上进行拆解：从高层次音符包络到细微的频谱细节。这种框架使得一系列表达丰富、可控性强和随机化的声效转换成为可能。 我们引入了一些新的操作方法，包括跨组件和跨层次合成、分层分解以及几种控制音色和事件密度的随机化策略。通过实际案例的可视化与重合生成演示，我们展示了NAEs如何作为灵活且具有解释性的工具用于基于对象的声音编辑中。
URL
https://arxiv.org/abs/2510.08816
