Abstract
Although deep neural networks have facilitated significant progress of neural vocoders in recent years, they usually suffer from intrinsic challenges like opaque modeling, inflexible retraining under different input configurations, and parameter-performance trade-off. These inherent hurdles can heavily impede the development of this field. To resolve these problems, in this paper, we propose a novel neural vocoder in the time-frequency (T-F) domain. Specifically, we bridge the connection between the classical range-null decomposition (RND) theory and the vocoder task, where the reconstruction of the target spectrogram is formulated into the superimposition between range-space and null-space. The former aims to project the representation in the original mel-domain into the target linear-scale domain, and the latter can be instantiated via neural networks to further infill the spectral details. To fully leverage the spectrum prior, an elaborate dual-path framework is devised, where the spectrum is hierarchically encoded and decoded, and the cross- and narrow-band modules are leveraged for effectively modeling along sub-band and time dimensions. To enable inference under various configurations, we propose a simple yet effective strategy, which transforms the multi-condition adaption in the inference stage into the data augmentation in the training stage. Comprehensive experiments are conducted on various benchmarks. Quantitative and qualitative results show that while enjoying lightweight network structure and scalable inference paradigm, the proposed framework achieves state-ofthe-art performance among existing advanced methods. Code is available at this https URL.
Abstract (translated)
尽管深度神经网络在过去几年中促进了神经声码器的显著进步，但它们通常会遇到诸如模型不透明、在不同输入配置下灵活重训练困难以及参数性能权衡等内在挑战。这些固有的障碍可能会严重阻碍这一领域的发展。为了解决这些问题，在本文中我们提出了一种基于时间-频率（T-F）域的新颖神经声码器。具体来说，我们将经典的范围空-零空间分解（RND）理论与声码器任务之间的连接建立了起来，并将目标频谱图的重建公式化为范围空间和零空间的叠加。前者旨在将原始梅尔域中的表示投影到目标线性尺度域中，而后者可以通过神经网络进一步填充频谱细节。为了充分利用频谱先验知识，我们设计了一个精致的双路径框架，在该框架下频谱被分层编码和解码，并利用跨频段和窄带模块沿子频段和时间维度进行有效建模。为实现在各种配置下的推断，我们提出了一种简单而有效的策略，将推理阶段中的多条件适应转换为训练阶段中的数据增强。我们在多个基准上进行了全面的实验。定量和定性结果表明，在享受轻量级网络结构和可扩展推断范式的同时，所提出的框架在现有的先进方法中实现了最佳性能。代码可在提供的链接获取。
URL
https://arxiv.org/abs/2603.08574
