Abstract
Robustness is critical in zero-shot singing voice conversion (SVC). This paper introduces two novel methods to strengthen the robustness of the kNN-VC framework for SVC. First, kNN-VC's core representation, WavLM, lacks harmonic emphasis, resulting in dull sounds and ringing artifacts. To address this, we leverage the bijection between WavLM, pitch contours, and spectrograms to perform additive synthesis, integrating the resulting waveform into the model to mitigate these issues. Second, kNN-VC overlooks concatenative smoothness, a key perceptual factor in SVC. To enhance smoothness, we propose a new distance metric that filters out unsuitable kNN candidates and optimize the summing weights of the candidates during inference. Although our techniques are built on the kNN-VC framework for implementation convenience, they are broadly applicable to general concatenative neural synthesis models. Experimental results validate the effectiveness of these modifications in achieving robust SVC. Demo: this http URL Code: this https URL
Abstract (translated)
在零样本歌声转换（Singing Voice Conversion，简称SVC）中，鲁棒性至关重要。本文介绍了两种新颖的方法来增强kNN-VC框架的鲁棒性。首先，kNN-VC的核心表示WavLM缺乏谐波强调，导致声音单调且有振铃效应。为了解决这个问题，我们利用了WavLM、音高轮廓和频谱图之间的双射关系来进行加法合成，并将生成的波形整合进模型中以减轻这些问题。 其次，kNN-VC忽略了连贯平滑性，这是SVC中的关键感知因素之一。为了增强这种平滑度，我们提出了一种新的距离度量方法来过滤掉不合适的k近邻候选者，并在推理过程中优化这些候选者的加权和。尽管我们的技术基于kNN-VC框架以方便实现，但它们广泛适用于一般的连贯神经合成模型。 实验结果验证了这些修改在实现鲁棒性SVC方面的有效性。 演示：[此处填写链接] 代码：[此处填写链接]
URL
https://arxiv.org/abs/2504.05686
