Abstract
Precise control over speech characteristics, such as pitch, duration, and speech rate, remains a significant challenge in the field of voice conversion. The ability to manipulate parameters like pitch and syllable rate is an important element for effective identity conversion, but can also be used independently for voice transformation, achieving goals that were historically addressed by vocoder-based methods. In this work, we explore a convolutional neural network-based approach that aims to provide means for modifying fundamental frequency (F0), phoneme sequences, intensity, and speaker identity. Rather than relying on disentanglement techniques, our model is explicitly conditioned on these factors to generate mel spectrograms, which are then converted into waveforms using a universal neural vocoder. Accordingly, during inference, F0 contours, phoneme sequences, and speaker embeddings can be freely adjusted, allowing for intuitively controlled voice transformations. We evaluate our approach on speaker conversion and expressive speech tasks using both perceptual and objective metrics. The results suggest that the proposed method offers substantial flexibility, while maintaining high intelligibility and speaker similarity.
Abstract (translated)
在语音转换领域，对音高、持续时间及语速等语音特征的精确控制仍然是一个重大挑战。操控音高和音节率这类参数的能力是有效身份转换的重要元素，但也可以独立使用这些参数进行声音变换，以实现以往基于声码器的方法才能达到的目标。本文提出了一种基于卷积神经网络的方法，旨在提供修改基频（F0）、音素序列、强度及说话人身份的手段。与依赖于特征解耦技术不同的是，我们的模型直接根据这些因素生成梅尔光谱图，并使用通用神经声码器将其转换为波形信号。因此，在推理过程中可以自由调整F0轮廓、音素序列和说话人嵌入，从而实现直观可控的声音变换。 我们在说话人转换及表情语音任务上采用感知与客观指标对我们的方法进行了评估。实验结果表明，所提出的方法在保持高清晰度和说话人口感相似性的前提下，提供了相当大的灵活性。
URL
https://arxiv.org/abs/2507.04817
