Abstract
This paper introduces PFlow-VC, a conditional flow matching voice conversion model that leverages fine-grained discrete pitch tokens and target speaker prompt information for expressive voice conversion (VC). Previous VC works primarily focus on speaker conversion, with further exploration needed in enhancing expressiveness (such as prosody and emotion) for timbre conversion. Unlike previous methods, we adopt a simple and efficient approach to enhance the style expressiveness of voice conversion models. Specifically, we pretrain a self-supervised pitch VQVAE model to discretize speaker-irrelevant pitch information and leverage a masked pitch-conditioned flow matching model for Mel-spectrogram synthesis, which provides in-context pitch modeling capabilities for the speaker conversion model, effectively improving the voice style transfer capacity. Additionally, we improve timbre similarity by combining global timbre embeddings with time-varying timbre tokens. Experiments on unseen LibriTTS test-clean and emotional speech dataset ESD show the superiority of the PFlow-VC model in both timbre conversion and style transfer. Audio samples are available on the demo page this https URL.
Abstract (translated)
这篇论文介绍了PFlow-VC，这是一种条件流匹配声音转换模型，利用细粒度的离散音高令牌和目标说话人的提示信息来进行富有表现力的声音转换（Voice Conversion, VC）。之前的声音转换工作主要集中在说话人转换上，在提升音色转换中的表达性方面（如韵律和情感）还有待进一步探索。与以往的方法不同，我们采用了一种简单且高效的方式来增强声音转换模型的风格表达能力。具体来说，我们预先训练了一个自我监督的音高VQVAE模型来离散化与说话人无关的音高信息，并利用一个遮蔽音高标准匹配流合成模型进行梅尔频谱图生成，这为说话人转换模型提供了上下文中的音高建模能力，有效地提升了语音风格传输的能力。此外，我们通过结合全局音色嵌入与时变音色调料来改进音色相似度。 在未见过的LibriTTS测试集（test-clean）和情感语音数据集ESD上的实验表明，PFlow-VC模型在音色转换和风格迁移方面均表现出优越性。音频样本可在该演示页面上找到：[此URL]（请将方括号中的文本替换为实际提供的链接）。
URL
https://arxiv.org/abs/2502.05471
