Abstract
Voice conversion is the task to transform voice characteristics of source speech while preserving content information. Nowadays, self-supervised representation learning models are increasingly utilized in content extraction. However, in these representations, a lot of hidden speaker information leads to timbre leakage while the prosodic information of hidden units lacks use. To address these issues, we propose a novel framework for expressive voice conversion called "SAVC" based on soft speech units from HuBert-soft. Taking soft speech units as input, we design an attribute encoder to extract content and prosody features respectively. Specifically, we first introduce statistic perturbation imposed by adversarial style augmentation to eliminate speaker information. Then the prosody is implicitly modeled on soft speech units with knowledge distillation. Experiment results show that the intelligibility and naturalness of converted speech outperform previous work.
Abstract (translated)
语音转换是将原始语音的语音特征进行转换，同时保留内容信息的过程。如今，自监督表示学习模型在内容提取中越来越受到欢迎。然而，在这些表示中，许多隐藏的说话者信息导致谐波泄漏，而隐藏单元的语调信息则缺乏使用。为了解决这些问题，我们提出了一种名为"SAVC"的新框架，基于HuBert-soft中的软语音单位。作为输入，我们设计了一个属性编码器来提取内容特征和语调特征。具体来说，我们首先引入了由对抗风格增强带来的统计畸变，以消除说话者信息。然后，我们通过知识蒸馏在软语音单位上隐含了语调信息。实验结果表明，转换后的语音的可听性和自然性超过了之前的 work。
URL
https://arxiv.org/abs/2405.00603
