Abstract
This paper presents SelfTTS, a text-to-speech (TTS) model designed for cross-speaker style transfer that eliminates the need for external pre-trained speaker or emotion encoders. The architecture achieves emotional expressivity in neutral speakers through an explicit disentanglement strategy utilizing Gradient Reversal Layers (GRL) combined with cosine similarity loss to decouple speaker and emotion information. We introduce Multi Positive Contrastive Learning (MPCL) to induce clustered representations of speaker and emotion embeddings based on their respective labels. Furthermore, SelfTTS employs a self-refinement strategy via Self-Augmentation, exploiting the model's voice conversion capabilities to enhance the naturalness of synthesized speech. Experimental results demonstrate that SelfTTS achieves superior emotional naturalness (eMOS) and robust stability in target timbre and emotion compared to state-of-the-art baselines.
Abstract (translated)
本文提出SelfTTS，一种旨在实现跨说话人风格迁移的文本转语音（TTS）模型，该模型无需依赖外部预训练的说话人或情感编码器。其架构通过结合梯度反转层（GRL）与余弦相似度损失的显式解耦策略，将说话人与情感信息分离，从而在中性说话人中实现情感表达性。我们引入多正例对比学习（MPCL），基于说话人与情感的标签诱导其嵌入表示的聚类。此外，SelfTTS采用通过自增强（Self-Augmentation）实现的自我精炼策略，利用模型的语音转换能力提升合成语音的自然度。实验结果表明，与最先进的基线方法相比，SelfTTS在目标音色与情感控制上具有更优的情感自然度（eMOS）和稳健稳定性。
URL
https://arxiv.org/abs/2603.22252
