Abstract
Regenerating singing voices with altered lyrics while preserving melody consistency remains challenging, as existing methods either offer limited controllability or require laborious manual alignment. We propose YingMusic-Singer, a fully diffusion-based model enabling melody-controllable singing voice synthesis with flexible lyric manipulation. The model takes three inputs: an optional timbre reference, a melody-providing singing clip, and modified lyrics, without manual alignment. Trained with curriculum learning and Group Relative Policy Optimization, YingMusic-Singer achieves stronger melody preservation and lyric adherence than Vevo2, the most comparable baseline supporting melody control without manual alignment. We also introduce LyricEditBench, the first benchmark for melody-preserving lyric modification evaluation. The code, weights, benchmark, and demos are publicly available at this https URL.
Abstract (translated)
在保持旋律一致性的同时重新生成歌词已改唱的歌声仍具挑战性，因为现有方法要么可控性有限，要么需要繁琐的手动对齐。我们提出了YingMusic-Singer，一个完全基于扩散的模型，能够实现旋律可控的歌声合成与灵活的歌词编辑。该模型接受三个输入：可选的音色参考、提供旋律的歌声片段以及修改后的歌词，无需手动对齐。通过课程学习与组相对策略优化进行训练，YingMusic-Singer在旋律保持和歌词贴合度上均优于Vevo2——这是目前唯一支持无需手动对齐的旋律可控基线模型。我们还引入了LyricEditBench，这是首个用于评估旋律保持型歌词修改的基准测试。代码、权重、基准测试及演示均已在以下链接公开。
URL
https://arxiv.org/abs/2603.24589
