Abstract
We propose a method for the task of text-conditioned speech insertion, i.e. inserting a speech sample in an input speech sample, conditioned on the corresponding complete text transcript. An example use case of the task would be to update the speech audio when corrections are done on the corresponding text transcript. The proposed method follows a transformer-based non-autoregressive approach that allows speech insertions of variable lengths, which are dynamically determined during inference, based on the text transcript and tempo of the available partial input. It is capable of maintaining the speaker's voice characteristics, prosody and other spectral properties of the available speech input. Results from our experiments and user study on LibriTTS show that our method outperforms baselines based on an existing adaptive text to speech method. We also provide numerous qualitative results to appreciate the quality of the output from the proposed method.
Abstract (translated)
我们提出了一种基于文本条件的语音插入方法，即在输入语音样本中插入一段新的语音样本，该操作依据对应的完整文本转录本进行。一个实际的应用场景是当对相应的文本转录本进行了修改时，更新音频中的相应部分。 所提出的这种方法采用了一个基于Transformer的非自回归（non-autoregressive）框架，允许根据文本转录本和现有不完整输入片段的速度，在推理过程中动态确定不同长度的语音插入。该方法能够保持说话人的音色、语调以及其他频谱属性，确保输出语音与原始语音样本的高度一致性。 我们通过在LibriTTS数据集上的实验及用户研究证明了所提出的方法优于基于现有自适应文本到语音转换技术的基线模型。此外，我们也提供了大量定性结果以展示该方法生成输出的质量。
URL
https://arxiv.org/abs/2508.17031
