Abstract
Personalized or target speech extraction (TSE) typically needs a clean enrollment -- hard to obtain in real-world crowded environments. We remove the essential need for enrollment by predicting, from the mixture itself, a small set of per-speaker embeddings that serve as the control signal for extraction. Our model maps a noisy mixture directly to a small set of candidate speaker embeddings trained to align with a strong single-speaker speaker-embedding space via permutation-invariant teacher supervision. On noisy LibriMix, the resulting embeddings form a structured and clusterable identity space, outperforming WavLM+K-means and separation-derived embeddings in standard clustering metrics. Conditioning these embeddings into multiple extraction back-ends consistently improves objective quality and intelligibility, and generalizes to real DNS-Challenge recordings.
Abstract (translated)
个性化或目标语音提取（TSE）通常需要干净的注册语音——这在真实嘈杂环境中难以获取。我们通过直接从混合音频中预测一小部分说话人嵌入向量来消除对注册语音的依赖，这些嵌入向量作为提取的控制信号。我们的模型将含噪混合音频直接映射为一组候选说话人嵌入向量，这些向量通过置换不变的教师监督进行训练，以与强单说话人嵌入空间对齐。在含噪的LibriMix数据集上，生成的嵌入向量形成了一个结构化且可聚类的身份空间，在标准聚类指标上优于WavLM+K-means以及从分离结果导出的嵌入向量。将这些嵌入向量输入多个提取后端，能持续提升客观质量和可懂度，并泛化到真实的DNS-Challenge录音数据。
URL
https://arxiv.org/abs/2604.03219
