Abstract
Target speaker extraction (TSE) aims to recover a target speaker's speech from a mixture using a reference utterance as a cue. Most TSE systems adopt conditional auto-encoder architectures with one-step inference. Inspired by test-time scaling, we propose a training-free multi-step inference method that enables iterative refinement with a frozen pretrained model. At each step, new candidates are generated by interpolating the original mixture and the previous estimate, and the best candidate is selected for further refinement until convergence. Experiments show that, when ground-truth target speech is available, optimizing an intrusive metric (SI-SDRi) yields consistent gains across multiple evaluation metrics. Without ground truth, optimizing non-intrusive metrics (UTMOS or SpkSim) improves the corresponding metric but may hurt others. We therefore introduce joint metric optimization to balance these objectives, enabling controllable extraction preferences for practical deployment.
Abstract (translated)
目标说话人提取（TSE）旨在通过使用参考语音作为线索，从混合信号中恢复出目标说话人的语音。大多数TSE系统采用条件自动编码器架构，并进行一步推理。受测试时间缩放的启发，我们提出了一种无需训练的多步推理方法，该方法能够利用冻结的预训练模型进行迭代细化。在每一步中，通过插值原始混合信号和前一次估计结果生成新的候选语音，然后选择最佳候选者继续进行进一步优化直至收敛。 实验表明，在有真实目标语音的情况下，优化侵入性指标（SI-SDRi）可以跨多个评估标准取得一致的改进效果。而在没有真实数据时，虽然通过非侵入性指标（UTMOS或SpkSim）进行优化能提高对应指标的表现，但可能会损害其他指标的效果。因此，我们引入了联合度量优化方法以平衡这些目标，使实用部署中可以实现可控提取偏好。
URL
https://arxiv.org/abs/2603.10921
