Abstract
Multi-talker overlapped speech poses a significant challenge for speech recognition and diarization. Recent research indicated that these two tasks are inter-dependent and complementary, motivating us to explore a unified modeling method to address them in the context of overlapped speech. A recent study proposed a cost-effective method to convert a single-talker automatic speech recognition (ASR) system into a multi-talker one, by inserting a Sidecar separator into the frozen well-trained ASR model. Extending on this, we incorporate a diarization branch into the Sidecar, allowing for unified modeling of both ASR and diarization with a negligible overhead of only 768 parameters. The proposed method yields better ASR results compared to the baseline on LibriMix and LibriSpeechMix datasets. Moreover, without sophisticated customization on the diarization task, our method achieves acceptable diarization results on the two-speaker subset of CALLHOME with only a few adaptation steps.
Abstract (translated)
多说话者重叠语音对语音识别和去噪构成了一个重要的挑战。最近的研究表明，这两个任务是相互依赖和互补的，因此我们可以考虑一种统一建模方法，在重叠语音的背景下解决这些问题。一项最近的研究提出了一种成本效益高的新方法，通过将 Sidecar 分离器插入已经训练好的单说话者自动语音识别(ASR)模型中，可以将 ASR 和去噪统一建模，而只需要很少的参数 overhead。 proposed 方法在 Libri 混合和 LibriSpeech 混合数据集上的 ASR 结果比基准更好。此外，在没有复杂的去噪任务定制的情况下，我们的方法和只需要几个适应步骤就可以在 Home 电话中实现可接受去噪结果。
URL
https://arxiv.org/abs/2305.16263
