Abstract
Large Audio-Language Models (LALMs) have demonstrated remarkable performance in end-to-end speaker diarization and recognition. However, their speaker discriminability remains limited due to the scarcity of large-scale conversational data and the absence of explicit speaker representation optimization. To address this, we propose GLSC-SDR, a paradigm that jointly trains speaker classification with diarization and recognition. We further introduce a Global-Local Speaker Classification strategy, which uses clustered speakers as global labels and re-encoded intra-cluster speakers as local labels. This hierarchical design enhances fine-grained speaker discrimination while preserving semantic transcription accuracy. Experiments on AliMeeting, AISHELL-4, and AMI-SDM demonstrate that GLSC-SDR achieves competitive or superior performance compared to simulation-based and multi-encoder approaches, without relying on large-scale real conversational data.
Abstract (translated)
大型音频语言模型（LALMs）在端到端说话人分离与识别任务中展现出卓越性能。然而，由于大规模对话数据的稀缺以及缺乏显式说话人表征优化，其说话人区分能力仍较为有限。针对这一问题，我们提出了GLSC-SDR范式，通过联合训练说话人分类、分离与识别任务。进一步引入全局-局部说话人分类策略，利用聚类后的说话人作为全局标签，并对簇内说话人进行重编码作为局部标签。该分层设计在保持语义转写准确性的同时增强了细粒度说话人区分能力。在AliMeeting、AISHELL-4和AMI-SDM数据集上的实验表明，GLSC-SDR无需依赖大规模真实对话数据，即可达到与基于模拟方法及多编码器方法相当或更优的性能。
URL
https://arxiv.org/abs/2603.25377
