Abstract
Speech LLM-based ASR often struggles with named entities and long-tail words due to strong internal language-model priors. Retrieval-augmented biasing can help, but its effectiveness depends on accurate hotword localization in full-utterance speech under weak supervision. We propose CLAR, a dual-encoder speech-text retriever that uses Continuous Integrate-and-Fire (CIF) to learn monotonic token-level alignments without timestamps. With length-aware localized matching, CLAR anchors short-entity acoustic cues and reduces representation dilution and attention drift. The retriever is trained with a multi-granularity objective combining global and local segment-level contrastive losses and a CIF quantity constraint. At inference, top-ranked hotwords are injected as contextual prompts for the Speech LLM, improving recognition without shallow fusion. Experiments show that CLAR significantly improves hotword retrieval and reduces both CER and B-WER against strong contextual ASR baselines.
Abstract (translated)
基于语音大语言模型（Speech LLM）的自动语音识别（ASR）系统，常因其内部强大的语言模型先验，在识别命名实体和长尾词汇时表现不佳。检索增强偏置（Retrieval-augmented biasing）虽能提供帮助，但其效果依赖于在弱监督条件下对整句语音中热词的精确定位。本文提出CLAR，一种双编码器语音-文本检索器，它利用连续积分放电（CIF）机制学习无时间戳的单调词级对齐。通过长度感知的局部化匹配，CLAR能够锚定短实体声学线索，并减少表征稀释与注意力漂移。该检索器通过结合全局与局部片段级对比损失以及CIF数量约束的多粒度目标进行训练。推理时，排名靠前的热词被注入作为语音大语言模型的上下文提示，从而在不采用浅层融合的情况下提升识别性能。实验表明，CLAR显著改善了热词检索效果，并在关键错误率（CER）和带偏置词错误率（B-WER）上均优于强上下文感知ASR基线。
URL
https://arxiv.org/abs/2603.25460
