Abstract
We address the fundamental incompatibility of attention-based encoder-decoder (AED) models with long-form acoustic encodings. AED models trained on segmented utterances learn to encode absolute frame positions by exploiting limited acoustic context beyond segment boundaries, but fail to generalize when decoding long-form segments where these cues vanish. The model loses ability to order acoustic encodings due to permutation invariance of keys and values in cross-attention. We propose four modifications: (1) injecting explicit absolute positional encodings into cross-attention for each decoded segment, (2) long-form training with extended acoustic context to eliminate implicit absolute position encoding, (3) segment concatenation to cover diverse segmentations needed during training, and (4) semantic segmentation to align AED-decoded segments with training segments. We show these modifications close the accuracy gap between continuous and segmented acoustic encodings, enabling auto-regressive use of the attention decoder.
Abstract (translated)
我们解决了基于注意力的编码器-解码器（AED）模型与长形式声学编码不兼容的基本问题。当在分割的话语上训练时，AED 模型会通过利用跨越段边界的一小部分声学上下文来学习对绝对帧位置进行编码，但在解码长形式片段时却无法泛化，因为在这些情况下这些线索消失。由于跨注意力中键和值的排列不变性，模型失去了按顺序排列声学编码的能力。我们提出了四项改进措施：(1) 在每次解码段的跨注意力中注入明确的绝对位置编码；(2) 通过使用扩展的声学上下文进行长形式训练来消除隐式的绝对位置编码；(3) 段连接以覆盖在训练过程中所需的各种分割情况；以及 (4) 语义分割，将 AED 解码片段与训练段对齐。我们展示了这些改进措施可以缩小连续和分段声学编码之间的准确度差距，使注意力解码器能够进行自回归使用。
URL
https://arxiv.org/abs/2512.14652
