Abstract
Audio representation learning typically evaluates design choices such as input frontend, sequence backbone, and sequence length in isolation. We show that these axes are coupled, and conclusions from one setting often do not transfer to others. We introduce HELIX, a controlled framework comparing pure Mamba, pure attention, and a minimal hybrid with a single attention bottleneck. All models are parameter-matched at about 8.3M parameters to isolate architectural effects. Across six datasets, we find that the preferred input representation depends on the backbone, and that attention hurts performance on short, stationary audio but becomes important at longer sequence lengths. On a 5-minute speaker identification task with 30,000 tokens, pure attention fails with out-of-memory errors, while HELIX closes an 11.5-point gap over pure Mamba.
Abstract (translated)
音频表示学习通常孤立地评估输入前端、序列主干和序列长度等设计选择。我们证明这些维度相互关联，且某一设置下的结论往往无法迁移至其他场景。我们引入HELIX——一个受控框架，用于比较纯Mamba、纯注意力机制以及仅含单一注意力瓶颈的最小混合模型。所有模型均进行参数匹配（约830万参数），以隔离架构影响。在六个数据集上的实验表明：首选输入表示取决于主干架构；注意力机制在短时平稳音频上会损害性能，但在更长序列长度下变得至关重要。在包含30,000个token的5分钟说话人识别任务中，纯注意力因内存不足而失败，而HELIX相较于纯Mamba缩小了11.5个百分点的性能差距。
URL
https://arxiv.org/abs/2603.21316
