Abstract
Multi-channel speech separation in dynamic environments is challenging as time-varying spatial and spectral features evolve at different temporal scales. Existing methods typically employ sequential architectures, forcing a single network stream to simultaneously model both feature types, creating an inherent modeling conflict. In this paper, we propose a dual-branch parallel spectral-spatial (PS2) architecture that separately processes spectral and spatial features through parallel streams. The spectral branch uses a bi-directional long short-term memory (BLSTM)-based frequency module, a Mamba-based temporal module, and a self-attention module to model spectral features. The spatial branch employs bi-directional gated recurrent unit (BGRU) networks to process spatial features that encode the evolving geometric relationships between sources and microphones. Features from both branches are integrated through a cross-attention fusion mechanism that adaptively weights their contributions. Experimental results demonstrate that the PS2 outperforms existing state-of-the-art (SOTA) methods by 1.6-2.2 dB in scale-invariant signal-to-distortion ratio (SI-SDR) for moving speaker scenarios, with robust separation quality under different reverberation times (RT60), noise levels, and source movement speeds. Even with fast source movements, the proposed model maintains SI-SDR improvements of over 13 dB. These improvements are consistently observed across multiple datasets, including WHAMR! and our generated WSJ0-Demand-6ch-Move dataset.
Abstract (translated)
在动态环境下的多通道语音分离具有挑战性，因为随时间变化的空间和频谱特征以不同的时间尺度演变。现有方法通常采用顺序架构，迫使单一网络流同时建模这两种特征类型，从而产生内在的建模冲突。在这篇论文中，我们提出了一种双分支并行频谱-空间（PS2）架构，该架构通过平行流分别处理频谱和空间特征。 频谱支路使用双向长短时记忆（BLSTM）频率模块、Mamba时间模块以及自注意力模块来建模频谱特性。空间支路则采用双向门控循环单元（BGRU）网络来处理编码了声源与麦克风之间随时间变化的几何关系的空间特征。 来自两分支的特征通过交叉注意融合机制进行整合，该机制能够根据各自的贡献灵活地加权它们的影响。实验结果表明，在移动说话者场景中，PS2架构比现有的最先进（SOTA）方法在可尺度不变信号与畸变比率（SI-SDR）上提高了1.6至2.2分贝。此外，PS2模型还展示了不同混响时间（RT60）、噪声水平和声源移动速度下的稳健分离性能，在即使是非常快速的源运动情况下也能保持超过13分贝的SI-SDR改进。 这些改善在包括WHAMR! 和我们生成的WSJ0-Demand-6ch-Move数据集在内的多个数据集中一致观察到。
URL
https://arxiv.org/abs/2602.22487
