Abstract
Speech separation in realistic acoustic environments remains challenging because overlapping speakers, background noise, and reverberation must be resolved simultaneously. Although recent time-frequency (TF) domain models have shown strong performance, most still rely on late-split architectures, where speaker disentanglement is deferred to the final stage, creating an information bottleneck and weakening discriminability under adverse conditions. To address this issue, we propose SR-CorrNet, an asymmetric encoder-decoder framework that introduces the separation-reconstruction (SepRe) strategy into a TF dual-path backbone. The encoder performs coarse separation from mixture observations, while the weight-shared decoder progressively reconstructs speaker-discriminative features with cross-speaker interaction, enabling stage-wise refinement. To complement this architecture, we formulate speech separation as a structured correlation-to-filter problem: spatio-spectro-temporal correlations computed from the observations are used as input features, and the corresponding deep filters are estimated to recover target signals. We further incorporate an attractor-based dynamic split module to adapt the number of output streams to the actual speaker configuration. Experimental results on WSJ0-2/3/4/5Mix, WHAMR!, and LibriCSS demonstrate consistent improvements across anechoic, noisy-reverberant, and real-recorded conditions in both single- and multi-channel settings, highlighting the effectiveness of TF-domain SepRe with correlation-based filter estimation for speech separation.
Abstract (translated)
在真实声学环境中的语音分离仍是一项挑战，因为重叠的说话人、背景噪声和混响必须同时得到解决。尽管近期时频域模型展现了强劲性能，但大多数仍依赖晚期分离架构，即将说话人解耦推迟至最后阶段，这造成了信息瓶颈并在恶劣条件下削弱了判别能力。为应对这一问题，我们提出了SR-CorrNet，一个将分离-重建策略引入时频双路径骨干网络的非对称编解码框架。编码器从混合观测中执行粗分离，而权重共享的解码器通过跨说话人交互逐步重建具有说话人判别性的特征，实现逐阶段细化。为补充此架构，我们将语音分离表述为一个结构化相关性-滤波器问题：利用从观测中计算出的时空频相关性作为输入特征，并估计相应的深度滤波器以恢复目标信号。我们进一步引入基于吸引子的动态分离模块，使输出流数量能自适应实际说话人配置。在WSJ0-2/3/4/5Mix、WHAMR!和LibriCSS数据集上的实验结果表明，在单通道和多通道设置下，该方法在消声、噪声混响及真实录音条件中均实现持续改进，凸显了基于相关性的滤波器估计与时频域分离-重建策略在语音分离中的有效性。
URL
https://arxiv.org/abs/2603.29097
