Abstract
Speech Enhancement (SE) in audio devices is often supported by auxiliary modules for Voice Activity Detection (VAD), SNR estimation, or Acoustic Scene Classification to ensure robust context-aware behavior and seamless user experience. Just like SE, these tasks often employ deep learning; however, deploying additional models on-device is computationally impractical, whereas cloud-based inference would introduce additional latency and compromise privacy. Prior work on SE employed Dynamic Channel Pruning (DynCP) to reduce computation by adaptively disabling specific channels based on the current input. In this work, we investigate whether useful signal properties can be estimated from these internal pruning masks, thus removing the need for separate models. We show that simple, interpretable predictors achieve up to 93% accuracy on VAD, 84% on noise classification, and an R2 of 0.86 on F0 estimation. With binary masks, predictions reduce to weighted sums, inducing negligible overhead. Our contribution is twofold: on one hand, we examine the emergent behavior of DynCP models through the lens of downstream prediction tasks, to reveal what they are learning; on the other, we repurpose and re-propose DynCP as a holistic solution for efficient SE and simultaneous estimation of signal properties.
Abstract (translated)
语音增强（SE）在音频设备中通常依赖于辅助模块，如声源活动检测（VAD）、信噪比估计或声音场景分类来确保稳健的上下文感知行为和无缝用户体验。与SE一样，这些任务也常采用深度学习技术；然而，在设备上部署额外模型从计算角度来看是不可行的，而基于云的推理则会增加延迟并损害隐私。先前的研究中，SE采用了动态通道剪枝（DynCP）来通过自适应地禁用特定通道减少计算量，具体依据当前输入进行调整。 在本项研究工作中，我们探讨了是否可以从这些内部剪枝掩码中估计出有用的信号属性，从而消除单独模型的需求。研究表明，简单的、可解释的预测器可以达到高达93%的VAD准确率，84%的噪声分类准确率以及F0估计上R2值为0.86的表现。使用二进制掩码时，预测简化为加权求和操作，几乎不会增加额外开销。 我们的贡献是双重的：一方面，我们通过下游预测任务的角度来考察DynCP模型产生的新兴行为，揭示它们所学的内容；另一方面，我们将重新定义并提议使用DynCP作为高效语音增强和同时估计信号属性的整体解决方案。
URL
https://arxiv.org/abs/2602.10666
