arXiv_SD
-
The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMs
2026-03-15 16:54:36Shree Harsha Bokkahalli Satish, Christoph Minixhofer, Maria Teleki, James Caverlee, Ond\v{r}ej Klejch, Peter Bell, Gustav Eje Henter, \'Eva Sz\'ekelyAbstract
Speech Large Language Models (SpeechLLMs) process spoken input directly, retaining cues such as accent and perceived gender that were previously removed in cascaded pipelines. This introduces speaker identity dependent variation in responses. We present a large-scale intersectional evaluation of accent and gender bias in three SpeechLLMs using 2,880 controlled interactions across six English accents and two gender presentations, keeping linguistic content constant through voice cloning. Using pointwise LLM-judge ratings, pairwise comparisons, and Best-Worst Scaling with human validation, we detect consistent disparities. Eastern European-accented speech receives lower helpfulness scores, particularly for female-presenting voices. The bias is implicit: responses remain polite but differ in helpfulness. While LLM judges capture the directional trend of these biases, human evaluators exhibit significantly higher sensitivity, uncovering sharper intersectional disparities.
Abstract (translated)
URL
https://arxiv.org/abs/2603.16941
PDF
-
Affectron: Emotional Speech Synthesis with Affective and Contextually Aligned Nonverbal Vocalizations
2026-03-15 15:16:25Deok-Hyeon Cho, Hyung-Seok Oh, Seung-Bin Kim, Seong-Whan LeeAbstract
Nonverbal vocalizations (NVs), such as laughter and sighs, are central to the expression of affective cues in emotional speech synthesis. However, learning diverse and contextually aligned NVs remains challenging in open settings due to limited NV data and the lack of explicit supervision. Motivated by this challenge, we propose Affectron as a framework for affective and contextually aligned NV generation. Built on a small-scale open and decoupled corpus, Affectron introduces an NV-augmented training strategy that expands the distribution of NV types and insertion locations. We further incorporate NV structural masking into a speech backbone pre-trained on purely verbal speech to enable diverse and natural NV synthesis. Experimental results demonstrate that Affectron produces more expressive and diverse NVs than baseline systems while preserving the naturalness of the verbal speech stream.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14432
PDF
-
CodecMOS-Accent: A MOS Benchmark of Resynthesized and TTS Speech from Neural Codecs Across English Accents
2026-03-15 11:32:21Wen-Chin Huang, Nicholas Sanders, Erica CooperAbstract
We present the CodecMOS-Accent dataset, a mean opinion score (MOS) benchmark designed to evaluate neural audio codec (NAC) models and the large language model (LLM)-based text-to-speech (TTS) models trained upon them, especially across non-standard speech like accented speech. The dataset comprises 4,000 codec resynthesis and TTS samples from 24 systems, featuring 32 speakers spanning ten accents. A large-scale subjective test was conducted to collect 19,600 annotations from 25 listeners across three dimensions: naturalness, speaker similarity, and accent similarity. This dataset does not only represent an up-to-date study of recent speech synthesis system performance but reveals insights including a tight relationship between speaker and accent similarity, the predictive power of objective metrics, and a perceptual bias when listeners share the same accent with the speaker. This dataset is expected to foster research on more human-centric evaluation for NAC and accented TTS.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14328
PDF
-
Semi-Automatic Flute Robot and Its Acoustic Sensing
2026-03-15 02:02:36Hikari Kuriyama, Hiroaki Sonoda, Kouki Tomiyoshi, Gou KoutakiAbstract
Flute performance requires mastery of complex fingering combinations and register-dependent embouchure control, particularly jet offset adjustment for low-register production. Existing haptic and semi-automated systems do not address both aspects simultaneously through mechanical actuation. To our knowledge, no prior system fully automates fingering while mechanically assisting low-register tone production without requiring embouchure control. We developed a semi-automatic flute robot with an automatic fingering mechanism: fourteen servo motors actuate all keys via wire-based and rack-and-pinion drives in response to MIDI input, enabling performers to produce complete musical pieces through airflow alone. A jet offset assist mechanism rotates the head joint by a calibrated $22^\circ$ during low-register passages, shifting the jet offset toward a low-register configuration without modifying the instrument or embouchure. Fundamental frequency estimation confirmed correct pitch production across the chromatic range (C4--C7) and during musical performance. All key and lever movements were completed within 77.50~ms, corresponding to tempo capacity exceeding standard requirements. Harmonic analysis ($\Delta\mathrm{SPL} = \mathrm{SPL}_2 - \mathrm{SPL}_3$) showed a consistent increase in $\Delta$SPL for all low-register notes when activated, consistent with the intended jet offset shift. Head joint rotation completed within 40.00~ms. These results demonstrate mechanical feasibility of integrating automated fingering and register-dependent jet offset assistance under controlled conditions.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14180
PDF
-
LightBeam: An Accurate and Memory-Efficient CTC Decoder for Speech Neuroprostheses
2026-03-14 16:06:15Ebrahim Feghhi, Junlin Hu, Nima Hadidi, Jonathan C. KaoAbstract
A promising pathway for restoring communication in patients with dysarthria and anarthria is speech neuroprostheses, which directly decode speech from cortical neural activity. Two benchmarks, Brain-to-Text '24 and '25, released intracranial recordings from patients with dysarthria along with a baseline algorithm trained with Connectionist Temporal Classification (CTC). Despite significant innovation on these benchmarks, all leading published prior work relies on a WFST-based CTC decoder that requires ${\sim}$320 GB of RAM. These memory requirements limit accessibility for both patients and researchers. Here, we propose LightBeam, a non-WFST based CTC decoder that requires only ${\sim}$10 GB of RAM and achieves state-of-the-art performance on both benchmarks. LightBeam achieves this by integrating an LLM into the beam-search process via delayed fusion, obviating the prior need for using a large N-gram LM. LightBeam is implemented in Python and is open-source.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14002
PDF
-
Distributed Acoustic Sensing for Urban Traffic Monitoring: Spatio-Temporal Attention in Recurrent Neural Networks
2026-03-14 11:30:39Izhan Fakhruzi, Manuel Titos, Carmen Ben\'itez, Luz Garc\'iaAbstract
Effective urban traffic monitoring is essential for improving mobility, enhancing safety, and supporting sustainable cities. Distributed Acoustic Sensing (DAS) enables large-scale traffic observation by transforming existing fiber-optic infrastructure into dense arrays of vibration sensors. However, modeling the high-resolution spatio-temporal structure of DAS data for reliable traffic event recognition remains challenging. This study presents a real-world DAS-based traffic monitoring experiment conducted in Granada, Spain, where vehicles cross a fiber deployed perpendicular to the roadway. Recurrent neural networks (RNNs) are employed to model intra- and inter-event temporal dependencies. Spatial and temporal attention mechanisms are systematically integrated within the RNN architecture to analyze their impact on recognition performance, parameter efficiency, and interpretability. Results show that an appropriate and complementary placement of attention modules improves the balance between accuracy and model complexity. Attention heatmaps provide physically meaningful interpretations of classification decisions by highlighting informative spatial locations and temporal segments. Furthermore, the proposed SA-bi-TA configuration demonstrates spatial transferability, successfully recognizing traffic events at sensing locations different from those used during training, with only moderate performance degradation. These findings support the development of scalable and interpretable DAS-based traffic monitoring systems capable of operating under heterogeneous urban sensing conditions.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13903
PDF
-
Integrated Spoofing-Robust Automatic Speaker Verification via a Three-Class Formulation and LLR
2026-03-14 06:20:10Kai Tan, Lin Zhang, Ruiteng Zhang, Johan Rohdin, Leibny Paola Garc\'ia-Perera, Zexin Cai, Sanjeev Khudanpur, Matthew Wiesner, Nicholas AndrewsAbstract
Spoofing-robust automatic speaker verification (SASV) aims to integrate automatic speaker verification (ASV) and countermeasure (CM). A popular solution is fusion of independent ASV and CM scores. To better modeling SASV, some frameworks integrate ASV and CM within a single network. However, these solutions are typically bi-encoder based, offer limited interpretability, and cannot be readily adapted to new evaluation parameters without retraining. Based on this, we propose a unified end-to-end framework via a three-class formulation that enables log-likelihood ratio (LLR) inference from class logits for a more interpretable decision pipeline. Experiments show comparable performance to existing methods on ASVSpoof5 and better results on SpoofCeleb. The visualization and analysis also prove that the three-class reformulation provides more interpretability.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13780
PDF
-
Evaluating Compositional Structure in Audio Representations
2026-03-14 01:38:15Chuyang Chen, Bea Steers, Brian McFee, Juan BelloAbstract
We propose a benchmark for evaluating compositionality in audio representations. Audio compositionality refers to representing sound scenes in terms of constituent sources and attributes, and combining them systematically. While central to auditory perception, this property is largely absent from current evaluation protocols. Our framework adapts ideas from vision and language to audio through two tasks: A-COAT, which tests consistency under additive transformations, and A-TRE, which probes reconstructibility from attribute-level primitives. Both tasks are supported by large synthetic datasets with controlled variation in acoustic attributes, providing the first benchmark of compositional structure in audio embeddings.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13685
PDF
-
Perpetual Dialogues: A Computational Analysis of Voice-Guitar Interaction in Carlos Paredes's Discography
2026-03-13 09:55:21Gilberto Bernardes, N\'adia Moura, Ant\'onio S\'a PintoAbstract
Computational musicology enables systematic analysis of performative and structural traits in recorded music, yet existing approaches remain largely tailored to notated, score-based repertoires. This study advances a methodology for analyzing voice-guitar interaction in Carlos Paredes's vocal collaborations - an oral-tradition context where compositional and performative layers co-emerge. Using source-separated stems, physics-informed harmonic modelling, and beat-level audio descriptors, we examine melodic, harmonic, and rhythmic relationships across eight recordings with four singers. Our commonality-diversity framework, combining multi-scale correlation analysis with residual-based detection of structural deviations, reveals that expressive coordination is predominantly piece-specific rather than corpus-wide. Diversity events systematically align with formal boundaries and textural shifts, demonstrating that the proposed approach can identify musically salient reorganizations with minimal human annotation. The framework further offers a generalizable computational strategy for repertoires without notated blueprints, extending Music Performance Analysis into oral-tradition and improvisation-inflected practices.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12854
PDF
-
RadEar: A Self-Supervised RF Backscatter System for Voice Eavesdropping and Separation
2026-03-12 20:54:42Qijun Wang, Peihao Yan, Chunqi Qian, Huacheng ZengAbstract
Eavesdropping on voice conversations presents a growing threat to personal privacy and information security. In this paper, we present RadEar, a novel RF backscatter-based system designed to enable covert voice eavesdropping through walls. RadEar consists of two key components: (i) a batteryless RF backscatter tag covertly deployed inside the target space, and (ii) an RF reader located outside the room that performs signal demodulation, voice separation, and denoising. The tag features a compact, dual-resonator design that achieves energy-efficient frequency modulation for continuous voice eavesdropping while mitigating self-interference by separating excitation and reflection frequencies. To overcome the challenges of weak signal reception and overlapping speech, the RF reader employs self-supervised learning models for voice separation and denoising, trained using a remix-based objective without requiring ground-truth labels. We fabricate and evaluate RadEar in real-world scenarios, demonstrating its ability to recover and separate human speech with high fidelity under practical constraints.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12446
PDF
-
RAF: Relativistic Adversarial Feedback For Universal Speech Synthesis
2026-03-12 08:43:04Yongjoon Lee, Jung-Woo ChoiAbstract
We propose Relativistic Adversarial Feedback (RAF), a novel training objective for GAN vocoders that improves in-domain fidelity and generalization to unseen scenarios. Although modern GAN vocoders employ advanced architectures, their training objectives often fail to promote generalizable representations. RAF addresses this problem by leveraging speech self-supervised learning models to assist discriminators in evaluating sample quality, encouraging the generator to learn richer representations. Furthermore, we utilize relativistic pairing for real and fake waveforms to improve the modeling of the training data distribution. Experiments across multiple datasets show consistent gains in both objective and subjective metrics on GAN-based vocoders. Importantly, the RAF-trained BigVGAN-base outperforms the LSGAN-trained BigVGAN in perceptual quality using only 12\% of the parameters. Comparative studies further confirm the effectiveness of RAF as a training framework for GAN vocoders.
Abstract (translated)
我们提出了一种名为相对对抗反馈（RAF）的新型训练目标，用于GAN音高编码器（GAN vocoders），以提高其在域内的保真度和对未见场景的泛化能力。尽管现代GAN音高编码器采用了先进的架构，但它们的训练目标往往无法促进具有通用性的表示形式的学习。通过利用语音自监督学习模型来帮助判别器评估样本质量，RAF解决了这个问题，并鼓励生成器学习更加丰富的表示形式。此外，我们还采用相对配对方法对真实和虚假波形进行处理，以改善对训练数据分布的建模。在多个数据集上的实验表明，在基于GAN的音高编码器上，无论是客观还是主观指标都得到了一致性的改进。尤为重要的是，使用RAF训练的BigVGAN-base仅利用12%的参数量就超过了LSGAN训练的BigVGAN在感知质量方面的表现。比较研究进一步证实了RAF作为GAN音高编码器训练框架的有效性。
URL
https://arxiv.org/abs/2603.11678
PDF
-
SEMamba++: A General Speech Restoration Framework Leveraging Global, Local, and Periodic Spectral Patterns
2026-03-12 08:37:28Yongjoon Lee, Jung-Woo ChoiAbstract
General speech restoration demands techniques that can interpret complex speech structures under various distortions. While State-Space Models like SEMamba have advanced the state-of-the-art in speech denoising, they are not inherently optimized for critical speech characteristics, such as spectral periodicity or multi-resolution frequency analysis. In this work, we introduce an architecture tailored to incorporate speech-specific features as inductive biases. In particular, we propose Frequency GLP, a frequency feature extraction block that effectively and efficiently leverages the properties of frequency bins. Then, we design a multi-resolution parallel time-frequency dual-processing block to capture diverse spectral patterns, and a learnable mapping to further enhance model performance. With all our ideas combined, the proposed SEMamba++ achieves the best performance among multiple baseline models while remaining computationally efficient.
Abstract (translated)
一般性的语音恢复技术需要能够在各种失真条件下解析复杂的语音结构的技术。虽然状态空间模型（如SEMamba）已经在语音降噪方面取得了最先进的成果，但它们并不固有地针对诸如频谱周期性或多分辨率频率分析等关键语音特征进行优化。在这项工作中，我们介绍了一种专门设计来整合特定于语音的特性作为归纳偏差的架构。具体来说，我们提出了频率GLP（Frequency GLP），这是一种频率特征提取模块，能够有效且高效地利用频率带的属性。然后，我们设计了一个多分辨率并行时频双处理块，以捕捉多样化的频谱模式，并引入了一种可学习映射来进一步提升模型性能。通过结合所有这些想法，所提出的SEMamba++在保持计算效率的同时，在多个基线模型中取得了最佳性能。
URL
https://arxiv.org/abs/2603.11669
PDF
-
Resonate: Reinforcing Text-to-Audio Generation via Online Feedback from Large Audio Language Models
2026-03-12 08:29:19Xiquan Li, Junxi Liu, Wenxi Chen, Haina Zhu, Ziyang Ma, Xie ChenAbstract
Reinforcement Learning (RL) has become an effective paradigm for enhancing Large Language Models (LLMs) and visual generative models. However, its application in text-to-audio (TTA) generation remains largely under-explored. Prior work typically employs offline methods like Direct Preference Optimization (DPO) and leverages Contrastive Language-Audio Pretraining (CLAP) models as reward functions. In this study, we investigate the integration of online Group Relative Policy Optimization (GRPO) into TTA generation. We adapt the algorithm for Flow Matching-based audio models and demonstrate that online RL significantly outperforms its offline counterparts. Furthermore, we incorporate rewards derived from Large Audio Language Models (LALMs), which can provide fine-grained scoring signals that are better aligned with human perception. With only 470M parameters, our final model, \textbf{Resonate}, establishes a new SOTA on TTA-Bench in terms of both audio quality and semantic alignment.
Abstract (translated)
强化学习（RL）已成为增强大型语言模型（LLMs）和视觉生成模型的有效方法。然而，其在文本到音频（TTA）生成中的应用仍然很大程度上未被探索。先前的研究通常采用离线方法如直接偏好优化（DPO），并利用对比语言-音频预训练（CLAP）模型作为奖励函数。在这项研究中，我们探讨了将在线组相对策略优化（GRPO）集成到TTA生成中的可能性。我们将算法适应于基于流匹配的音频模型，并证明在线RL比其离线方法显著更优。此外，我们还引入了来自大型音频语言模型（LALMs）的奖励机制，这些模型能够提供与人类感知更好地对齐的细粒度评分信号。凭借仅4.7亿参数量，我们的最终模型“Resonate”在TTA-Bench上建立了新的最先进水平，在音频质量和语义一致性方面均表现出色。
URL
https://arxiv.org/abs/2603.11661
PDF
-
Edge-Cloud Collaborative Speech Emotion Captioning via Token-Level Speculative Decoding in Audio-Language Models
2026-03-12 00:19:02Xiangyuan Xue, Jiajun Lu, Yan Gao, Gongping Huang, Ting Dang, Hong JiaAbstract
Speech Emotion Captioning (SEC) leverages large audio-language models to generate rich, context-aware affective descriptions from speech. However, real-world deployment remains challenging due to the substantial computational demands on resource-constrained edge devices and the privacy risks of transmitting biometric audio. While smaller audio-language models enable efficient on-device SEC, their limited capacity often weakens subtle paralinguistic modeling and fine-grained affective grounding. We propose an edge-cloud collaborative framework based on Uncertainty-Guided Speculative Decoding (UGSD). A lightweight edge model drafts captions locally, and only high-uncertainty token blocks are selectively escalated to a stronger cloud verifier for validation. Experiments on the MER2024 benchmark demonstrate substantial BLEU improvements up to 62.7%. UGSD further achieves 1.4x lower latency and 8.5x higher token throughput compared to an edge-only model. These results empirically characterize the quality-efficiency-privacy trade-off in deployable SEC systems.
Abstract (translated)
语音情感字幕（SEC）技术利用大型音频语言模型从语音中生成丰富且上下文感知的情感描述。然而，由于资源受限的边缘设备计算需求较大以及传输生物识别音频带来的隐私风险，其实用部署仍然面临挑战。虽然较小的音频语言模型能够在设备上高效实现SEC，但它们有限的能力往往削弱了细微的副语言建模和情感细粒度定位。 为解决这些问题，我们提出了一种基于不确定性引导推测解码（UGSD）的边缘-云协作框架。在这个框架中，轻量级的边缘模型在本地生成字幕，而只有具有高不确定性的标记块才会被选择性地提升到更强的云端验证器进行验证。我们在MER2024基准测试上的实验表明，BLEU评分提高了高达62.7%。此外，UGSD还实现了1.4倍更低的延迟和8.5倍更高的令牌吞吐量，相较于仅边缘模型的方法。这些结果实证地描述了部署SEC系统时质量、效率和隐私之间的权衡关系。 这种框架不仅增强了情感字幕的质量和生成速度，同时也降低了传输敏感数据的风险，为在资源受限环境中的实用部署提供了可能。
URL
https://arxiv.org/abs/2603.11397
PDF
-
Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
2026-03-11 23:45:06Hillary Mutisya, John MuganeAbstract
We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic speech recognition (ASR). Our approach combines unlabeled audio with limited labeled data through pseudo-labeled CPT followed by supervised finetuning. With 20,000 labeled samples, we achieve 3.24% WER on Common Voice Swahili-an 82% relative improvement over the baseline. This result surpasses the best previously reported academic system (8.3% WER from XLS-R) by 61% relative improvement. We provide concrete data requirements and a replicable methodology applicable to other low-resource languages.
Abstract (translated)
我们研究了用于将wav2vec2-bert-2.0模型适应斯瓦希里语自动语音识别（ASR）的持续预训练（CPT）。我们的方法结合了未标记音频和有限标签数据，通过伪标注CPT然后进行监督微调。使用20,000个标注样本，在Common Voice 斯瓦希里语测试集上实现了3.24%的WER，相较于基线模型相对改进了82%。这一结果超越了之前最佳的学术系统（XLS-R系统的8.3% WER），相对提高了61%。我们提供了具体的数据需求和可重复的方法论，适用于其他低资源语言。
URL
https://arxiv.org/abs/2603.11378
PDF
-
Fair-Gate: Fairness-Aware Interpretable Risk Gating for Sex-Fair Voice Biometrics
2026-03-11 22:50:15Yangyang Qu, Todisco Massimiliano, Galdi Chiara, Evans NicholasAbstract
Voice biometric systems can exhibit sex-related performance gaps even when overall verification accuracy is strong. We attribute these gaps to two practical mechanisms: (i) demographic shortcut learning, where speaker classification training exploits spurious correlations between sex and speaker identity, and (ii) feature entanglement, where sex-linked acoustic variation overlaps with identity cues and cannot be removed without degrading speaker discrimination. We propose Fair-Gate, a fairness-aware and interpretable risk-gating framework that addresses both mechanisms in a single pipeline. Fair-Gate applies risk extrapolation to reduce variation in speaker-classification risk across proxy sex groups, and introduces a local complementary gate that routes intermediate features into an identity branch and a sex branch. The gate provides interpretability by producing an explicit routing mask that can be inspected to understand which features are allocated to identity versus sex-related pathways. Experiments on VoxCeleb1 show that Fair-Gate improves the utility--fairness trade-off, yielding more sex-fair ASV performance under challenging evaluation conditions.
Abstract (translated)
声纹系统可能会表现出性别相关的性能差距，即使其总体验证准确性很强。我们把这种差距归因于两种实际机制：(i) 人口统计捷径学习，在此过程中，说话人分类训练利用了性别与说话人身份之间的虚假关联；(ii) 特征纠缠，其中与性别的声学变异重叠了身份线索，这些特征无法在不降低说话人识别准确性的情况下被移除。我们提出了一种公平感知且可解释的风险门控框架——Fair-Gate，该框架在一个单一的工作流程中解决了这两种机制。Fair-Gate 应用了风险外推法来减少不同代理性别组间的说话人分类风险差异，并引入了一个局部互补门，将中间特征路由至身份分支和性别分支。此门提供了可解释性，因为它生成了可以被检查的明确路由掩码，以了解哪些特征分配给了身份路径与性别相关的路径。在 VoxCeleb1 数据集上的实验表明，Fair-Gate 改善了效用-公平性的权衡，在具有挑战性的评估条件下实现了更公平的声纹认证性能。
URL
https://arxiv.org/abs/2603.11360
PDF
-
Beyond Deep Learning: Speech Segmentation and Phone Classification with Neural Assemblies
2026-03-11 21:25:03Trevor Adelson, Vidhyasaharan Sethu, Ting DangAbstract
Deep learning dominates speech processing but relies on massive datasets, global backpropagation-guided weight updates, and produces entangled representations. Assembly Calculus (AC), which models sparse neuronal assemblies via Hebbian plasticity and winner-take-all competition, offers a biologically grounded alternative, yet prior work focused on discrete symbolic inputs. We introduce an AC-based speech processing framework that operates directly on continuous speech by combining three key contributions:(i) neural encoding that converts speech into assembly-compatible spike patterns using probabilistic mel binarisation and population-coded MFCCs; (ii) a multi-area architecture organising assemblies across hierarchical timescales and classes; and (iii) cross-area update schemes for downstream tasks. Applied to two core tasks of boundary detection and segment classification, our framework detects phone (F1=0.69) and word (F1=0.61) boundaries without any weight training, and achieves 47.5% and 45.1% accuracy on phone and command recognition. These results show that AC-based dynamical systems are a viable alternative to deep learning for speech processing.
Abstract (translated)
URL
https://arxiv.org/abs/2603.16923
PDF
-
Learnable Pulse Accumulation for On-Device Speech Recognition: How Much Attention Do You Need?
2026-03-11 20:49:32Yakov Pyotr ShkolnikovAbstract
Self-attention scales quadratically with sequence length, limiting transformer-based speech models on edge devices. We introduce the Learnable Pulse Accumulator (LPA), an O(n) replacement that substitutes key-query dot products with learned gating functions: content-dependent rectangular pulses, periodic windows, and position-dependent basis functions. An MSE diagnostic sweep determines per-layer replacement difficulty and ordering. Replacing 8 of 12 wav2vec2-base layers yields 10.61% word error rate (WER) on LibriSpeech test-clean, +7.24 percentage points (pp) over the 3.37% baseline, with 3.27x speedup at 120s audio on Apple M4 Pro via an optimized MLX inference path. Cross-domain validation on SepFormer speech enhancement shows all 16 intra-chunk attention layers can be replaced without collapse, suggesting the depth wall arises from linguistic computation rather than an LPA limitation. LPA's near-binary gates at inference enable dense GPU computation with no CPU-GPU synchronization, and all operations map to mobile neural accelerators.
Abstract (translated)
URL
https://arxiv.org/abs/2603.16922
PDF
-
Cough activity detection for automatic tuberculosis screening
2026-03-11 19:09:26Joshua Jansen van V\"uren, Devendra Singh Parihar, Daphne Naidoo, Kimsey Zajac, Willy Ssengooba, Grant Theron, Thomas NieslerAbstract
The automatic identification of cough segments in audio through the determination of start and end points is pivotal to building scalable screening tools in health technologies for pulmonary related diseases. We propose the application of two current pre-trained architectures to the task of cough activity detection. A dataset of recordings containing cough from patients symptomatic for tuberculosis (TB) who self-present at community-level care centres in South Africa and Uganda is employed. When automatic start and end points are determined using XLS-R, an average precision of 0.96 and an area under the receiver-operating characteristic of 0.99 are achieved for the test set. We show that best average precision is achieved by utilising only the first three layers of the network, which has the dual benefits of reduced computational and memory requirements, pivotal for smartphone-based applications. This XLS-R configuration is shown to outperform an audio spectrogram transformer (AST) as well as a logistic regression baseline by 9% and 27% absolute in test set average precision respectively. Furthermore, a downstream TB classification model trained using the coughs automatically isolated by XLS-R comfortably outperforms a model trained on the coughs isolated by AST, and is only narrowly outperformed by a classifier trained on the ground truth coughs. We conclude that the application of large pre-trained transformer models is an effective approach to identifying cough end-points and that the integration of such a model into a screening tool is feasible.
Abstract (translated)
通过确定咳嗽段的起始和结束点来自动识别音频中的咳嗽片段，对于构建用于肺部相关疾病的可扩展筛查工具至关重要。我们提议将两个当前预训练的架构应用于咳嗽活动检测任务中。该研究使用了一组记录数据集，其中包含南非和乌干达社区级护理中心自诊为结核病（TB）患者发出的咳嗽声。 当使用XLS-R自动确定起始和结束点时，在测试集中实现了平均精度0.96和接收者操作特性曲线下面积（AUC-ROC）为0.99的成绩。我们展示了通过仅利用网络的前三个层可以获得最佳的平均精度，这不仅减少了计算资源和内存需求，对基于智能手机的应用来说尤其重要。 该XLS-R配置在测试集平均精度上分别比音频光谱图变换器（AST）和逻辑回归基线高出9%和27%，显示了其优越性。此外，使用XLS-R自动隔离的咳嗽训练出的下游TB分类模型明显优于基于AST隔离的咳嗽训练出的模型，并且仅略微逊于基于真实数据标签训练的分类器。 综上所述，我们认为应用大规模预训练转换模型来识别咳嗽结束点是有效的方法，并且将此类模型集成到筛查工具中是可行的。
URL
https://arxiv.org/abs/2603.11241
PDF
-
Can LLMs Help Localize Fake Words in Partially Fake Speech?
2026-03-11 18:21:14Lin Zhang, Thomas Thebaud, Zexin Cai, Sanjeev Khudanpur, Daniel Povey, Leibny Paola Garc\'ia-Perera, Matthew Wiesner, Nicholas AndrewsAbstract
Large language models (LLMs), trained on large-scale text, have recently attracted significant attention for their strong performance across many tasks. Motivated by this, we investigate whether a text-trained LLM can help localize fake words in partially fake speech, where only specific words within a speech are edited. We build a speech LLM to perform fake word localization via next token prediction. Experiments and analyses on AV-Deepfake1M and PartialEdit indicates that the model frequently leverages editing-style pattern learned from the training data, particularly word-level polarity substitutions for those two databases we discussed, as cues for localizing fake words. Although such particular patterns provide useful information in an in-domain scenario, how to avoid over-reliance on such particular pattern and improve generalization to unseen editing styles remains an open question.
Abstract (translated)
大规模语言模型（LLM），经过大量文本训练，因其在多项任务中的出色表现而最近备受关注。受此启发，我们研究了一个基于文本训练的LLM是否能够在部分伪造演讲中帮助定位假词，其中只有特定词汇被修改。我们构建了一个语音LLM，并通过下一个标记预测来执行伪造单词定位。在AV-Deepfake1M和PartialEdit数据库上的实验与分析表明，该模型经常利用从训练数据中学到的编辑风格模式，特别是针对这两个数据库中提到的单字级极性替换作为定位假词的线索。尽管这种特定模式在同类场景下提供了有用的信息，但如何避免过度依赖这些特定模式，并提高对未见编辑风格的泛化能力仍然是一个开放的问题。
URL
https://arxiv.org/abs/2603.11205
PDF
