RNN
-
QKAN-LSTM: Quantum-inspired Kolmogorov-Arnold Long Short-term Memory
2026-02-27 18:55:13Yu-Chao Hsu, Jiun-Cheng Jiang, Chun-Hua Lin, Kuo-Chung Peng, Nan-Yow Chen, Samuel Yen-Chi Chen, En-Jui Kuo, Hsi-Sheng GoanAbstract
Long short-term memory (LSTM) models are a particular type of recurrent neural networks (RNNs) that are central to sequential modeling tasks in domains such as urban telecommunication forecasting, where temporal correlations and nonlinear dependencies dominate. However, conventional LSTMs suffer from high parameter redundancy and limited nonlinear expressivity. In this work, we propose the Quantum-inspired Kolmogorov-Arnold Long Short-Term Memory (QKAN-LSTM), which integrates Data Re-Uploading Activation (DARUAN) modules into the gating structure of LSTMs. Each DARUAN acts as a quantum variational activation function (QVAF), enhancing frequency adaptability and enabling an exponentially enriched spectral representation without multi-qubit entanglement. The resulting architecture preserves quantum-level expressivity while remaining fully executable on classical hardware. Empirical evaluations on three datasets, Damped Simple Harmonic Motion, Bessel Function, and Urban Telecommunication, demonstrate that QKAN-LSTM achieves superior predictive accuracy and generalization with a 79% reduction in trainable parameters compared to classical LSTMs. We extend the framework to the Jiang-Huang-Chen-Goan Network (JHCG Net), which generalizes KAN to encoder-decoder structures, and then further use QKAN to realize the latent KAN, thereby creating a Hybrid QKAN (HQKAN) for hierarchical representation learning. The proposed HQKAN-LSTM thus provides a scalable and interpretable pathway toward quantum-inspired sequential modeling in real-world data environments.
Abstract (translated)
长短期记忆（LSTM）模型是一种特别类型的递归神经网络（RNN），在诸如城市电信预测等领域中用于序列建模任务，这些领域中的时间相关性和非线性依赖关系占主导地位。然而，传统的LSTM模型存在参数冗余高和非线性表达能力有限的问题。为此，我们提出了一种量子启发的Kolmogorov-Arnold长短期记忆（QKAN-LSTM）模型，该模型将数据重新上传激活（DARUAN）模块整合到LSTM的门控结构中。每个DARUAN作为一个量子变分激活函数（QVAF），增强了频率适应性，并在没有多量子位纠缠的情况下实现了指数级丰富的谱表示。这种架构保留了量子级别的表达能力，同时完全可以在经典硬件上执行。 通过三个数据集——阻尼简谐运动、贝塞尔函数和城市电信的数据进行的实证评估表明，QKAN-LSTM比传统的LSTMs模型减少了79%的可训练参数，并且在预测准确性和泛化性方面表现出色。我们还扩展了Jiang-Huang-Chen-Goan网络（JHCG Net）框架，该框架将KAN推广到了编码器-解码器结构中，然后进一步使用QKAN实现潜在的KAN，从而创建了一个混合型QKAN（HQKAN），用于层次表示学习。因此，所提出的HQKAN-LSTM为在实际数据环境中进行量子启发式的序列建模提供了一条可扩展且易于解释的道路。 这个研究工作表明了量子计算理念如何可以被应用于传统的机器学习模型中以提高其性能和表达能力，尤其是在处理复杂的时间序列问题时。这种方法不仅减少了所需的参数数量，还提升了模型的预测准确性和泛化能力，这为未来将量子算法集成到经典硬件上进行高效数据处理提供了可能的方向。
URL
https://arxiv.org/abs/2512.05049
PDF
-
Memory Caching: RNNs with Growing Memory
2026-02-27 18:53:41Ali Behrouz, Zeman Li, Yuan Deng, Peilin Zhong, Meisam Razaviyayn, Vahab MirrokniAbstract
Transformers have been established as the de-facto backbones for most recent advances in sequence modeling, mainly due to their growing memory capacity that scales with the context length. While plausible for retrieval tasks, it causes quadratic complexity and so has motivated recent studies to explore viable subquadratic recurrent alternatives. Despite showing promising preliminary results in diverse domains, such recurrent architectures underperform Transformers in recall-intensive tasks, often attributed to their fixed-size memory. In this paper, we introduce Memory Caching (MC), a simple yet effective technique that enhances recurrent models by caching checkpoints of their memory states (a.k.a. hidden states). Memory Caching allows the effective memory capacity of RNNs to grow with sequence length, offering a flexible trade-off that interpolates between the fixed memory (i.e., $O(L)$ complexity) of RNNs and the growing memory (i.e., $O(L^2)$ complexity) of Transformers. We propose four variants of MC, including gated aggregation and sparse selective mechanisms, and discuss their implications on both linear and deep memory modules. Our experimental results on language modeling, and long-context understanding tasks show that MC enhances the performance of recurrent models, supporting its effectiveness. The results of in-context recall tasks indicate that while Transformers achieve the best accuracy, our MC variants show competitive performance, close the gap with Transformers, and performs better than state-of-the-art recurrent models.
Abstract (translated)
Transformer 模型已被确立为大多数最近序列建模进展的核心架构，主要归功于其随上下文长度增加而扩大的内存容量。虽然对于检索任务来说是合理的，但对于 Transformer 来说，这种增长导致了二次复杂度的问题，并促使近期研究探索可行的亚二次递归替代方案。尽管在多个领域展示了初步的有希望的结果，这些递归架构在依赖于高召回的任务中表现不如 Transformer，这通常被归因于其固定的内存大小。 在本文中，我们介绍了一种简单却有效的方法——Memory Caching（MC），通过缓存递归模型的记忆状态检查点来增强递归模型。这种方法使得 RNN 的有效记忆容量随着序列长度的增长而增长，并提供了一个灵活的折衷方案，在固定大小的内存复杂度 ($O(L)$) 和不断扩大的 Transformer 内存复杂度 ($O(L^2)$) 之间进行平衡。 我们提出了四种 MC 变体，包括门控聚合和稀疏选择机制，并讨论了它们在线性和深度记忆模块中的影响。我们在语言建模和长上下文理解任务上的实验结果表明，MC 技术增强了递归模型的表现力和支持其有效性。针对上下文中回忆的任务的结果显示，虽然 Transformer 在准确性方面表现出最佳性能，但我们的 MC 变体展示了具有竞争力的性能，缩小了与 Transformer 的差距，并优于最先进的递归模型。
URL
https://arxiv.org/abs/2602.24281
PDF
-
FaultXformer: A Transformer-Encoder Based Fault Classification and Location Identification model in PMU-Integrated Active Electrical Distribution System
2026-02-27 18:28:12Kriti Thakur, Alivelu Manga Parimi, Mayukha PalAbstract
Accurate fault detection and localization in electrical distribution systems is crucial, especially with the increasing integration of distributed energy resources (DERs), which inject greater variability and complexity into grid operations. In this study, FaultXformer is proposed, a Transformer encoder-based architecture developed for automatic fault analysis using real-time current data obtained from phasor measurement unit (PMU). The approach utilizes time-series current data to initially extract rich temporal information in stage 1, which is crucial for identifying the fault type and precisely determining its location across multiple nodes. In Stage 2, these extracted features are processed to differentiate among distinct fault types and identify the respective fault location within the distribution system. Thus, this dual-stage transformer encoder pipeline enables high-fidelity representation learning, considerably boosting the performance of the work. The model was validated on a dataset generated from the IEEE 13-node test feeder, simulated with 20 separate fault locations and several DER integration scenarios, utilizing current measurements from four strategically located PMUs. To demonstrate robust performance evaluation, stratified 10-fold cross-validation is performed. FaultXformer achieved average accuracies of 98.76% in fault type classification and 98.92% in fault location identification across cross-validation, consistently surpassing conventional deep learning baselines convolutional neural network (CNN), recurrent neural network (RNN). long short-term memory (LSTM) by 1.70%, 34.95%, and 2.04% in classification accuracy and by 10.82%, 40.89%, and 6.27% in location accuracy, respectively. These results demonstrate the efficacy of the proposed model with significant DER penetration.
Abstract (translated)
在电力分配系统中，准确地检测和定位故障至关重要，尤其是在分布式能源资源（DERs）的集成不断增加的情况下，这些资源向电网操作注入了更大的可变性和复杂性。在这项研究中，提出了FaultXformer，这是一种基于Transformer编码器架构的设计，旨在利用从相量测量单元(PMU)获得的实时电流数据进行自动故障分析。 该方法利用时间序列电流数据在第一阶段提取丰富的时态信息，这对于识别故障类型以及精确确定多个节点上的故障位置至关重要。在第二阶段，这些提取出的特征被处理以区分不同的故障类型，并识别分布系统中的相应故障位置。因此，这种双阶段Transformer编码器流水线能够实现高保真度的表示学习，在很大程度上提高了工作的性能。 该模型在一个由IEEE 13节点测试馈电线生成的数据集上进行了验证，模拟了20个单独的故障位置以及多种DER集成场景，并利用四个战略性部署的PMU的电流测量。为了展示强大的性能评估，采用了分层10折交叉验证方法。在交叉验证中，FaultXformer分别实现了98.76%和98.92%的平均准确率，即在故障分类中的类型识别准确性和位置识别准确性。该模型在分类精度上比传统的深度学习基准（卷积神经网络(CNN)、递归神经网络(RNN)和长短期记忆(LSTM)）高出1.70%，34.95% 和2.04%，并且在定位精度上分别高出10.82%，40.89% 和6.27%。这些结果证明了所提出模型在显著的DER渗透情况下的有效性。
URL
https://arxiv.org/abs/2602.24254
PDF
-
HTM-EAR: Importance-Preserving Tiered Memory with Hybrid Routing under Saturation
Abstract
Memory constraints in long-running agents require structured management of accumulated facts while preserving essential information under bounded context limits. We introduce HTM-EAR, a hierarchical tiered memory substrate that integrates HNSW-based working memory (L1) with archival storage (L2), combining importance-aware eviction and hybrid routing. When L1 reaches capacity, items are evicted using a weighted score of importance and usage. Queries are first resolved in L1; if similarity or entity coverage is insufficient, retrieval falls back to L2, and candidates are re-ranked using a cross-encoder. We evaluate the system under sustained saturation (15,000 facts; L1 capacity 500; L2 capacity 5000) using synthetic streams across five random seeds and real BGL system logs. Ablation studies compare the full system against variants without cross-encoder re-ranking, without routing gates, with LRU eviction, and an oracle with unbounded memory. Under saturation, the full model preserves active-query precision (MRR = 1.000) while enabling controlled forgetting of stale history, approaching oracle active performance (0.997 +/- 0.003). In contrast, LRU minimizes latency (21.1 ms) but permanently evicts 2416 essential facts. On BGL logs, the full system achieves MRR 0.336, close to the oracle (0.370), while LRU drops to 0.069. Code is publicly available at: this https URL
Abstract (translated)
长期运行的代理程序中的内存限制要求对累积的事实进行结构化的管理，同时在有限的上下文范围内保持重要信息。我们引入了HTM-EAR，这是一种分层分级的记忆系统，结合了基于HNSW的工作记忆（L1）和归档存储（L2），并集成了感知重要性的驱逐机制和混合路由功能。 当工作记忆L1达到容量上限时，使用基于项目的重要性和使用的加权评分来移除项。查询首先在L1中解决；如果相似度或实体覆盖不足，则检索会退回到L2，并重新根据交叉编码器对候选对象进行排序。 我们通过合成流（五种随机种子）和实际BGL系统日志，在持续饱和条件下（总事实数：15000个，L1容量：500个，L2容量：5000个）来评估该系统的性能。消融实验比较了完整系统与没有交叉编码器重排名的变体、不使用路由门的变体、使用LRU驱逐策略的变体以及具有无界内存的理想情况。 在饱和条件下，完整的模型能够保持活跃查询精度（MRR = 1.000），同时实现对过时历史记录的有控制地遗忘，接近理想的活跃性能表现（0.997 ± 0.003）。相比之下，LRU策略虽然最小化了延迟（21.1 ms），但永久性地移除了2416个重要事实。在BGL日志上，完整系统实现了MRR为0.336，接近理想的性能（0.370），而使用LRU的变体则降到了0.069。 代码可在此处公开获取：[此URL] (请将"this https URL"替换为您实际提供的代码链接)。
URL
https://arxiv.org/abs/2603.10032
PDF
-
MEBM-Speech: Multi-scale Enhanced BrainMagic for Robust MEG Speech Detection
2026-02-27 13:15:42Li Songyi, Zheng Linze, Liang Jinghua, Zhang ZifengAbstract
We propose MEBM-Speech, a multi-scale enhanced neural decoder for speech activity detection from non-invasive magnetoencephalography (MEG) signals. Built upon the BrainMagic backbone, MEBM-Speech integrates three complementary temporal modeling mechanisms: a multi-scale convolutional module for short-term pattern extraction, a bidirectional LSTM (BiLSTM) for long-range context modeling, and a depthwise separable convolutional layer for efficient cross-scale feature fusion. A lightweight temporal jittering strategy and average pooling further improve onset robustness and boundary stability. The model performs continuous probabilistic decoding of MEG signals, enabling fine-grained detection of speech versus silence states - an ability crucial for both cognitive neuroscience and clinical applications. Comprehensive evaluations on the LibriBrain Competition 2025 Track1 benchmark demonstrate strong performance, achieving an average F1 macro of 89.3% on the validation set and comparable results on the official test leaderboard. These findings highlight the effectiveness of multi-scale temporal representation learning for robust MEG-based speech decoding.
Abstract (translated)
我们提出了MEBM-Speech，这是一种用于从非侵入性脑磁图（MEG）信号中检测语音活动的多尺度增强神经解码器。基于BrainMagic骨干网络，MEBM-Speech集成了三种互补的时间建模机制：一个多尺度卷积模块用于短期模式提取，一个双向长短时记忆网络（BiLSTM）用于长距离上下文建模，以及一个深度可分离卷积层用于高效的跨尺度特征融合。一种轻量级的时间抖动策略和平均池化进一步提高了起始点的鲁棒性和边界稳定性。该模型对MEG信号进行连续的概率解码，能够精细地检测语音与静默状态——这一能力对于认知神经科学和临床应用都至关重要。在LibriBrain Competition 2025 Track1基准上的全面评估显示了其强大的性能，在验证集上平均F1宏评分达到了89.3%，并在官方测试排行榜上取得了类似的结果。这些发现突显了多尺度时间表示学习对于基于MEG的语音解码的有效性。
URL
https://arxiv.org/abs/2603.02255
PDF
-
Scattering Transform for Auditory Attention Decoding
2026-02-26 13:48:27Ren\'e Pallenberg, Fabrice Katzberg, Alfred Mertins, Marco MaassAbstract
The use of hearing aids will increase in the coming years due to demographic change. One open problem that remains to be solved by a new generation of hearing aids is the cocktail party problem. A possible solution is electroencephalography-based auditory attention decoding. This has been the subject of several studies in recent years, which have in common that they use the same preprocessing methods in most cases. In this work, in order to achieve an advantage, the use of a scattering transform is proposed as an alternative to these preprocessing methods. The two-layer scattering transform is compared with a regular filterbank, the synchrosqueezing short-time Fourier transform and the common preprocessing. To demonstrate the performance, the known and the proposed preprocessing methods are compared for different classification tasks on two widely used datasets, provided by the KU Leuven (KUL) and the Technical University of Denmark (DTU). Both established and new neural-network-based models, CNNs, LSTMs, and recent Transformer/graph-based models are used for classification. Various evaluation strategies were compared, with a focus on the task of classifying speakers who are unknown from the training. We show that the two-layer scattering transform can significantly improve the performance for subject-related conditions, especially on the KUL dataset. However, on the DTU dataset, this only applies to some of the models, or when larger amounts of training data are provided, as in 10-fold cross-validation. This suggests that the scattering transform is capable of extracting additional relevant information.
Abstract (translated)
未来的几年里，助听器的使用量将会因为人口结构的变化而增加。新一代助听器仍需解决的一个开放性问题就是“鸡尾酒会问题”。一种可能的解决方案是基于脑电图（EEG）的听觉注意力解码方法，近年来对此进行了多项研究，它们通常采用相同的预处理方法。在这项工作中，为了获得优势，提出使用散射变换作为这些预处理方法的一种替代方案。两层散射变换与常规滤波器组、同步挤压短时傅里叶变换和常用预处理方法进行了比较。 为了展示性能，本文将已知的和提出的预处理方法在两个广泛使用的数据集上进行不同分类任务的对比，这两个数据集分别由鲁汶大学（KU Leuven, KUL）和丹麦技术大学（Technical University of Denmark, DTU）提供。用于分类的任务包括了传统的卷积神经网络（CNNs）、长短时记忆模型（LSTMs）以及最近提出的基于Transformer或图的方法。 比较了各种评估策略，重点是在测试集中包含训练集之外说话者的情况下进行分类任务的性能。研究结果显示，两层散射变换在与受试者相关的条件下显著提高了性能，特别是在KUL数据集上表现突出。然而，在DTU数据集上，只有某些模型或者在提供大量训练数据（例如10倍交叉验证）时才会表现出这一优势。这表明，散射变换能够提取出更多的相关信息。
URL
https://arxiv.org/abs/2602.23003
PDF
-
Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
2026-02-26 01:33:54An-Ci Peng, Kuan-Tang Huang, Tien-Hong Lo, Hung-Shin Lee, Hsin-Min Wang, Berlin ChenAbstract
Taiwanese Hakka is a low-resource, endangered language that poses significant challenges for automatic speech recognition (ASR), including high dialectal variability and the presence of two distinct writing systems (Hanzi and Pinyin). Traditional ASR models often encounter difficulties in this context, as they tend to conflate essential linguistic content with dialect-specific variations across both phonological and lexical dimensions. To address these challenges, we propose a unified framework grounded in the Recurrent Neural Network Transducers (RNN-T). Central to our approach is the introduction of dialect-aware modeling strategies designed to disentangle dialectal "style" from linguistic "content", which enhances the model's capacity to learn robust and generalized representations. Additionally, the framework employs parameter-efficient prediction networks to concurrently model ASR (Hanzi and Pinyin). We demonstrate that these tasks create a powerful synergy, wherein the cross-script objective serves as a mutual regularizer to improve the primary ASR tasks. Experiments conducted on the HAT corpus reveal that our model achieves 57.00% and 40.41% relative error rate reduction on Hanzi and Pinyin ASR, respectively. To our knowledge, this is the first systematic investigation into the impact of Hakka dialectal variations on ASR and the first single model capable of jointly addressing these tasks.
Abstract (translated)
台湾客家话是一种资源较少且濒临灭绝的语言，给自动语音识别（ASR）带来了重大挑战，包括高度的方言变异性以及两种不同的书写系统（汉字和拼音）。传统的ASR模型在这种情况下往往遇到困难，因为它们倾向于混淆语言内容与特定于方言的变化，这些变化在音韵和词汇维度上都有体现。为了应对这些挑战，我们提出了一种基于循环神经网络转换器（RNN-T）的统一框架。我们的方法核心在于引入了意识方言建模策略，旨在将方言特有的“风格”从语言通用的“内容”中分离出来，从而增强模型学习稳健且泛化的表示能力。此外，该框架还采用参数高效的预测网络来同时处理ASR任务（汉字和拼音）。我们证明，这些任务之间存在着强大的协同效应，即跨书写系统的目标可以作为相互的正则化器以提升主要的ASR任务性能。 在HAT语料库上进行的实验表明，我们的模型分别实现了57.00% 和40.41% 的相对错误率降低，针对汉字和拼音ASR。据我们所知，这是首次系统地研究客家方言变异性对ASR的影响，并且是首个能够同时处理这些任务的单一模型。
URL
https://arxiv.org/abs/2602.22522
PDF
-
Moving Speaker Separation via Parallel Spectral-Spatial Processing
2026-02-25 23:55:24Yuzhu Wang, Archontis Politis, Konstantinos Drossos, Tuomas VirtanenAbstract
Multi-channel speech separation in dynamic environments is challenging as time-varying spatial and spectral features evolve at different temporal scales. Existing methods typically employ sequential architectures, forcing a single network stream to simultaneously model both feature types, creating an inherent modeling conflict. In this paper, we propose a dual-branch parallel spectral-spatial (PS2) architecture that separately processes spectral and spatial features through parallel streams. The spectral branch uses a bi-directional long short-term memory (BLSTM)-based frequency module, a Mamba-based temporal module, and a self-attention module to model spectral features. The spatial branch employs bi-directional gated recurrent unit (BGRU) networks to process spatial features that encode the evolving geometric relationships between sources and microphones. Features from both branches are integrated through a cross-attention fusion mechanism that adaptively weights their contributions. Experimental results demonstrate that the PS2 outperforms existing state-of-the-art (SOTA) methods by 1.6-2.2 dB in scale-invariant signal-to-distortion ratio (SI-SDR) for moving speaker scenarios, with robust separation quality under different reverberation times (RT60), noise levels, and source movement speeds. Even with fast source movements, the proposed model maintains SI-SDR improvements of over 13 dB. These improvements are consistently observed across multiple datasets, including WHAMR! and our generated WSJ0-Demand-6ch-Move dataset.
Abstract (translated)
在动态环境下的多通道语音分离具有挑战性，因为随时间变化的空间和频谱特征以不同的时间尺度演变。现有方法通常采用顺序架构，迫使单一网络流同时建模这两种特征类型，从而产生内在的建模冲突。在这篇论文中，我们提出了一种双分支并行频谱-空间（PS2）架构，该架构通过平行流分别处理频谱和空间特征。 频谱支路使用双向长短时记忆（BLSTM）频率模块、Mamba时间模块以及自注意力模块来建模频谱特性。空间支路则采用双向门控循环单元（BGRU）网络来处理编码了声源与麦克风之间随时间变化的几何关系的空间特征。 来自两分支的特征通过交叉注意融合机制进行整合，该机制能够根据各自的贡献灵活地加权它们的影响。实验结果表明，在移动说话者场景中，PS2架构比现有的最先进（SOTA）方法在可尺度不变信号与畸变比率（SI-SDR）上提高了1.6至2.2分贝。此外，PS2模型还展示了不同混响时间（RT60）、噪声水平和声源移动速度下的稳健分离性能，在即使是非常快速的源运动情况下也能保持超过13分贝的SI-SDR改进。 这些改善在包括WHAMR! 和我们生成的WSJ0-Demand-6ch-Move数据集在内的多个数据集中一致观察到。
URL
https://arxiv.org/abs/2602.22487
PDF
-
A Fusion of context-aware based BanglaBERT and Two-Layer Stacked LSTM Framework for Multi-Label Cyberbullying Detection
2026-02-25 22:24:05Mirza Raquib, Asif Pervez Polok, Kedar Nath Biswas, Rahat Uddin Azad, Saydul Akbar Murad, Nick RahimiAbstract
Cyberbullying has become a serious and growing concern in todays virtual world. When left unnoticed, it can have adverse consequences for social and mental health. Researchers have explored various types of cyberbullying, but most approaches use single-label classification, assuming that each comment contains only one type of abuse. In reality, a single comment may include overlapping forms such as threats, hate speech, and harassment. Therefore, multilabel detection is both realistic and essential. However, multilabel cyberbullying detection has received limited attention, especially in low-resource languages like Bangla, where robust pre-trained models are scarce. Developing a generalized model with moderate accuracy remains challenging. Transformers offer strong contextual understanding but may miss sequential dependencies, while LSTM models capture temporal flow but lack semantic depth. To address these limitations, we propose a fusion architecture that combines BanglaBERT-Large with a two-layer stacked LSTM. We analyze their behavior to jointly model context and sequence. The model is fine-tuned and evaluated on a publicly available multilabel Bangla cyberbullying dataset covering cyberbully, sexual harassment, threat, and spam. We apply different sampling strategies to address class imbalance. Evaluation uses multiple metrics, including accuracy, precision, recall, F1-score, Hamming loss, Cohens kappa, and AUC-ROC. We employ 5-fold cross-validation to assess the generalization of the architecture.
Abstract (translated)
网络欺凌已成为当今虚拟世界中一个严重且日益增长的问题。如果被忽视，它会对社会和心理健康产生不利影响。研究人员已经探索了各种类型的网络欺凌，但大多数方法采用单标签分类，假设每个评论只包含一种形式的滥用行为。实际上，一条单独的评论可能包括多种形式的重叠内容，如威胁、仇恨言论和骚扰。因此，多标签检测既现实又至关重要。然而，多标签网络欺凌检测在资源较少的语言中（例如孟加拉语）受到了相对较少的关注，而这些语言中稳健的预训练模型也较为稀缺。开发一个具有适度准确性的通用模型仍然是一项挑战。Transformer 模型提供强大的上下文理解能力，但可能会错过序列依赖性；而LSTM 模型可以捕捉时间流程，但在语义深度方面有所欠缺。 为了解决这些问题，我们提出了一种融合架构，结合了BanglaBERT-Large与两层堆叠的LSTM模型。我们分析这两种模型的行为，并共同建模上下文和序列关系。该模型在公开可用的多标签孟加拉语网络欺凌数据集上进行微调和评估，该数据集涵盖了网络欺凌、性骚扰、威胁以及垃圾信息等类别。为了应对类不平衡问题，我们采用了不同的抽样策略。评估指标包括准确性、精确度、召回率、F1 值、汉明损失、Cohen's Kappa 和 AUC-ROC 等多项标准。采用五倍交叉验证来评估该架构的泛化能力。
URL
https://arxiv.org/abs/2602.22449
PDF
-
Quality of Automatic Speech Recognition -- Polish Language case study -- from Wav2Vec to Scribe ElevenLabs
2026-02-25 11:44:31Marcin Pietro\'n, Szymon Pi\'orkowski, Kamil Faber, Dominik \.Zurek, Micha{\l} Karwatowski, Jerzy Duda, Hubert Zieli\'nski, Piotr Lipnicki, Miko{\l}aj LeszczukAbstract
This article concerns comparative studies on the Automatic Speech Recognition (ASR) model incorporated with the Large Language Model (LLM) used for medical interviews. The proposed solution is tested on polish language benchmarks and dataset with medical interviews. The latest ASR technologies are based on convolutional neural networks (CNNs), recurrent neural networks (RNNs) and Transformers. Most of them work as end-to-end solutions. The presented approach in the case of the Whisper model shows a two-stage solution with End-To-End ASR and LLM working together in a pipeline. The ASR output is an input for LLM. The LLM is a component by which the output from ASR is corrected and improved. Comparative studies for automatic recognition of the Polish language between modern End-To-End deep learning architectures and the ASR hybrid model were performed. The medical interview tests were performed with two state-of-the-art ASR models: OpenAI Whisper incorporated with LLM and Scribe ElevenLabs. Additionally, the results were compared with five more end-to-end models (QuartzNet, FastConformer, Wav2Vec 2.0 XLSR and ESPnet Model Zoo) on Mozilla Common Voice and VoxPopuli databases. Tests were conducted for clean audio signal, signal with bandwidth limitation, and degraded. The tested models were evaluated on the basis of Word Error Rate (WER) and Character Error Rate (CER). The results show that the Whisper model performs by far the best among the open-source models. ElevenLabs Scribe model, on the other hand, performs best for Polish on both general benchmark and medical data.
Abstract (translated)
本文探讨了将大型语言模型（LLM）集成到自动语音识别（ASR）模型中，特别是在医疗访谈场景中的比较研究。所提出的解决方案在波兰语基准测试和包含医学访谈的语料库上进行了测试。最新的ASR技术基于卷积神经网络（CNN）、循环神经网络（RNN）和Transformer架构，大多数实现为端到端解决方案。文中提出的方法对于Whisper模型则展示了一个两阶段方法：一个是由端到端ASR系统生成输出，并由LLM进行修正和优化的管道流程。 针对波兰语自动识别，在现代端到端深度学习架构与基于混合模型的ASR之间进行了比较研究。医疗访谈测试使用了两种前沿的ASR模型：OpenAI的Whisper模型结合LLM和ElevenLabs的Scribe ElevenLabs模型。同时，这些结果还与其他五种端到端模型（QuartzNet、FastConformer、Wav2Vec 2.0 XLSR及ESPnet Model Zoo）在Mozilla Common Voice和VoxPopuli数据库上的性能进行了比较。 测试涵盖了无噪声音频信号、带宽限制的信号以及退化的信号。评估这些模型的表现基于单词错误率（WER）和字符错误率（CER）。结果显示，开源模型中Whisper表现最佳；而ElevenLabs Scribe模型在波兰语通用基准测试及医疗数据上表现出色。
URL
https://arxiv.org/abs/2603.02246
PDF
-
LMU-Based Sequential Learning and Posterior Ensemble Fusion for Cross-Domain Infant Cry Classification
2026-02-24 23:44:41Niloofar Jazaeri, Hilmi R. Dajani, Marco Janeczek, Martin BouchardAbstract
Decoding infant cry causes remains challenging for healthcare monitoring due to short nonstationary signals, limited annotations, and strong domain shifts across infants and datasets. We propose a compact acoustic framework that fuses MFCC, STFT, and pitch features within a multi-branch CNN encoder and models temporal dynamics using an enhanced Legendre Memory Unit (LMU). Compared to LSTMs, the LMU backbone provides stable sequence modeling with substantially fewer recurrent parameters, supporting efficient deployment. To improve cross-dataset generalization, we introduce calibrated posterior ensemble fusion with entropy-gated weighting to preserve domain-specific expertise while mitigating dataset bias. Experiments on Baby2020 and Baby Crying demonstrate improved macro-F1 under cross-domain evaluation, along with leakageaware splits and real-time feasibility for on-device monitoring.
Abstract (translated)
解码婴儿啼哭的原因对于医疗保健监测来说仍然具有挑战性，这是因为信号短且非平稳、标注有限以及不同婴儿和数据集之间的领域差异较大。我们提出了一种紧凑的声学框架，在多分支CNN编码器中融合了梅尔频率倒谱系数（MFCC）、短时傅立叶变换（STFT）和音高特征，并使用增强型勒让德记忆单元（LMU）来建模时间动态特性。相比于长短期记忆网络（LSTM），LMU骨干架构提供更稳定的序列建模能力，同时大幅减少了递归参数的数量，支持高效部署。 为了提高跨数据集的泛化性能，我们引入了基于熵门控加权的校准后验融合方法，这种方法在保持领域特定专业知识的同时减轻了数据集偏差。我们在Baby2020和婴儿啼哭数据集中进行了实验，在跨域评估中实现了更高的宏平均F1得分，并且通过泄漏感知拆分以及实时可行性验证，展示了设备端监测的应用潜力。
URL
https://arxiv.org/abs/2603.02245
PDF
-
Explicit Grammar Semantic Feature Fusion for Robust Text Classification
2026-02-24 10:25:29Azrin Sultana, Firoz AhmedAbstract
Natural Language Processing enables computers to understand human language by analysing and classifying text efficiently with deep-level grammatical and semantic features. Existing models capture features by learning from large corpora with transformer models, which are computationally intensive and unsuitable for resource-constrained environments. Therefore, our proposed study incorporates comprehensive grammatical rules alongside semantic information to build a robust, lightweight classification model without resorting to full parameterised transformer models or heavy deep learning architectures. The novelty of our approach lies in its explicit encoding of sentence-level grammatical structure, including syntactic composition, phrase patterns, and complexity indicators, into a compact grammar vector, which is then fused with frozen contextual embeddings. These heterogeneous elements unified a single representation that captures both the structural and semantic characteristics of the text. Deep learning models such as Deep Belief Networks (DBNs), Long Short-Term Memory (LSTMs), BiLSTMs, and transformer-based BERT and XLNET were used to train and evaluate the model, with the number of epochs varied. Based on experimental results, the unified feature representation model captures both the semantic and structural properties of text, outperforming baseline models by 2%-15%, enabling more effective learning across heterogeneous domains. Unlike prior syntax-aware transformer models that inject grammatical structure through additional attention layers, tree encoders, or full fine-tuning, the proposed framework treats grammar as an explicit inductive bias rather than a learnable module, resulting in a very lightweight model that delivers better performance on edge devices
Abstract (translated)
自然语言处理（NLP）通过分析和分类文本中的深层次语法和语义特征，使计算机能够理解人类语言。现有模型利用大规模语料库训练的变压器模型来捕捉这些特征，但这种方法计算密集且不适合资源受限环境。因此，我们提出的研究结合了全面的语法规则与语义信息，构建了一个无需使用全参数化的变压器模型或重型深度学习架构的健壮、轻量级分类模型。 我们的方法新颖之处在于明确地将句子级别的语法结构（包括句法构成、短语模式和复杂度指标）编码为一个紧凑的语法规则向量，并将其与冻结的上下文嵌入相结合。这些异构元素统一在一个单一表示中，该表示同时捕捉文本的结构和语义特性。 我们使用深度信念网络（DBNs）、长短期记忆网络（LSTMs）、双向LSTMs以及基于变压器的BERT和XLNET等深度学习模型来训练和评估我们的模型，并通过改变训练轮数来进行实验。根据实验结果，统一特征表示模型能够捕捉文本的语义与结构属性，在各类基准模型上表现出2%-15%的优势，实现了异构领域中的更有效学习。 不同于之前的语法感知变换器模型（这些模型通过附加注意力层、树编码器或完全微调来注入语法结构），我们的框架将语法视为一种显式的归纳偏差而非可学习模块。这导致了一个非常轻量级的模型，在边缘设备上表现出更好的性能。
URL
https://arxiv.org/abs/2602.20749
PDF
-
Voices of the Mountains: Deep Learning-Based Vocal Error Detection System for Kurdish Maqams
2026-02-24 10:17:16Darvan Shvan Khairaldeen, Hossein HassaniAbstract
Maqam, a singing type, is a significant component of Kurdish music. A maqam singer receives training in a traditional face-to-face or through self-training. Automatic Singing Assessment (ASA) uses machine learning (ML) to provide the accuracy of singing styles and can help learners to improve their performance through error detection. Currently, the available ASA tools follow Western music rules. The musical composition requires all notes to stay within their expected pitch range from start to finish. The system fails to detect micro-intervals and pitch bends, so it identifies Kurdish maqam singing as incorrect even though the singer performs according to traditional rules. Kurdish maqam requires recognizing performance errors within microtonal spaces, which is beyond Western equal temperament. This research is the first attempt to address the mentioned gap. While many error types happen during singing, our focus is on pitch, rhythm, and modal stability errors in the context of Bayati-Kurd. We collected 50 songs from 13 vocalists ( 2-3 hours) and annotated 221 error spans (150 fine pitch, 46 rhythm, 25 modal drift). The data was segmented into 15,199 overlapping windows and converted to log-mel spectrograms. We developed a two-headed CNN-BiLSTM with attention mode to decide whether a window contains an error and to classify it based on the chosen errors. Trained for 20 epochs with early stopping at epoch 10, the model reached a validation macro-F1 of 0.468. On the full 50-song evaluation at a 0.750 threshold, recall was 39.4% and precision 25.8% . Within detected windows, type macro-F1 was 0.387, with F1 of 0.492 (fine pitch), 0.536 (rhythm), and 0.133 (modal drift); modal drift recall was 8.0%. The better performance on common error types shows that the method works, while the poor modal-drift recall shows that more data and balancing are needed.
Abstract (translated)
玛卡姆（Maqam）是库尔德音乐中的一个重要组成部分，是一种歌唱类型。玛卡姆歌手通常通过传统的面对面指导或自我学习接受训练。自动歌唱评估（ASA）使用机器学习来提供对演唱风格准确性的评价，并能帮助学习者通过检测错误改进其表现。当前可用的ASA工具遵循西方音乐规则。这些系统要求所有音符在整个持续时间内的音高范围内保持不变，无法识别微间隔和音高的弯曲，因此即使歌手按照传统规则演唱，该系统的判定也可能认为库尔德玛卡姆歌唱是不正确的。库尔德玛卡姆需要在微调音空间中识别表演错误，这超出了西方等温律的范围。 本研究首次尝试解决上述差距。尽管在唱歌过程中会出现多种类型的错误，但我们的重点是在拜亚提-库尔（Bayati-Kurd）语境下对音高、节奏和模态稳定性的错误进行分析。我们从13名歌手那里收集了50首歌曲（每人大约2至3小时），并标注了221个错误片段（其中包括150个细音高错误，46个节奏错误以及25个调式漂移）。数据被分割成重叠的15,199个窗口，并转换为对数梅尔频谱图。我们开发了一种双头卷积神经网络-双向长短期记忆模型（CNN-BiLSTM）带注意力模式，用于判定一个窗口是否包含错误并根据选定的标准对其进行分类。 经过20轮训练，在第10轮时提前停止了训练，模型在验证阶段的宏平均F1得分为0.468。在50首歌曲全部进行评估且阈值设为0.750的情况下，召回率为39.4%，精度为25.8%。对于检测到的窗口，类型宏平均F1得分为0.387，具体而言，音高（Fine pitch）类型的F1得分为0.492，节奏（Rhythm）类型的F1得分是0.536，调式漂移（Modal drift）类型的F1得分仅为0.133；调式漂移的召回率更是低至8.0%。虽然对于常见的错误类型显示了较好的性能表明该方法有效，但较低的模态漂移召回率则显示出需要更多数据和平衡以改进评估结果。 这段研究总结描述了一种新开发的方法，旨在改善自动歌唱评估系统在识别库尔德玛卡姆音乐中特定误差方面的效果，并指出了进一步优化的方向。
URL
https://arxiv.org/abs/2602.20744
PDF
-
Enhancing Hate Speech Detection on Social Media: A Comparative Analysis of Machine Learning Models and Text Transformation Approaches
2026-02-24 07:26:17Saurabh Mishra, Shivani Thakur, Radhika MamidiAbstract
The proliferation of hate speech on social media platforms has necessitated the development of effective detection and moderation tools. This study evaluates the efficacy of various machine learning models in identifying hate speech and offensive language and investigates the potential of text transformation techniques to neutralize such content. We compare traditional models like CNNs and LSTMs with advanced neural network models such as BERT and its derivatives, alongside exploring hybrid models that combine different architectural features. Our results indicate that while advanced models like BERT show superior accuracy due to their deep contextual understanding, hybrid models exhibit improved capabilities in certain scenarios. Furthermore, we introduce innovative text transformation approaches that convert negative expressions into neutral ones, thereby potentially mitigating the impact of harmful content. The implications of these findings are discussed, highlighting the strengths and limitations of current technologies and proposing future directions for more robust hate speech detection systems.
Abstract (translated)
社交媒体平台上仇恨言论的泛滥促使开发有效的检测和管理工具。本研究评估了各种机器学习模型在识别仇恨言论和冒犯性语言方面的有效性，并探讨了文本转换技术消除此类内容的潜力。我们比较了传统的CNNs（卷积神经网络）和LSTMs（长短时记忆网络）模型与先进的神经网络模型如BERT及其衍生产品，同时探索结合不同架构特性的混合模型。我们的研究结果表明，虽然像BERT这样的先进模型由于其深刻的语境理解在准确性方面表现出色，但混合模型在某些情况下展示了改进的能力。此外，我们引入了创新的文本转换方法，将负面表达转化为中立表述，从而可能减轻有害内容的影响。文中讨论了这些发现的意义，并强调了当前技术的优势和局限性，并提出了未来构建更强大的仇恨言论检测系统的方向。
URL
https://arxiv.org/abs/2602.20634
PDF
-
Pip-Stereo: Progressive Iterations Pruner for Iterative Optimization based Stereo Matching
2026-02-24 02:51:37Jintu Zheng, Qizhe Liu, HuangXin Xu, Zhuojie ChenAbstract
While iterative stereo matching achieves high accuracy, its dependence on Recurrent Neural Networks (RNN) hinders edge deployment, a challenge underexplored in existing researches. We analyze iterative refinement and reveal that disparity updates are spatially sparse and temporally redundant. First, we introduce a progressive iteration pruning strategy that suppresses redundant update steps, effectively collapsing the recursive computation into a near-single-pass inference. Second, we propose a collaborative monocular prior transfer framework that implicitly embeds depth priors without requiring a dedicated monocular encoder, thereby eliminating its associated computational burden. Third, we develop FlashGRU, a hardware-aware RNN operator leveraging structured sparsity and I/O-conscious design, achieving a 7.28$\times$ speedup, 76.6\% memory peak reduction and 80.9\% global memory requests reduction over natvie ConvGRUs under 2K resolution. Our PipStereo enables real-time, high-fidelity stereo matching on edge hardware: it processes 320$\times$640 frames in just 75ms on an NVIDIA Jetson Orin NX (FP16) and 19ms on RTX 4090, matching the accuracy of large iterative based models, and our generalization ability and accuracy far exceeds that of existing real-time methods. Our embedded AI projects will be updated at: this https URL.
Abstract (translated)
虽然迭代立体匹配方法能够实现高精度，但其对递归神经网络（RNN）的依赖阻碍了边缘设备部署这一挑战在现有研究中较少被探索。我们分析了迭代细化过程，并发现视差更新具有空间稀疏性和时间冗余性。首先，我们提出了一种渐进式迭代剪枝策略，该策略可以抑制多余的更新步骤，从而将递归计算有效简化为近似单次推理过程。其次，我们设计了一个协作的单目先验转移框架，该框架能够隐式嵌入深度先验信息，而无需专门的单目编码器，从而减轻了相应的计算负担。第三，我们开发了一种称为FlashGRU的新硬件感知RNN操作符，它利用结构化稀疏性和输入/输出意识的设计，在2K分辨率下实现了7.28倍的速度提升，峰值内存减少了76.6%，全局内存请求减少达80.9%。 我们的PipStereo方法能够在边缘设备硬件上实现实时、高保真的立体匹配：在NVIDIA Jetson Orin NX（FP16）和RTX 4090平台上分别只需要75毫秒和19毫秒即可处理320x640像素的图像帧，其精度与大型迭代方法相媲美，并且我们的泛化能力和准确性远超现有的实时方法。有关我们嵌入式AI项目的最新进展，请访问：此链接。 （请将“this https URL”替换为实际发布的网址或具体说明。）
URL
https://arxiv.org/abs/2602.20496
PDF
-
Natural Language Processing Models for Robust Document Categorization
2026-02-23 20:33:22Radoslaw Roszczyk, Pawel Tecza, Maciej Stodolski, Krzysztof SiwekAbstract
This article presents an evaluation of several machine learning methods applied to automated text classification, alongside the design of a demonstrative system for unbalanced document categorization and distribution. The study focuses on balancing classification accuracy with computational efficiency, a key consideration when integrating AI into real world automation pipelines. Three models of varying complexity were examined: a Naive Bayes classifier, a bidirectional LSTM network, and a fine tuned transformer based BERT model. The experiments reveal substantial differences in performance. BERT achieved the highest accuracy, consistently exceeding 99\%, but required significantly longer training times and greater computational resources. The BiLSTM model provided a strong compromise, reaching approximately 98.56\% accuracy while maintaining moderate training costs and offering robust contextual understanding. Naive Bayes proved to be the fastest to train, on the order of milliseconds, yet delivered the lowest accuracy, averaging around 94.5\%. Class imbalance influenced all methods, particularly in the recognition of minority categories. A fully functional demonstrative system was implemented to validate practical applicability, enabling automated routing of technical requests with throughput unattainable through manual processing. The study concludes that BiLSTM offers the most balanced solution for the examined scenario, while also outlining opportunities for future improvements and further exploration of transformer architectures.
Abstract (translated)
本文评估了几种机器学习方法在自动文本分类中的应用，并设计了一个用于不平衡文档分类和分布的演示系统。研究重点在于平衡分类准确性和计算效率，这是将AI集成到现实世界自动化流水线中时的关键考虑因素。研究人员考察了三种不同复杂度的模型：朴素贝叶斯分类器、双向LSTM网络以及经过微调的基于Transformer的BERT模型。实验结果显示性能存在显著差异。 BERT取得了最高的准确性，超过了99%，但需要更长的训练时间和更多的计算资源。BiLSTM模型则提供了一个强有力的折中方案，在保持适中的训练成本的同时达到了大约98.56%的准确率，并且具有较强的理解上下文的能力。相比之下，朴素贝叶斯分类器训练速度最快，仅需毫秒级时间，然而其准确性最低，平均在94.5%左右。 类别不平衡影响了所有方法，尤其在识别少数类别的表现上更加明显。为验证实际应用的可行性，实施了一个完全功能的演示系统，该系统能够自动处理技术请求，大大提高了处理效率并达到了手动处理无法企及的速度水平。研究结论认为BiLSTM模型是目前所考察场景中的最优方案，同时指出未来改进和进一步探索Transformer架构的机会。
URL
https://arxiv.org/abs/2602.20336
PDF
-
EEG-Driven Intention Decoding: Offline Deep Learning Benchmarking on a Robotic Rover
2026-02-23 16:50:21Ghadah Alosaimi, Maha Alsayyari, Yixin Sun, Stamos Katsigiannis, Amir Atapour-Abarghouei, Toby P. BreckonAbstract
Brain-computer interfaces (BCIs) provide a hands-free control modality for mobile robotics, yet decoding user intent during real-world navigation remains challenging. This work presents a brain-robot control framework for offline decoding of driving commands during robotic rover operation. A 4WD Rover Pro platform was remotely operated by 12 participants who navigated a predefined route using a joystick, executing the commands forward, reverse, left, right, and stop. Electroencephalogram (EEG) signals were recorded with a 16-channel OpenBCI cap and aligned with motor actions at Delta = 0 ms and future prediction horizons (Delta > 0 ms). After preprocessing, several deep learning models were benchmarked, including convolutional neural networks, recurrent neural networks, and Transformer architectures. ShallowConvNet achieved the highest performance for both action prediction and intent prediction. By combining real-world robotic control with multi-horizon EEG intention decoding, this study introduces a reproducible benchmark and reveals key design insights for predictive deep learning-based BCI systems.
Abstract (translated)
脑机接口（BCI）为移动机器人提供了一种免手持控制方式，但在现实世界导航中解码用户意图仍然具有挑战性。本研究提出了一种用于在机器人漫游操作期间离线解码驾驶指令的脑-机器人控制系统框架。12名参与者使用操纵杆远程操控一个4WD Rover Pro平台，并沿着预定义路线执行前进、后退、左转、右转和停止等命令，同时记录了这些动作相关的脑电图（EEG）信号。这些信号通过佩戴16通道的OpenBCI帽采集，在每个运动行为发生时（Δ=0毫秒）以及未来的预测时间点（Δ>0毫秒）进行同步。 经过预处理后，研究人员评估了几种深度学习模型的效果，包括卷积神经网络、循环神经网络和Transformer架构。其中ShallowConvNet在动作预测和意图预测方面表现最佳。通过将现实世界的机器人控制与多步长EEG意图解码相结合，该研究引入了一个可重复的基准测试，并揭示了基于预测深度学习的BCI系统的关键设计见解。
URL
https://arxiv.org/abs/2602.20041
PDF
-
DTT-BSR: GAN-based DTTNet with RoPE Transformer Enhancement for Music Source Restoration
2026-02-23 13:23:22Shihong Tan, Haoyu Wang, Youran Ni, Yingzhao Hou, Jiayue Luo, Zipei Hu, Han Dou, Zerui Han, Ningning Pan, Yuzhu Wang, Gongping HuangAbstract
Music source restoration (MSR) aims to recover unprocessed stems from mixed and mastered recordings. The challenge lies in both separating overlapping sources and reconstructing signals degraded by production effects such as compression and reverberation. We therefore propose DTT-BSR, a hybrid generative adversarial network (GAN) combining rotary positional embeddings (RoPE) transformer for long-term temporal modeling with dual-path band-split recurrent neural network (RNN) for multi-resolution spectral processing. Our model achieved 3rd place on the objective leaderboard and 4th place on the subjective leaderboard on the ICASSP 2026 MSR Challenge, demonstrating exceptional generation fidelity and semantic alignment with a compact size of 7.1M parameters.
Abstract (translated)
音乐源恢复（MSR）的目标是从混合和母带处理的录音中恢复未经处理的音轨。挑战在于既要分离重叠的声音来源，又要重建因如压缩、混响等生产效果而退化的信号。因此，我们提出了DTT-BSR模型，这是一种结合了旋转位置嵌入（RoPE）变压器进行长期时序建模和双路径频带分割递归神经网络（RNN）进行多分辨率谱处理的混合生成对抗网络（GAN）。我们的模型在ICASSP 2026 MSR挑战赛中以7.1M参数量获得了客观排行榜第3名和主观排行榜第4名的成绩，展示了卓越的生成保真度和语义对齐能力。
URL
https://arxiv.org/abs/2602.19825
PDF
-
PerSoMed: A Large-Scale Balanced Dataset for Persian Social Media Text Classification
2026-02-22 20:53:08Isun Chehreh, Ebrahim AnsariAbstract
This research introduces the first large-scale, well-balanced Persian social media text classification dataset, specifically designed to address the lack of comprehensive resources in this domain. The dataset comprises 36,000 posts across nine categories (Economic, Artistic, Sports, Political, Social, Health, Psychological, Historical, and Science & Technology), each containing 4,000 samples to ensure balanced class distribution. Data collection involved 60,000 raw posts from various Persian social media platforms, followed by rigorous preprocessing and hybrid annotation combining ChatGPT-based few-shot prompting with human verification. To mitigate class imbalance, we employed undersampling with semantic redundancy removal and advanced data augmentation strategies integrating lexical replacement and generative prompting. We benchmarked several models, including BiLSTM, XLM-RoBERTa (with LoRA and AdaLoRA adaptations), FaBERT, SBERT-based architectures, and the Persian-specific TookaBERT (Base and Large). Experimental results show that transformer-based models consistently outperform traditional neural networks, with TookaBERT-Large achieving the best performance (Precision: 0.9622, Recall: 0.9621, F1- score: 0.9621). Class-wise evaluation further confirms robust performance across all categories, though social and political texts exhibited slightly lower scores due to inherent ambiguity. This research presents a new high-quality dataset and provides comprehensive evaluations of cutting-edge models, establishing a solid foundation for further developments in Persian NLP, including trend analysis, social behavior modeling, and user classification. The dataset is publicly available to support future research endeavors.
Abstract (translated)
这项研究引入了第一个大规模、平衡良好的波斯语社交媒体文本分类数据集，旨在解决该领域内综合性资源不足的问题。该数据集包含九个类别（经济、艺术、体育、政治、社会、健康、心理、历史和科技）的共计36,000篇帖子，每个类别包括4,000份样本，确保了类别的分布均衡。数据收集阶段从各种波斯语社交媒体平台中获取了60,000条原始帖子，并随后进行了严格的预处理以及结合ChatGPT少样本提示和人工验证的混合标注过程。 为缓解类别不平衡的问题，我们采用了欠采样并移除语义冗余的方法，同时采用包括词汇替换和生成性提示在内的高级数据增强策略。我们在多个模型（包括BiLSTM、XLM-RoBERTa及其LoRA和AdaLoRA适应版本、FaBERT、基于SBERT的架构以及专为波斯语设计的TookaBERT基础版和大型版）上进行了基准测试。 实验结果显示，基于转换器的模型始终优于传统的神经网络模型，其中TookaBERT-Large表现最佳（精确率：0.9622，召回率：0.9621，F1分数：0.9621）。类别级别的评估进一步证实了所有类别的稳健性能，尽管社会和政治文本由于固有的模糊性而得分略低。 这项研究不仅提供了一个新的高质量数据集，还全面评测了前沿模型的性能，为波斯语自然语言处理（NLP）领域的未来发展奠定了坚实的基础，包括趋势分析、社交行为建模及用户分类等。该数据集对未来的科研工作开放，可供下载使用。
URL
https://arxiv.org/abs/2602.19333
PDF
-
Setting up for failure: automatic discovery of the neural mechanisms of cognitive errors
2026-02-22 14:13:38Puria Radmard, Paul M. Bays, M\'at\'e LengyelAbstract
Discovering the neural mechanisms underpinning cognition is one of the grand challenges of neuroscience. However, previous approaches for building models of RNN dynamics that explain behaviour required iterative refinement of architectures and/or optimisation objectives, resulting in a piecemeal, and mostly heuristic, human-in-the-loop process. Here, we offer an alternative approach that automates the discovery of viable RNN mechanisms by explicitly training RNNs to reproduce behaviour, including the same characteristic errors and suboptimalities, that humans and animals produce in a cognitive task. Achieving this required two main innovations. First, as the amount of behavioural data that can be collected in experiments is often too limited to train RNNs, we use a non-parametric generative model of behavioural responses to produce surrogate data for training RNNs. Second, to capture all relevant statistical aspects of the data, we developed a novel diffusion model-based approach for training RNNs. To showcase the potential of our approach, we chose a visual working memory task as our test-bed, as behaviour in this task is well known to produce response distributions that are patently multimodal (due to swap errors). The resulting network dynamics correctly qualitative features of macaque neural data. Importantly, these results were not possible to obtain with more traditional approaches, i.e., when only a limited set of behavioural signatures (rather than the full richness of behavioural response distributions) were fitted, or when RNNs were trained for task optimality (instead of reproducing behaviour). Our approach also yields novel predictions about the mechanism of swap errors, which can be readily tested in experiments. These results suggest that fitting RNNs to rich patterns of behaviour provides a powerful way to automatically discover mechanisms of important cognitive functions.
Abstract (translated)
探索神经机制背后的认知原理是神经科学面临的重大挑战之一。然而，以往构建能够解释行为的RNN（递归神经网络）动态模型的方法需要对架构和/或优化目标进行迭代改进，导致了一个零散且主要依赖直觉的人工干预过程。在这里，我们提供了一种替代方法，通过明确训练RNN来复制人类和动物在认知任务中产生的行为及其特征性错误和次优表现，从而自动化发现可行的RNN机制。 实现这一目标需要两个主要创新： 1. **生成模型**：由于实验中收集的行为数据量通常不足以训练RNN，我们使用非参数生成模型来产生用于训练RNN的替代数据。 2. **扩散模型**：为了捕捉数据的所有相关统计特性，我们开发了一种基于扩散模型的新方法，用于训练RNN。 为展示我们方法的潜力，我们选择了一个视觉工作记忆任务作为测试平台，因为在该任务中行为产生的反应分布显然是多模态（由于交换错误）。由此产生的网络动态正确地再现了猕猴神经数据中的定性特征。重要的是，这些结果通过传统的做法是无法获得的，例如当仅拟合有限的行为特征集而不是完整的行为响应分布时，或者当RNN被训练以实现任务最优而非复制行为时。 我们的方法还提供了关于交换错误机制的新预测，这些预测可以在实验中直接测试。这些结果表明，将RNN拟合到丰富的行为模式上可以自动发现重要认知功能的机制，为理解大脑如何支持复杂认知过程提供了一种强有力的方法。
URL
https://arxiv.org/abs/2512.04808
PDF
