RNN
-
Endogenous Information in Routing Games: Memory-Constrained Equilibria, Recall Braess Paradoxes, and Memory Design
Abstract
We study routing games in which travelers optimize over routes that are remembered or surfaced, rather than over a fixed exogenous action set. The paper develops a tractable design theory for endogenous recall and then connects it back to an explicit finite-memory micro model. At the micro level, each traveler carries a finite memory state, receives surfaced alternatives, chooses via a logit rule, and updates memory under a policy such as LRU. This yields a stationary Forgetful Wardrop Equilibrium (FWE); existence is proved under mild regularity, and uniqueness follows in a contraction regime for the reduced fixed-point map. The paper's main design layer is a stationary salience model that summarizes persistent memory and interface effects as route-specific weights. Salience-weighted stochastic user equilibrium is the unique minimizer of a strictly convex potential, which yields a clean optimization and implementability theory. In this layer we characterize governed implementability under ratio budgets and affine tying constraints, and derive constructive algorithms on parallel and series-parallel networks. The bridge between layers is exact for last-choice memory (B=1): the micro model is then equivalent to the salience model, so any interior salience vector can be realized by an appropriate surfacing policy. For larger memories, we develop an explicit LRU-to-TTL-to-salience approximation pipeline and add contraction-based bounds that translate surrogate-map error into fixed-point and welfare error. Finally, we define a Recall Braess Paradox, in which improving recall increases equilibrium delay without changing physical capacity, and show that it can arise on every two-terminal network with at least two distinct s-t paths. Targeted experiments support the approximation regime, governed-design predictions, and the computational advantages of the reduced layer.
Abstract (translated)
URL
https://arxiv.org/abs/2604.11733
PDF
-
Deep Learning for Sequential Decision Making under Uncertainty: Foundations, Frameworks, and Frontiers
2026-04-13 14:11:06I. Esra BuyuktahtakinAbstract
Artificial intelligence (AI) is moving increasingly beyond prediction to support decisions in complex, uncertain, and dynamic environments. This shift creates a natural intersection with operations research and management sciences (OR/MS), which have long offered conceptual and methodological foundations for sequential decision-making under uncertainty. At the same time, recent advances in deep learning, including feedforward neural networks, LSTMs, transformers, and deep reinforcement learning, have expanded the scope of data-driven modeling and opened new possibilities for large-scale decision systems. This tutorial presents an OR/MS-centered perspective on deep learning for sequential decision-making under uncertainty. Its central premise is that deep learning is valuable not as a replacement for optimization, but as a complement to it. Deep learning brings adaptability and scalable approximation, whereas OR/MS provides the structural rigor needed to represent constraints, recourse, and uncertainty. The tutorial reviews key decision-making foundations, connects them to the major neural architectures in modern AI, and discusses leading approaches to integrating learning and optimization. It also highlights emerging impact in domains such as supply chains, healthcare and epidemic response, agriculture, energy, and autonomous operations. More broadly, it frames these developments as part of a wider transition from predictive AI toward decision-capable AI and highlights the role of OR/MS in shaping the next generation of integrated learning--optimization systems.
Abstract (translated)
URL
https://arxiv.org/abs/2604.11507
PDF
-
From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction
2026-04-13 12:15:52Adrienne Kline, Abhijit Gaonkar, Daniel Pittman, Chris Kuehn, Nils ForkertAbstract
Removing patient-specific information from medical images is crucial to enable sharing and open science without compromising patient identities. However, many methods currently used for deidentification have negative effects on downstream image analysis tasks because of removal of relevant but non-identifiable information. This work presents an end-to-end deep learning framework for transforming raw clinical image volumes into de-identified, analysis-ready datasets without compromising downstream utility. The methodology developed and tested in this work first detects and redacts regions likely to contain protected health information (PHI), such as burned-in text and metadata, and then uses a generative deep learning model to inpaint the redacted areas with anatomically and imaging plausible content. The proposed pipeline leverages a lightweight hybrid architecture, combining CRNN-based redaction with a latent-diffusion inpainting restoration module (Stable Diffusion 2). We evaluate the approach using both privacy-oriented metrics, which quantify residual PHI and success of redaction, and image-quality and task-based metrics, which assess the fidelity of restored volumes for representative deep learning applications. Our results suggest that the proposed method yields de-identified medical images that are visually coherent, maintaining fidelity for downstream models, while substantially reducing the risk of patient re-identification. By automating anonymization and image reconstruction within a single workflow, and dissemination of large-scale medical imaging collections, thereby lowering a key barrier to data sharing and multi-institutional collaboration in medical imaging AI.
Abstract (translated)
URL
https://arxiv.org/abs/2604.11376
PDF
-
Aletheia: Physics-Conditioned Localized Artifact Attention for End-to-End Generalizable and Robust Deepfake Video Detection
2026-04-13 08:14:50Devendra GhoriAbstract
State-of-the-art deepfake detectors achieve near-perfect in-domain accuracy yet degrade under cross-generator shifts, heavy compression, and adversarial perturbations. The core limitation remains the decoupling of semantic artifact learning from physical invariants: optical-flow discontinuities, specular-reflection inconsistencies, and cardiac-modulated reflectance (rPPG) are treated either as post-hoc features or ignored. We introduce PhyLAA-X, a novel physics-conditioned extension of Localized Artifact Attention (LAA-X). PhyLAA-X injects three end-to-end differentiable physics-derived feature volumes - optical-flow curl, specular-reflectance skewness, and spatially-upsampled rPPG power spectra - directly into the LAA-X attention computation via cross-attention gating and a resonance consistency loss. This forces the network to learn manipulation boundaries where semantic inconsistencies and physical violations co-occur - regions inherently harder for generative models to replicate consistently. PhyLAA-X is embedded across an efficient spatiotemporal ensemble (EfficientNet-B4+BiLSTM, ResNeXt-101+Transformer, Xception+causal Conv1D) with uncertainty-aware adaptive weighting. On FaceForensics++ (c23), Aletheia reaches 97.2% accuracy / 0.992 AUC-ROC; on Celeb-DF v2, 94.9% / 0.981; on DFDC, 90.8% / 0.966 - outperforming the strongest published baseline (LAA-Net [1]) by 4.1-7.3% in cross-generator settings and maintaining 79.4% accuracy under epsilon = 0.02 PGD-10 attacks. Single-backbone ablations confirm PhyLAA-X alone delivers a 4.2% cross-dataset AUC gain. The full production system is open-sourced at this https URL (v1.2, April 2026) with pretrained weights, the adversarial corpus (referred to as ADC-2026 in this work), and complete reproducibility artifacts.
Abstract (translated)
URL
https://arxiv.org/abs/2604.16486
PDF
-
Speech-preserving active noise control: a deep learning approach in reverberant environments
Abstract
Traditional Active Noise Control (ANC) systems are mostly based on FxLMS algorithms, but such algorithms rely on linear assumptions and are often limited in handling broadband non-stationary noise or nonlinear acoustic paths. Not only that, the traditional method is used to eliminating all signals together, and noise reduction often accidentally damages the voice signal and affects normal communication. To tackle these issues, this study proposes a speech preserving deep learning ANC system, which aims to achieve stable noise reduction while effectively retaining speech in a complex acoustic environment. This study builds an end-to-end control architecture, the core of which adopts a Convolutional Recurrent Network (CRN). The structure uses the long short-term memory (LSTM) network to capture the time-related characteristics of acoustic signals. Combined with complex spectrum mapping (CSM) technology, the nonlinear distortion problem is effectively solved. In order to retain useful voice while removing noise, this study also designs a special voice retention loss function. This design guidance model selectively retains the target voice while suppressing environmental noise by identifying the characteristics of the spectrum structure. In addition, in order to verify whether the system is effective in real scenes, we use the Image Source Method (ISM) to build a high-fidelity acoustic simulation environment, which also simulates the real reverberation effect. Experimental results demonstrate that the proposed Deep ANC system achieves significantly better noise reduction than the traditional FxLMS algorithm, especially for non-stationary noises like crowd babble. Meanwhile, PESQ and STOI based evaluations confirm that the system preserves both the naturalness and intelligibility of the target speech.
Abstract (translated)
URL
https://arxiv.org/abs/2604.10979
PDF
-
MSGL-Transformer: A Multi-Scale Global-Local Transformer for Rodent Social Behavior Recognition
2026-04-08 20:34:47Muhammad Imran Sharif, Doina CarageaAbstract
Recognition of rodent behavior is important for understanding neural and behavioral mechanisms. Traditional manual scoring is time-consuming and prone to human error. We propose MSGL-Transformer, a Multi-Scale Global-Local Transformer for recognizing rodent social behaviors from pose-based temporal sequences. The model employs a lightweight transformer encoder with multi-scale attention to capture motion dynamics across different temporal scales. The architecture integrates parallel short-range, medium-range, and global attention branches to explicitly capture behavior dynamics at multiple temporal scales. We also introduce a Behavior-Aware Modulation (BAM) block, inspired by SE-Networks, which modulates temporal embeddings to emphasize behavior-relevant features prior to attention. We evaluate on two datasets: RatSI (5 behavior classes, 12D pose inputs) and CalMS21 (4 behavior classes, 28D pose inputs). On RatSI, MSGL-Transformer achieves 75.4% mean accuracy and F1-score of 0.745 across nine cross-validation splits, outperforming TCN, LSTM, and Bi-LSTM. On CalMS21, it achieves 87.1% accuracy and F1-score of 0.8745, a +10.7% improvement over HSTWFormer, and outperforms ST-GCN, MS-G3D, CTR-GCN, and STGAT. The same architecture generalizes across both datasets with only input dimensionality and number of classes adjusted.
Abstract (translated)
URL
https://arxiv.org/abs/2604.07578
PDF
-
Event-Level Detection of Surgical Instrument Handovers in Videos with Interpretable Vision Models
2026-04-08 20:31:28Katerina Katsarou, George Zountsas, Karam Tomotaki-Dawoud, Alexander Ehrenhoefer, Paul Chojecki, David Przewozny, Igor Maximilian Sauer, Amira Mouakher, Sebastian BosseAbstract
Reliable monitoring of surgical instrument exchanges is essential for maintaining procedural efficiency and patient safety in the operating room. Automatic detection of instrument handovers in intraoperative video remains challenging due to frequent occlusions, background clutter, and the temporally evolving nature of interaction events. We propose a spatiotemporal vision framework for event-level detection and direction classification of surgical instrument handovers in surgical videos. The model combines a Vision Transformer (ViT) backbone for spatial feature extraction with a unidirectional Long Short-Term Memory (LSTM) network for temporal aggregation. A unified multi-task formulation jointly predicts handover occurrence and interaction direction, enabling consistent modeling of transfer dynamics while avoiding error propagation typical of cascaded pipelines. Predicted confidence scores form a temporal signal over the video, from which discrete handover events are identified via peak detection. Experiments on a dataset of kidney transplant procedures demonstrate strong performance, achieving an F1-score of 0.84 for handover detection and a mean F1-score of 0.72 for direction classification, outperforming both a single-task variant and a VideoMamba-based baseline for direction prediction while maintaining comparable detection performance. To improve interpretability, we employ Layer-CAM attribution to visualize spatial regions driving model decisions, highlighting hand-instrument interaction cues.
Abstract (translated)
URL
https://arxiv.org/abs/2604.07577
PDF
-
ClickGuard: A Trustworthy Adaptive Fusion Framework for Clickbait Detection
2026-04-08 16:33:47Chhavi Dhiman, Naman Chawla, Riya Dhami, Gaurav Kumar, Ganesh NaikAbstract
The widespread use of clickbait headlines, crafted to mislead and maximize engagement, poses a significant challenge to online credibility. These headlines employ sensationalism, misleading claims, and vague language, underscoring the need for effective detection to ensure trustworthy digital content. The paper introduces, ClickGuard: a trustworthy adaptive fusion framework for clickbait detection. It combines BERT embeddings and structural features using a Syntactic-Semantic Adaptive Fusion Block (SSAFB) for dynamic integration. The framework incorporates a hybrid CNN-BiLSTM to capture patterns and dependencies. The model achieved 96.93% testing accuracy, outperforming state-of-the-art approaches. The model's trustworthiness is evaluated using LIME and Permutation Feature Importance (PFI) for interpretability and perturbation analysis. These methods assess the model's robustness and sensitivity to feature changes by measuring the average prediction variation. Ablation studies validated the SSAFB's effectiveness in optimizing feature fusion. The model demonstrated robust performance across diverse datasets, providing a scalable, reliable solution for enhancing online content credibility by addressing syntactic-semantic modelling challenges. Code of the work is available at: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2604.07272
PDF
-
CWRNN-INVR: A Coupled WarpRNN based Implicit Neural Video Representation
2026-04-08 01:26:06Yiyang Li, Yanbo Gao, Shuai Li, Zhenyu Du, Jinglin Zhang, Hui Yuan, Mao Ye, Xingyu GaoAbstract
Implicit Neural Video Representation (INVR) has emerged as a novel approach for video representation and compression, using learnable grids and neural networks. Existing methods focus on developing new grid structures efficient for latent representation and neural network architectures with large representation capability, lacking the study on their roles in video representation. In this paper, the difference between INVR based on neural network and INVR based on grid is first investigated from the perspective of video information composition to specify their own advantages, i.e., neural network for general structure while grid for specific detail. Accordingly, an INVR based on mixed neural network and residual grid framework is proposed, where the neural network is used to represent the regular and structured information and the residual grid is used to represent the remaining irregular information in a video. A Coupled WarpRNN-based multi-scale motion representation and compensation module is specifically designed to explicitly represent the regular and structured information, thus terming our method as CWRNN-INVR. For the irregular information, a mixed residual grid is learned where the irregular appearance and motion information are represented together. The mixed residual grid can be combined with the coupled WarpRNN in a way that allows for network reuse. Experiments show that our method achieves the best reconstruction results compared with the existing methods, with an average PSNR of 33.73 dB on the UVG dataset under the 3M model and outperforms existing INVR methods in other downstream tasks. The code can be found at this https URL}{this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.06564
PDF
-
Predicting Alzheimer's disease progression using rs-fMRI and a history-aware graph neural network
2026-04-07 21:13:50Mahdi Moghaddami, Mohammad-Reza Siadat, Austin Toma, Connor Laming, Huirong FuAbstract
Alzheimer's disease (AD) is a neurodegenerative disorder that affects more than seven million people in the United States alone. AD currently has no cure, but there are ways to potentially slow its progression if caught early enough. In this study, we propose a graph neural network (GNN)-based model for predicting whether a subject will transition to a more severe stage of cognitive impairment at their next clinical visit. We consider three stages of cognitive impairment in order of severity: cognitively normal (CN), mild cognitive impairment (MCI), and AD. We use functional connectivity graphs derived from resting-state functional magnetic resonance imaging (rs-fMRI) scans of 303 subjects, each with a different number of visits. Our GNN-based model incorporates a recurrent neural network (RNN) block, enabling it to process data from the subject's entire visit history. It can also work with irregular time gaps between visits by incorporating visit distance information into our input features. Our model demonstrates robust predictive performance, even with missing visits in the subjects' visit histories. It achieves an accuracy of 82.9%, with an especially impressive accuracy of 68.8% on CN to MCI conversions - a task that poses a substantial challenge in the field. Our results highlight the effectiveness of rs-fMRI in predicting the onset of MCI or AD and, in conjunction with other modalities, could offer a viable method for enabling timely interventions to slow the progression of cognitive impairment.
Abstract (translated)
URL
https://arxiv.org/abs/2604.06469
PDF
-
Team Fusion@ SU@ BC8 SympTEMIST track: transformer-based approach for symptom recognition and linking
2026-04-07 20:00:59Georgi Grazhdanski, Sylvia Vassileva, Ivan Koychev, Svetla BoytchevaAbstract
This paper presents a transformer-based approach to solving the SympTEMIST named entity recognition (NER) and entity linking (EL) tasks. For NER, we fine-tune a RoBERTa-based (1) token-level classifier with BiLSTM and CRF layers on an augmented train set. Entity linking is performed by generating candidates using the cross-lingual SapBERT XLMR-Large (2), and calculating cosine similarity against a knowledge base. The choice of knowledge base proves to have the highest impact on model accuracy.
Abstract (translated)
URL
https://arxiv.org/abs/2604.06424
PDF
-
SE-Enhanced ViT and BiLSTM-Based Intrusion Detection for Secure IIoT and IoMT Environments
2026-04-06 20:48:50Afrah Gueriani, Hamza Kheddar, Ahmed Cherif Mazari, Seref Sagiroglu, Onur CeranAbstract
With the rapid growth of interconnected devices in Industrial and Medical Internet of Things (IIoT and MIoT) ecosystems, ensuring timely and accurate detection of cyber threats has become a critical challenge. This study presents an advanced intrusion detection framework based on a hybrid Squeeze-and-Excitation Attention Vision Transformer-Bidirectional Long Short-Term Memory (SE ViT-BiLSTM) architecture. In this design, the traditional multi-head attention mechanism of the Vision Transformer is replaced with Squeeze-and-Excitation attention, and integrated with BiLSTM layers to enhance detection accuracy and computational efficiency. The proposed model was trained and evaluated on two real-world benchmark datasets; EdgeIIoT and CICIoMT2024; both before and after data balancing using the Synthetic Minority Over-sampling Technique (SMOTE) and RandomOverSampler. Experimental results demonstrate that the SE ViT-BiLSTM model outperforms existing approaches across multiple metrics. Before balancing, the model achieved accuracies of 99.11% (FPR: 0.0013%, latency: 0.00032 sec/inst) on EdgeIIoT and 96.10% (FPR: 0.0036%, latency: 0.00053 sec/inst) on CICIoMT2024. After balancing, performance further improved, reaching 99.33% accuracy with 0.00035 sec/inst latency on EdgeIIoT and 98.16% accuracy with 0.00014 sec/inst latency on CICIoMT2024.
Abstract (translated)
URL
https://arxiv.org/abs/2604.06254
PDF
-
Sampling Matters: The Effect of ECG Frequency on Deep Learning-Based Atrial Fibrillation Detection
2026-04-06 20:34:42Arjan Mahmuod, Adrian Rod Hammerstad, Muzaffar Yousef, Yngve Sebastian Heill, Jonas L. Isaksen, J{\o}rgen K. Kanters, Pal Halvorsen, Vajira ThambawitaAbstract
Deep learning models for atrial fibrillation (AF) detection are increasingly trained on heterogeneous electrocardiogram (ECG) datasets with varying sampling frequencies, yet the specific consequences of these discrepancies on model performance, calibration, and robustness remain insufficiently characterized. To address this, we conducted a systematic benchmark using 12-lead, 10-second recordings from the PTB-XL dataset, resampled to target frequencies of 62, 100, 250, and 500 Hz, to evaluate a standard 1-D Convolutional Neural Network (CNN) and a hybrid CNN-Long Short-Term Memory (LSTM) architecture under a rigorous patient-safe cross-validation framework. Our analysis reveals that sampling frequency significantly impacts detection metrics in an architecture-dependent manner; the hybrid CNN-LSTM model demonstrated optimal performance and consistent calibration at intermediate frequencies (100-250 Hz), whereas the 1-D CNN baseline exhibited marked degradation in accuracy and sensitivity at 500 Hz, suggesting increased susceptibility to high-frequency noise. We conclude that ECG sampling frequency is a critical, underappreciated factor in arrhythmia detection, and future foundation models must explicitly control for temporal resolution to ensure clinical reliability and reproducibility.
Abstract (translated)
URL
https://arxiv.org/abs/2604.16437
PDF
-
AI Appeals Processor: A Deep Learning Approach to Automated Classification of Citizen Appeals in Government Services
2026-04-04 10:03:53Vladimir BeskorovainyiAbstract
Government agencies worldwide face growing volumes of citizen appeals, with electronic submissions increasing significantly over recent years. Traditional manual processing averages 20 minutes per appeal with only 67% classification accuracy, creating significant bottlenecks in public service delivery. This paper presents AI Appeals Processor, a microservice-based system that integrates natural language processing and deep learning techniques for automated classification and routing of citizen appeals. We evaluate multiple approaches -- including Bag-of-Words with SVM, TF-IDF with SVM, fastText, Word2Vec with LSTM, and BERT -- on a representative dataset of 10,000 real citizen appeals across three primary categories (complaints, applications, and proposals) and seven thematic domains. Our experiments demonstrate that a Word2Vec+LSTM architecture achieves 78% classification accuracy while reducing processing time by 54%, offering an optimal balance between accuracy and computational efficiency compared to transformer-based models.
Abstract (translated)
全球政府机构正面临日益增长的公民上诉量，其中电子提交方式在近年显著增加。传统人工处理每起上诉平均耗时20分钟，分类准确率仅67%，造成公共服务交付的严重瓶颈。本文提出AI上诉处理器——一个基于微服务的系统，集成自然语言处理与深度学习技术，实现公民上诉的自动分类与路由。我们在涵盖三类主要诉求（投诉、申请、建议）及七个主题领域的10,000条真实公民上诉数据集上，评估了多种方法：包括词袋模型结合支持向量机、TF-IDF结合支持向量机、fastText、Word2Vec结合长短期记忆网络，以及BERT。实验表明，Word2Vec+LSTM架构在保持78%分类准确率的同时，将处理时间缩短54%，相较于基于Transformer的模型，在准确率与计算效率间实现了最佳平衡。
URL
https://arxiv.org/abs/2604.03672
PDF
-
Leveraging Large Language Models for Sentiment Analysis: Multi-Modal Analysis of Decentraland's MANA Token
2026-04-04 04:04:13Xintong Wu, Peiting Tsai, Jing Yuan, Michael Yu, Greg Sun, Luyao ZhangAbstract
Decentraland, a decentralized virtual reality platform operating within the expanding Metaverse ecosystem, utilizes its native MANA token to facilitate virtual asset transactions and governance. This study investigates the integration of Discord community sentiment with multi-modal financial data to enhance cryptocurrency price prediction within virtual world economies. We address: (1) identifying sentiment patterns within Decentraland's Discord community, and (2) evaluating the impact of multi-modal features on token return forecasting. Using a BERT-based large language model for sentiment analysis, we develop two LSTM architectures: a baseline incorporating historical prices and a multi-modal variant integrating sentiment scores, trading volume, and market capitalization. Results indicate predominantly neutral community sentiment with a positive skew. The multi-modal model significantly outperforms the price-only baseline in prediction accuracy. These findings demonstrate the predictive value of community-derived signals for virtual economy forecasting and establish a foundation for future research at the intersection of immersive virtual environments, natural language processing, and cryptocurrency market analysis.
Abstract (translated)
URL
https://arxiv.org/abs/2605.20192
PDF
-
Olmo Hybrid: From Theory to Practice and Back
2026-04-03 20:36:34William Merrill, Yanhong Li, Tyler Romero, Anej Svete, Caia Costello, Pradeep Dasigi, Dirk Groeneveld, David Heineman, Bailey Kuehl, Nathan Lambert, Jacob Morrison, Luca Soldaini, Finbarr Timbers, Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi, Ashish SabharwalAbstract
Recent work has demonstrated the potential of non-transformer language models, especially linear recurrent neural networks (RNNs) and hybrid models that mix recurrence and attention. Yet there is no consensus on whether the potential benefits of these new architectures justify the risk and effort of scaling them up. To address this, we provide evidence for the advantages of hybrid models over pure transformers on several fronts. First, theoretically, we show that hybrid models do not merely inherit the expressivity of transformers and linear RNNs, but can express tasks beyond both, such as code execution. Putting this theory to practice, we train Olmo Hybrid, a 7B-parameter model largely comparable to Olmo 3 7B but with the sliding window layers replaced by Gated DeltaNet layers. We show that Olmo Hybrid outperforms Olmo 3 across standard pretraining and mid-training evaluations, demonstrating the benefit of hybrid models in a controlled, large-scale setting. We find that the hybrid model scales significantly more efficiently than the transformer, explaining its higher performance. However, its unclear why greater expressivity on specific formal problems should result in better scaling or superior performance on downstream tasks unrelated to those problems. To explain this apparent gap, we return to theory and argue why increased expressivity should translate to better scaling efficiency, completing the loop. Overall, our results suggest that hybrid models mixing attention and recurrent layers are a powerful extension to the language modeling paradigm: not merely to reduce memory during inference, but as a fundamental way to obtain more expressive models that scale better during pretraining.
Abstract (translated)
近期研究揭示了非Transformer语言模型的潜力，尤其是线性循环神经网络（RNNs）以及混合循环与注意力机制的混合模型。然而，学界尚未就这些新架构的潜在优势是否值得承担其扩展的风险与成本达成共识。为探讨此问题，我们从多个方面提供了混合模型优于纯Transformer的证据。首先，理论上我们证明混合模型不仅继承了Transformer与线性RNN的表达能力，更能表达两者无法处理的任务（如代码执行）。将理论付诸实践，我们训练了Olmo Hybrid——一个70亿参数模型，其架构与Olmo 3 7B高度相似，但将滑动窗口层替换为门控DeltaNet层。实验表明，Olmo Hybrid在标准预训练与中期评估中均超越Olmo 3，证实了混合模型在受控大规模场景中的优势。我们发现混合模型的扩展效率显著高于Transformer，这解释了其更优性能。然而，为何在特定形式化问题上更强的表达能力能转化为与这些问题无关的下游任务中更好的扩展性或性能，仍不明确。为解释这一明显矛盾，我们回归理论论证增强的表达能力如何转化为更优的扩展效率，从而形成完整逻辑闭环。总体而言，我们的结果表明，混合注意力与循环层的混合模型是对语言建模范式的有力拓展：其价值不仅在于降低推理时的内存消耗，更在于提供了一种获取更强表达能力、实现更优预训练扩展性的根本性途径。
URL
https://arxiv.org/abs/2604.03444
PDF
-
Activity-Dependent Plasticity in Morphogenetically-Grown Recurrent Networks
2026-04-03 18:35:13Sergii Medvid, Andrii Valenia, Mykola GlybovetsAbstract
Developmental approaches to neural architecture search grow functional networks from compact genomes through self-organisation, but the resulting networks operate with fixed post-growth weights. We characterise Hebbian and anti-Hebbian plasticity across 50,000 morphogenetically grown recurrent controllers (5M+ configurations on CartPole and Acrobot), then test whether co-evolutionary experiments -- where plasticity parameters are encoded in the genome and evolved alongside the developmental architecture -- recover these patterns independently. Our characterisation reveals that (1) anti-Hebbian plasticity significantly outperforms Hebbian for competent networks (Cohen's d = 0.53-0.64), (2) regret (fraction of oracle improvement lost under the best fixed setting) reaches 52-100%, and (3) plasticity's role shifts from fine-tuning to genuine adaptation under non-stationarity. Co-evolution independently discovers these patterns: on CartPole, 70% of runs evolve anti-Hebbian plasticity (p = 0.043); on Acrobot, evolution finds near-zero eta with mixed signs -- exactly matching the characterisation. A random-RNN control shows that anti-Hebbian dominance is generic to small recurrent networks, but the degree of topology-dependence is developmental-specific: regret is 2-6x higher for morphogenetically grown networks than for random graphs with matched topology statistics.
Abstract (translated)
神经架构搜索的发展方法通过自组织从紧凑基因组中生成功能网络，但生成的网络使用固定的生长后权重运行。我们在5万个形态发生生长的循环控制器（在CartPole和Acrobot上超过500万个配置）中刻画赫布与反赫布可塑性，并检验协同进化实验——其中可塑性参数被编码在基因组中并与发育架构共同进化——是否能独立复现这些模式。我们的刻画揭示：(1) 对于胜任的网络，反赫布可塑性显著优于赫布可塑性（Cohen's d = 0.53-0.64）；(2) 遗憾（在最佳固定设置下损失的神谕改进比例）达到52-100%；(3) 在非平稳条件下，可塑性的角色从微调转变为真正的适应。协同进化独立发现了这些模式：在CartPole上，70%的运行进化出反赫布可塑性（p = 0.043）；在Acrobot上，进化发现接近零的eta且符号混合——与刻画完全匹配。随机RNN对照表明，反赫布主导性对小规模循环网络具有普遍性，但拓扑依赖程度具有发育特异性：形态发生生长的网络的遗憾比拓扑统计匹配的随机图高2-6倍。
URL
https://arxiv.org/abs/2604.03386
PDF
-
A Flow Matching Framework for Soft-Robot Inverse Dynamics
2026-04-03 12:45:46Hang Yang, Fangju Yang, Yangming Zhang, Ibrahim Alsarraj, Yuhao Wang, Zhenye Luo, Zixi Chen, Ke WuAbstract
Learning the inverse dynamics of soft continuum robots remains challenging due to high-dimensional nonlinearities and complex actuation coupling. Conventional feedback-based controllers often suffer from control chattering due to corrective oscillations, while deterministic regression-based learners struggle to capture the complex nonlinear mappings required for accurate dynamic tracking. Motivated by these limitations, we propose an inverse-dynamics framework for open-loop feedforward control that learns the system's differential dynamics as a generative transport map. Specifically, inverse dynamics is reformulated as a conditional flow-matching problem, and Rectified Flow (RF) is adopted as a lightweight instance to generate physically consistent control inputs rather than conditional averages. Two variants are introduced to further enhance physical consistency: RF-Physical, utilizing a physics-based prior for residual modeling; and RF-FWD, integrating a forward-dynamics consistency loss during flow matching. Extensive evaluations demonstrate that our framework reduces trajectory tracking RMSE by over 50% compared to standard regression baselines (MLP, LSTM, Transformer). The system sustains stable open-loop execution at a peak end-effector velocity of 1.14 m/s with sub-millisecond inference latency (0.995 ms). This work demonstrates flow matching as a robust, high-performance paradigm for learning differential inverse dynamics in soft robotic systems.
Abstract (translated)
学习软体连续体机器人的逆动力学仍是一项挑战，因其具有高维非线性与复杂驱动耦合特性。传统基于反馈的控制器常因校正振荡导致控制颤振，而基于确定性回归的学习器则难以捕捉精确动态跟踪所需的复杂非线性映射。受限于上述不足，本文提出一种用于开环前馈控制的逆动力学框架，该框架将系统微分动力学学习为生成式传输映射。具体而言，逆动力学被重构为条件流匹配问题，并采用整流流作为轻量级实例，以生成物理一致的控制输入而非条件均值。为进一步提升物理一致性，引入两种变体：利用物理先验进行残差建模的RF-Physical，以及在流匹配中集成前向动力学一致性损失的RF-FWD。广泛评估表明，与标准回归基线（MLP、LSTM、Transformer）相比，本框架将轨迹跟踪均方根误差降低了50%以上。系统在峰值末端执行器速度达1.14米/秒时仍能保持稳定开环运行，且推理延迟低于毫秒级（0.995毫秒）。本研究验证了流匹配作为学习软体机器人系统微分逆动力学的稳健高性能范式。
URL
https://arxiv.org/abs/2604.03006
PDF
-
Towards Intelligent Energy Security: A Unified Spatio-Temporal and Graph Learning Framework for Scalable Electricity Theft Detection in Smart Grids
2026-04-03 09:57:37AbdulQoyum A. Olowookere, Usman A. Oguntola, Ebenezer. Leke Odekanle, Maridiyah A. Madehin, Aisha A. AdesopeAbstract
Electricity theft and non-technical losses (NTLs) remain critical challenges in modern smart grids, causing significant economic losses and compromising grid reliability. This study introduces the SmartGuard Energy Intelligence System (SGEIS), an integrated artificial intelligence framework for electricity theft detection and intelligent energy monitoring. The proposed system combines supervised machine learning, deep learning-based time-series modeling, Non-Intrusive Load Monitoring (NILM), and graph-based learning to capture both temporal and spatial consumption patterns. A comprehensive data processing pipeline is developed, incorporating feature engineering, multi-scale temporal analysis, and rule-based anomaly labeling. Deep learning models, including Long Short-Term Memory (LSTM), Temporal Convolutional Networks (TCN), and Autoencoders, are employed to detect abnormal usage patterns. In parallel, ensemble learning methods such as Random Forest, Gradient Boosting, XGBoost, and LightGBM are utilized for classification. To model grid topology and spatial dependencies, Graph Neural Networks (GNNs) are applied to identify correlated anomalies across interconnected nodes. The NILM module enhances interpretability by disaggregating appliance-level consumption from aggregate signals. Experimental results demonstrate strong performance, with Gradient Boosting achieving a ROC-AUC of 0.894, while graph-based models attain over 96% accuracy in identifying high-risk nodes. The hybrid framework improves detection robustness by integrating temporal, statistical, and spatial intelligence. Overall, SGEIS provides a scalable and practical solution for electricity theft detection, offering high accuracy, improved interpretability, and strong potential for real-world smart grid deployment.
Abstract (translated)
窃电与非技术损耗（NTLs）仍是现代智能电网面临的关键挑战，不仅造成重大经济损失，还损害了电网可靠性。本研究提出了智能卫士能源智能系统（SGEIS），这是一个用于窃电检测与智能能源监测的集成人工智能框架。该系统结合了监督机器学习、基于深度学习的时间序列建模、非侵入式负载监测（NILM）以及基于图的学习，以捕捉时空用电模式。研究开发了全面的数据处理流程，涵盖特征工程、多尺度时间分析和基于规则的异常标注。采用深度学习模型（包括长短期记忆网络（LSTM）、时间卷积网络（TCN）和自编码器）检测异常用电模式，同时利用随机森林、梯度提升、XGBoost和LightGBM等集成学习方法进行分类。为模拟电网拓扑与空间依赖性，应用图神经网络（GNNs）识别互联节点间的关联异常。NILM模块通过从聚合信号中分解电器级用电量，增强了系统可解释性。实验结果表明系统性能优异：梯度提升模型的ROC-AUC值达0.894，基于图的模型在识别高风险节点时准确率超过96%。该混合框架通过整合时间、统计与空间智能，提升了检测鲁棒性。总体而言，SGEIS为窃电检测提供了可扩展且实用的解决方案，具备高精度、强可解释性，并在实际智能电网部署中展现出巨大潜力。
URL
https://arxiv.org/abs/2604.03344
PDF
-
A Multi-head-based architecture for effective morphological tagging in Russian with open dictionary
Abstract
The article proposes a new architecture based on Multi-head attention to solve the problem of morphological tagging for the Russian language. The preprocessing of the word vectors includes splitting the words into subtokens, followed by a trained procedure for aggregating the vectors of the subtokens into vectors for tokens. This allows to support an open dictionary and analyze morphological features taking into account parts of words (prefixes, endings, etc.). The open dictionary allows in future to analyze words that are absent in the training dataset. The performed computational experiment on the SinTagRus and Taiga datasets shows that for some grammatical categories the proposed architecture gives accuracy 98-99% and above, which outperforms previously known results. For nine out of ten words, the architecture precisely predicts all grammatical categories and indicates when the categories must not be analyzed for the word. At the same time, the model based on the proposed architecture can be trained on consumer-level graphics accelerators, retains all the advantages of Multi-head attention over RNNs (RNNs are not used in the proposed approach), does not require pretraining on large collections of unlabeled texts (like BERT), and shows higher processing speed than previous results.
Abstract (translated)
本文提出了一种基于多头注意力机制的新架构，用于解决俄语形态标注问题。词向量的预处理包括将单词拆分为子词，随后通过训练好的程序将子词向量聚合为词向量。这支持使用开放词典，并能在分析形态特征时考虑单词的各个部分（如前缀、词尾等）。开放词典使得未来能够分析训练数据集中未出现的单词。在SinTagRus和Taiga数据集上进行的计算实验表明，对于某些语法范畴，该架构的准确率可达98%-99%以上，优于此前已知结果。每十个词中有九个，该架构能精确预测所有语法范畴，并能在无需分析某词的语法范畴时进行标注。同时，基于该架构的模型可在消费级显卡上训练，保留了多头注意力机制相对于循环神经网络（本方法未使用RNN）的所有优势，无需在大规模无标注文本上进行预训练（如BERT），且处理速度高于此前成果。
URL
https://arxiv.org/abs/2604.02926
PDF
