RNN
-
DTT-BSR: GAN-based DTTNet with RoPE Transformer Enhancement for Music Source Restoration
2026-02-23 13:23:22Shihong Tan, Haoyu Wang, Youran Ni, Yingzhao Hou, Jiayue Luo, Zipei Hu, Han Dou, Zerui Han, Ningning Pan, Yuzhu Wang, Gongping HuangAbstract
Music source restoration (MSR) aims to recover unprocessed stems from mixed and mastered recordings. The challenge lies in both separating overlapping sources and reconstructing signals degraded by production effects such as compression and reverberation. We therefore propose DTT-BSR, a hybrid generative adversarial network (GAN) combining rotary positional embeddings (RoPE) transformer for long-term temporal modeling with dual-path band-split recurrent neural network (RNN) for multi-resolution spectral processing. Our model achieved 3rd place on the objective leaderboard and 4th place on the subjective leaderboard on the ICASSP 2026 MSR Challenge, demonstrating exceptional generation fidelity and semantic alignment with a compact size of 7.1M parameters.
Abstract (translated)
音乐源恢复（MSR）的目标是从混合和母带处理的录音中恢复未经处理的音轨。挑战在于既要分离重叠的声音来源，又要重建因如压缩、混响等生产效果而退化的信号。因此，我们提出了DTT-BSR模型，这是一种结合了旋转位置嵌入（RoPE）变压器进行长期时序建模和双路径频带分割递归神经网络（RNN）进行多分辨率谱处理的混合生成对抗网络（GAN）。我们的模型在ICASSP 2026 MSR挑战赛中以7.1M参数量获得了客观排行榜第3名和主观排行榜第4名的成绩，展示了卓越的生成保真度和语义对齐能力。
URL
https://arxiv.org/abs/2602.19825
PDF
-
PerSoMed: A Large-Scale Balanced Dataset for Persian Social Media Text Classification
2026-02-22 20:53:08Isun Chehreh, Ebrahim AnsariAbstract
This research introduces the first large-scale, well-balanced Persian social media text classification dataset, specifically designed to address the lack of comprehensive resources in this domain. The dataset comprises 36,000 posts across nine categories (Economic, Artistic, Sports, Political, Social, Health, Psychological, Historical, and Science & Technology), each containing 4,000 samples to ensure balanced class distribution. Data collection involved 60,000 raw posts from various Persian social media platforms, followed by rigorous preprocessing and hybrid annotation combining ChatGPT-based few-shot prompting with human verification. To mitigate class imbalance, we employed undersampling with semantic redundancy removal and advanced data augmentation strategies integrating lexical replacement and generative prompting. We benchmarked several models, including BiLSTM, XLM-RoBERTa (with LoRA and AdaLoRA adaptations), FaBERT, SBERT-based architectures, and the Persian-specific TookaBERT (Base and Large). Experimental results show that transformer-based models consistently outperform traditional neural networks, with TookaBERT-Large achieving the best performance (Precision: 0.9622, Recall: 0.9621, F1- score: 0.9621). Class-wise evaluation further confirms robust performance across all categories, though social and political texts exhibited slightly lower scores due to inherent ambiguity. This research presents a new high-quality dataset and provides comprehensive evaluations of cutting-edge models, establishing a solid foundation for further developments in Persian NLP, including trend analysis, social behavior modeling, and user classification. The dataset is publicly available to support future research endeavors.
Abstract (translated)
这项研究引入了第一个大规模、平衡良好的波斯语社交媒体文本分类数据集，旨在解决该领域内综合性资源不足的问题。该数据集包含九个类别（经济、艺术、体育、政治、社会、健康、心理、历史和科技）的共计36,000篇帖子，每个类别包括4,000份样本，确保了类别的分布均衡。数据收集阶段从各种波斯语社交媒体平台中获取了60,000条原始帖子，并随后进行了严格的预处理以及结合ChatGPT少样本提示和人工验证的混合标注过程。 为缓解类别不平衡的问题，我们采用了欠采样并移除语义冗余的方法，同时采用包括词汇替换和生成性提示在内的高级数据增强策略。我们在多个模型（包括BiLSTM、XLM-RoBERTa及其LoRA和AdaLoRA适应版本、FaBERT、基于SBERT的架构以及专为波斯语设计的TookaBERT基础版和大型版）上进行了基准测试。 实验结果显示，基于转换器的模型始终优于传统的神经网络模型，其中TookaBERT-Large表现最佳（精确率：0.9622，召回率：0.9621，F1分数：0.9621）。类别级别的评估进一步证实了所有类别的稳健性能，尽管社会和政治文本由于固有的模糊性而得分略低。 这项研究不仅提供了一个新的高质量数据集，还全面评测了前沿模型的性能，为波斯语自然语言处理（NLP）领域的未来发展奠定了坚实的基础，包括趋势分析、社交行为建模及用户分类等。该数据集对未来的科研工作开放，可供下载使用。
URL
https://arxiv.org/abs/2602.19333
PDF
-
Setting up for failure: automatic discovery of the neural mechanisms of cognitive errors
2026-02-22 14:13:38Puria Radmard, Paul M. Bays, M\'at\'e LengyelAbstract
Discovering the neural mechanisms underpinning cognition is one of the grand challenges of neuroscience. However, previous approaches for building models of RNN dynamics that explain behaviour required iterative refinement of architectures and/or optimisation objectives, resulting in a piecemeal, and mostly heuristic, human-in-the-loop process. Here, we offer an alternative approach that automates the discovery of viable RNN mechanisms by explicitly training RNNs to reproduce behaviour, including the same characteristic errors and suboptimalities, that humans and animals produce in a cognitive task. Achieving this required two main innovations. First, as the amount of behavioural data that can be collected in experiments is often too limited to train RNNs, we use a non-parametric generative model of behavioural responses to produce surrogate data for training RNNs. Second, to capture all relevant statistical aspects of the data, we developed a novel diffusion model-based approach for training RNNs. To showcase the potential of our approach, we chose a visual working memory task as our test-bed, as behaviour in this task is well known to produce response distributions that are patently multimodal (due to swap errors). The resulting network dynamics correctly qualitative features of macaque neural data. Importantly, these results were not possible to obtain with more traditional approaches, i.e., when only a limited set of behavioural signatures (rather than the full richness of behavioural response distributions) were fitted, or when RNNs were trained for task optimality (instead of reproducing behaviour). Our approach also yields novel predictions about the mechanism of swap errors, which can be readily tested in experiments. These results suggest that fitting RNNs to rich patterns of behaviour provides a powerful way to automatically discover mechanisms of important cognitive functions.
Abstract (translated)
探索神经机制背后的认知原理是神经科学面临的重大挑战之一。然而，以往构建能够解释行为的RNN（递归神经网络）动态模型的方法需要对架构和/或优化目标进行迭代改进，导致了一个零散且主要依赖直觉的人工干预过程。在这里，我们提供了一种替代方法，通过明确训练RNN来复制人类和动物在认知任务中产生的行为及其特征性错误和次优表现，从而自动化发现可行的RNN机制。 实现这一目标需要两个主要创新： 1. **生成模型**：由于实验中收集的行为数据量通常不足以训练RNN，我们使用非参数生成模型来产生用于训练RNN的替代数据。 2. **扩散模型**：为了捕捉数据的所有相关统计特性，我们开发了一种基于扩散模型的新方法，用于训练RNN。 为展示我们方法的潜力，我们选择了一个视觉工作记忆任务作为测试平台，因为在该任务中行为产生的反应分布显然是多模态（由于交换错误）。由此产生的网络动态正确地再现了猕猴神经数据中的定性特征。重要的是，这些结果通过传统的做法是无法获得的，例如当仅拟合有限的行为特征集而不是完整的行为响应分布时，或者当RNN被训练以实现任务最优而非复制行为时。 我们的方法还提供了关于交换错误机制的新预测，这些预测可以在实验中直接测试。这些结果表明，将RNN拟合到丰富的行为模式上可以自动发现重要认知功能的机制，为理解大脑如何支持复杂认知过程提供了一种强有力的方法。
URL
https://arxiv.org/abs/2512.04808
PDF
-
AMV-L: Lifecycle-Managed Agent Memory for Tail-Latency Control in Long-Running LLM Systems
Abstract
Long-running LLM agents require persistent memory to preserve state across interactions, yet most deployed systems manage memory with age-based retention (e.g., TTL). While TTL bounds item lifetime, it does not bound the computational footprint of memory on the request path: as retained items accumulate, retrieval candidate sets and vector similarity scans can grow unpredictably, yielding heavy-tailed latency and unstable throughput. We present AMV-L (Adaptive Memory Value Lifecycle), a memory-management framework that treats agent memory as a managed systems resource. AMV-L assigns each memory item a continuously updated utility score and uses value-driven promotion, demotion, and eviction to maintain lifecycle tiers; retrieval is restricted to a bounded, tier-aware candidate set that decouples the request-path working set from total retained memory. We implement AMV-L in a full-stack LLM serving system and evaluate it under identical long-running workloads against two baselines: TTL and an LRU working-set policy, with fixed prompt-injection caps. Relative to TTL, AMV-L improves throughput by 3.1x and reduces latency by 4.2x (median), 4.7x (p95), and 4.4x (p99), while reducing the fraction of requests exceeding 2s from 13.8% to 0.007%. Compared to LRU, AMV-L trades a small regression in median/p95 latency (+26% / +3%) for improved extreme-tail behavior (-15% p99; -98% >2s) and lower token overhead (approximately 6% fewer tokens/request), while matching retrieval quality (value means within approximately 0-2%). The gains arise primarily from bounding retrieval-set size and vector-search work, not from shortening prompts. Our results show that predictable performance for long-running LLM agents requires explicit control of memory working-set size and value-driven lifecycle management, rather than retention time alone.
Abstract (translated)
长时间运行的LLM代理需要持久性内存来跨交互保持状态，但大多数部署系统使用基于年龄的保留策略（例如TTL）管理内存。虽然TTL可以限制项目的生命期，但它并不能约束请求路径上内存的计算开销：随着被保留项目的积累，检索候选集和向量相似度扫描可能会无序增长，导致尾部延迟加重和吞吐量不稳定。我们提出了AMV-L（自适应记忆价值生命周期），这是一个将代理内存视为管理系统的资源的记忆管理框架。AMV-L为每个内存项分配一个持续更新的效用分数，并使用基于价值驱动的提升、降级和驱逐机制来维持生命周期层级；检索被限制在一个有界且层次感知的候选集中，从而解耦请求路径的工作集与总保留内存的关系。 我们在整个LLM服务系统中实现了AMV-L，并在其下进行了评估，条件是在相同的长时间运行工作负载下对比两个基准：TTL和一个固定提示注入上限的LRU工作集策略。相比TTL，AMV-L在吞吐量上提高了3.1倍，在延迟上减少了4.2倍（中位数）、4.7倍（p95）以及4.4倍（p99），同时将超过2秒请求的比例从13.8%降低至0.007%。 与LRU相比，AMV-L在中位数和p95延迟方面略有退步（+26%，+3%），但在极端尾部行为上表现更好（-15% p99；-98% > 2s）并且具有较低的令牌开销（约减少每请求6%的令牌），同时检索质量相当（值均在大约0至2%范围内）。这些改进主要来源于限制检索集合大小和向量搜索工作的计算负担，而不是缩短提示。 我们的结果显示，对于长时间运行的LLM代理而言，可预测性能需要对内存工作集大小进行显式控制以及基于价值驱动的生命周期管理，而不仅仅是保留时间。
URL
https://arxiv.org/abs/2603.04443
PDF
-
TactEx: An Explainable Multimodal Robotic Interaction Framework for Human-Like Touch and Hardness Estimation
2026-02-21 22:18:17Felix Verstraete, Lan Wei, Wen Fan, Dandan ZhangAbstract
Accurate perception of object hardness is essential for safe and dexterous contact-rich robotic manipulation. Here, we present TactEx, an explainable multimodal robotic interaction framework that unifies vision, touch, and language for human-like hardness estimation and interactive guidance. We evaluate TactEx on fruit-ripeness assessment, a representative task that requires both tactile sensing and contextual understanding. The system fuses GelSight-Mini tactile streams with RGB observations and language prompts. A ResNet50+LSTM model estimates hardness from sequential tactile data, while a cross-modal alignment module combines visual cues with guidance from a large language model (LLM). This explainable multimodal interface allows users to distinguish ripeness levels with statistically significant class separation (p < 0.01 for all fruit pairs). For touch placement, we compare YOLO with Grounded-SAM (GSAM) and find GSAM to be more robust for fine-grained segmentation and contact-site selection. A lightweight LLM parses user instructions and produces grounded natural-language explanations linked to the tactile outputs. In end-to-end evaluations, TactEx attains 90% task success on simple user queries and generalises to novel tasks without large-scale tuning. These results highlight the promise of combining pretrained visual and tactile models with language grounding to advance explainable, human-like touch perception and decision-making in robotics.
Abstract (translated)
准确地感知物体的硬度对于安全和灵巧的操作密集型机器人操作至关重要。在这里，我们介绍了TactEx，这是一个可解释的多模态机器人交互框架，它将视觉、触觉和语言统一起来以实现人类级别的硬度估计和互动指导。我们在水果成熟度评估这一任务上对TactEx进行了评价，该任务需要触摸感应与上下文理解能力。系统融合了GelSight-Mini触觉流、RGB观察结果以及语言提示。 一个基于ResNet50+LSTM的模型从连续触觉数据中估计硬度，而跨模态对齐模块则结合视觉线索和大型语言模型（LLM）提供的指导信息。这种可解释的多模态界面允许用户通过显著的类别分离（所有水果配对的p值均小于0.01）来区分成熟度级别。 对于触摸放置的位置选择，我们将YOLO与Grounded-SAM (GSAM)进行了比较，并发现GSAM在细粒度分割和接触点选取方面更为稳健。一个轻量级的语言模型解析用户指令并生成与触觉输出相关联的自然语言解释。在端到端评估中，TactEx对简单用户查询的成功率为90%，并且能够泛化到新任务而无需大规模调整。 这些结果突显了结合预训练视觉和触觉模型以及语言接地以推进机器人领域可解释、人类级别的触摸感知与决策制定的潜力。
URL
https://arxiv.org/abs/2602.18967
PDF
-
From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning
2026-02-21 10:51:49Suhasnadh Reddy Veluru, Sai Teja Erukude, Viswa Chaitanya MarellaAbstract
Accurate short-term power load forecasting is important to effectively manage, optimize, and ensure the robustness of modern power systems. This paper performs an empirical evaluation of a traditional statistical model and deep learning approaches for predicting short-term energy load. Four models, namely ARIMA, LSTM, BiLSTM, and Transformer, were leveraged on the PJM Hourly Energy Consumption data. The data processing involved interpolation, normalization, and a sliding-window sequence method. Each model's forecasting performance was evaluated for the 24-hour horizon using MAE, RMSE, and MAPE. Of the models tested, the Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE, with performance above any model in both accuracy and robustness. These findings underscore the growing potential of attention-based architectures in accurately capturing complex temporal patterns in power consumption data.
Abstract (translated)
准确的短期电力负荷预测对于有效管理、优化和确保现代电力系统的鲁棒性至关重要。本文对传统统计模型和深度学习方法在预测短期能源负载方面的性能进行了实证评估。研究中使用了PJM小时用电量数据，测试了四种模型：ARIMA、LSTM、BiLSTM 和 Transformer。数据处理包括插值、归一化以及滑动窗口序列法。每种模型的预测性能均通过24小时范围内的MAE（平均绝对误差）、RMSE（均方根误差）和MAPE（平均绝对百分比误差）进行了评估。 在所有测试的模型中，基于自注意力算法的Transformer模型表现最佳，其MAPE值为3.8%，不仅准确率高于其他任何一种模型，在鲁棒性方面也同样表现出色。这些发现强调了基于注意力机制的架构在捕捉电力消耗数据中的复杂时间模式方面的巨大潜力和价值。
URL
https://arxiv.org/abs/2603.06622
PDF
-
UFO: Unlocking Ultra-Efficient Quantized Private Inference with Protocol and Algorithm Co-Optimization
2026-02-21 08:45:05Wenxuan Zeng, Chao Yang, Tianshi Xu, Bo Zhang, Changrui Ren, Jin Dong, Meng LiAbstract
Private convolutional neural network (CNN) inference based on secure two-party computation (2PC) suffers from high communication and latency overhead, especially from convolution layers. In this paper, we propose UFO, a quantized 2PC inference framework that jointly optimizes the 2PC protocols and quantization algorithm. UFO features a novel 2PC protocol that systematically combines the efficient Winograd convolution algorithm with quantization to improve inference efficiency. However, we observe that naively combining quantization and Winograd convolution faces the following challenges: 1) From the inference perspective, Winograd transformations introduce extensive additions and require frequent bit width conversions to avoid inference overflow, leading to non-negligible communication overhead; 2) From the training perspective, Winograd transformations introduce weight outliers that make quantization-aware training (QAT) difficult, resulting in inferior model accuracy. To address these challenges, we co-optimize both protocol and algorithm. 1) At the protocol level, we propose a series of graph-level optimizations for 2PC inference to minimize the communication. 2) At the algorithm level, we develop a mixed-precision QAT algorithm based on layer sensitivity to optimize model accuracy given communication constraints. To accommodate the outliers, we further introduce a 2PC-friendly bit re-weighting algorithm to increase the representation range without explicitly increasing bit widths. With extensive experiments, UFO demonstrates 11.7x, 3.6x, and 6.3x communication reduction with 1.29%, 1.16%, and 1.29% higher accuracy compared to state-of-the-art frameworks SiRNN, COINN, and CoPriv, respectively.
Abstract (translated)
基于安全的两方计算（2PC）的私有卷积神经网络（CNN）推理在通信和延迟开销方面面临挑战，尤其是在卷积层。本文提出了UFO框架，这是一个量化后的2PC推理框架，它同时优化了2PC协议和量化算法。UFO的特点是一种新型的2PC协议，该协议系统地将高效的Winograd卷积算法与量化相结合，以提高推断效率。然而，我们观察到直接结合量化和Winograd卷积会面临以下挑战：1）从推理角度来看，Winograd变换引入了大量加法运算，并需要频繁进行位宽转换以避免超出推断范围，导致不可忽视的通信开销；2）从训练角度看，Winograd变换引入了权重异常值，使得量化感知训练（QAT）变得困难，从而降低了模型精度。为了解决这些挑战，我们对协议和算法进行了协同优化：1）在协议层面，我们提出了一系列针对2PC推理的图级优化措施以最小化通信开销；2）在算法层面上，我们开发了一种基于层次敏感性的混合精度QAT算法，在给定通信约束的情况下提高模型精度。为了适应异常值，我们进一步引入了一个友好的位重新加权算法，以增加表示范围而无需显式增加位宽。 通过广泛的实验，UFO分别比现有的SiRNN、COINN和CoPriv框架减少了11.7倍、3.6倍和6.3倍的通信开销，并且在精度上分别提高了1.29%、1.16%和1.29%，展示了其卓越性能。
URL
https://arxiv.org/abs/2602.18758
PDF
-
Subgroups of $U$ Induce Natural RNN and Transformer Architectures
Abstract
This paper presents a direct framework for sequence models with hidden states on closed subgroups of U(d). We use a minimal axiomatic setup and derive recurrent and transformer templates from a shared skeleton in which subgroup choice acts as a drop-in replacement for state space, tangent projection, and update map. We then specialize to O(d) and evaluate orthogonal-state RNN and transformer models on Tiny Shakespeare and Penn Treebank under parameter-matched settings. We also report a general linear-mixing extension in tangent space, which applies across subgroup choices and improves finite-budget performance in the current O(d) experiments.
Abstract (translated)
本文提出了一种直接框架，用于在U(d)的闭子群上的隐状态序列模型。我们使用了最小公理设置，并从一个共享骨架中推导出了递归和变换模板，在此骨架中，子群选择作为状态空间、切线投影和更新映射的即插即用替换。然后，我们将框架应用于O(d)，并在参数匹配设置下对Tiny Shakespeare和Penn Treebank数据集进行了正交状态RNN和Transformer模型的评估。此外，我们还报告了一种通用的线性混合扩展方法，在切空间中应用，并且这种扩展适用于各种子群选择，可改进当前O(d)实验中的有限预算性能。
URL
https://arxiv.org/abs/2602.18417
PDF
-
Leakage and Second-Order Dynamics Improve Hippocampal RNN Replay
2026-02-20 18:07:09Josue Casco-Rodriguez, Nanda H. Krishna, Richard G. BaraniukAbstract
Biological neural networks (like the hippocampus) can internally generate "replay" resembling stimulus-driven activity. Recent computational models of replay use noisy recurrent neural networks (RNNs) trained to path-integrate. Replay in these networks has been described as Langevin sampling, but new modifiers of noisy RNN replay have surpassed this description. We re-examine noisy RNN replay as sampling to understand or improve it in three ways: (1) Under simple assumptions, we prove that the gradients replay activity should follow are time-varying and difficult to estimate, but readily motivate the use of hidden state leakage in RNNs for replay. (2) We confirm that hidden state adaptation (negative feedback) encourages exploration in replay, but show that it incurs non-Markov sampling that also slows replay. (3) We propose the first model of temporally compressed replay in noisy path-integrating RNNs through hidden state momentum, connect it to underdamped Langevin sampling, and show that, together with adaptation, it counters slowness while maintaining exploration. We verify our findings via path-integration of 2D triangular and T-maze paths and of high-dimensional paths of synthetic rat place cell activity.
Abstract (translated)
生物学神经网络（如海马体）可以内部生成类似刺激驱动活动的“回放”。最近关于回放的计算模型使用了通过路径积分训练的带有噪声的递归神经网络（RNN）。在这些网络中的回放被描述为兰乔VIN采样，但对带噪RNN回放的新修正已经超越了这一描述。我们从三个角度重新审视带噪RNN回放作为采样的理解或改进：(1) 在简单的假设下，我们证明回放活动应该遵循的时间变化梯度难以估计，但却能轻易地推动在RNN中使用隐藏状态泄漏进行回放。(2) 我们确认了隐藏状态适应（负反馈）鼓励回放在探索中的作用，但同时也表明这导致非马尔可夫采样，并且减缓了回放的速度。(3) 我们提出首个通过隐藏状态动量在带噪路径积分RNN中实现时间压缩回放的模型，并将其与欠阻尼兰乔VIN采样联系起来，展示出它结合适应性时能克服缓慢性并保持探索。我们通过二维三角和T形迷宫以及合成大鼠位置细胞活动的高维路径中的路径积来验证我们的发现。
URL
https://arxiv.org/abs/2602.18401
PDF
-
PhysE-Inv: A Physics-Encoded Inverse Modeling approach for Arctic Snow Depth Prediction
2026-02-20 17:09:57Akila Sampath, Vandana Janeja, Jianwu WangAbstract
The accurate estimation of Arctic snow depth remains a critical time-varying inverse problem due to the extreme scarcity and noise inherent in associated sea ice parameters. Existing process-based and data-driven models are either highly sensitive to sparse data or lack the physical interpretability required for climate-critical applications. To address this gap, we introduce PhysE-Inv, a novel framework that integrates a sophisticated sequential architecture, an LSTM Encoder-Decoder with Multi-head Attention and physics-guided contrastive learning, with physics-guided this http URL core innovation lies in a surjective, physics-constrained inversion methodology. This methodology first leverages the hydrostatic balance forward model as a target-formulation proxy, enabling effective learning in the absence of direct $h_s$ ground truth; second, it uses reconstruction physics regularization over a latent space to dynamically discover hidden physical parameters from noisy, incomplete time-series input. Evaluated against state-of-the-art baselines, PhysE-Inv significantly improves prediction performance, reducing error by 20\% while demonstrating superior physical consistency and resilience to data sparsity compared to empirical methods. This approach pioneers a path for noise-tolerant, interpretable inverse modeling, with wide applicability in geospatial and cryospheric domains.
Abstract (translated)
北极雪深的精确估算仍然是一个关键的时间变化逆问题，由于与之相关的海冰参数极其稀少且存在噪声。现有的基于过程和数据驱动的模型要么对稀疏数据非常敏感，要么缺乏气候应用所需的物理可解释性。为了填补这一空白，我们引入了PhysE-Inv，这是一个新颖的框架，该框架集成了一个复杂的序列架构——LSTM编码器解码器与多头注意机制以及基于物理学引导的对比学习，并在此基础上进行了创新，核心在于一种单射、受物理约束的反演方法。此方法首先利用水静力平衡正向模型作为目标形式化代理，在没有直接 $h_s$ 地面真实值的情况下实现有效学习；其次，它在潜在空间中使用重建物理学正则化来动态发现隐藏的物理参数，从嘈杂、不完整的时序输入中进行挖掘。与最先进的基准相比，PhysE-Inv显著提高了预测性能，误差降低了20%，同时展示了优于经验方法的优异物理一致性和数据稀疏性的鲁棒性。这种方法为耐噪声、可解释逆向建模开创了一条新路径，在地理空间和冰冻圈领域具有广泛的应用前景。
URL
https://arxiv.org/abs/2601.17074
PDF
-
On the 'Induction Bias' in Sequence Models
2026-02-20 16:39:07M. Reza Ebrahimi, Micha\"el Defferrard, Sunny Panchal, Roland MemisevicAbstract
Despite the remarkable practical success of transformer-based language models, recent work has raised concerns about their ability to perform state tracking. In particular, a growing body of literature has shown this limitation primarily through failures in out-of-distribution (OOD) generalization, such as length extrapolation. In this work, we shift attention to the in-distribution implications of these limitations. We conduct a large-scale experimental study of the data efficiency of transformers and recurrent neural networks (RNNs) across multiple supervision regimes. We find that the amount of training data required by transformers grows much more rapidly with state-space size and sequence length than for RNNs. Furthermore, we analyze the extent to which learned state-tracking mechanisms are shared across different sequence lengths. We show that transformers exhibit negligible or even detrimental weight sharing across lengths, indicating that they learn length-specific solutions in isolation. In contrast, recurrent models exhibit effective amortized learning by sharing weights across lengths, allowing data from one sequence length to improve performance on others. Together, these results demonstrate that state tracking remains a fundamental challenge for transformers, even when training and evaluation distributions match.
Abstract (translated)
尽管基于变压器的语言模型在实践中取得了显著的成功，但最近的研究对它们执行状态跟踪的能力提出了担忧。特别是，越来越多的文献通过分布外（OOD）泛化中的失败来展示了这种限制，例如长度推断。在这项工作中，我们将注意力转向这些限制在分布内的影响。我们进行了大规模实验研究，比较了变压器和循环神经网络（RNNs）在不同监督模式下的数据效率。我们发现，随着状态空间大小和序列长度的增加，变压器所需的数据量增长得比RNN要快得多。 此外，我们还分析了学习到的状态跟踪机制在不同序列长度之间共享的程度。结果显示，变压器在不同长度之间的权重共享非常有限或甚至是有害的，这表明它们是孤立地学习特定于长度的解决方案。相比之下，循环模型通过跨长度共享权重来实现有效的样本外推学习，使得一个序列长度的数据可以改善其他长度序列的表现。 综合这些结果表明，即使训练和评估分布相匹配的情况下，状态跟踪依然是变压器面临的一个基本挑战。
URL
https://arxiv.org/abs/2602.18333
PDF
-
Financial time series augmentation using transformer based GAN architecture
2026-02-19 22:02:09Andrzej Podobi\'nski, Jaros{\l}aw A. ChudziakAbstract
Time-series forecasting is a critical task across many domains, from engineering to economics, where accurate predictions drive strategic decisions. However, applying advanced deep learning models in challenging, volatile domains like finance is difficult due to the inherent limitation and dynamic nature of financial time series data. This scarcity often results in sub-optimal model training and poor generalization. The fundamental challenge lies in determining how to reliably augment scarce financial time series data to enhance the predictive accuracy of deep learning forecasting models. Our main contribution is a demonstration of how Generative Adversarial Networks (GANs) can effectively serve as a data augmentation tool to overcome data scarcity in the financial domain. Specifically, we show that training a Long Short-Term Memory (LSTM) forecasting model on a dataset augmented with synthetic data generated by a transformer-based GAN (TTS-GAN) significantly improves the forecasting accuracy compared to using real data alone. We confirm these results across different financial time series (Bitcoin and S\&P500 price data) and various forecasting horizons. Furthermore, we propose a novel, time series specific quality metric that combines Dynamic Time Warping (DTW) and a modified Deep Dataset Dissimilarity Measure (DeD-iMs) to reliably monitor the training progress and evaluate the quality of the generated data. These findings provide compelling evidence for the benefits of GAN-based data augmentation in enhancing financial predictive capabilities.
Abstract (translated)
时间序列预测在工程、经济等多个领域中是一项关键任务，准确的预测能够推动战略决策。然而，在金融等挑战性和波动性较强的领域中应用高级深度学习模型较为困难，因为这些领域的时序数据具有固有的局限性和动态特性，导致训练数据稀缺，进而影响到模型训练效果和泛化能力。核心问题在于如何可靠地增强稀疏的金融时间序列数据以提高深度学习预测模型的准确性。 我们的主要贡献是展示了生成对抗网络（GANs）可以有效地作为数据扩充工具来克服金融领域中的数据稀缺性。具体而言，我们表明通过使用基于Transformer的GAN（TTS-GAN）生成的合成数据增强的数据集训练长短期记忆网络（LSTM）预测模型，与仅使用真实数据相比，能够显著提高预测准确性。我们在不同金融时间序列（比特币和标普500价格数据）以及各种预测时长上都确认了这些结果。 此外，我们还提出了一种新的、特定于时间序列的质量度量标准，该标准结合动态时间规整（DTW）和改进的深度数据集差异测量方法（DeD-iMs），以可靠地监测训练进度并评估生成的数据质量。我们的发现为基于GAN的数据扩充在提升金融预测能力方面的优势提供了强有力的证据。
URL
https://arxiv.org/abs/2602.17865
PDF
-
AdaptStress: Online Adaptive Learning for Interpretable and Personalized Stress Prediction Using Multivariate and Sparse Physiological Signals
2026-02-19 20:57:35Xueyi Wang, Claudine J. C. Lamoth, Elisabeth WilhelmAbstract
Continuous stress forecasting could potentially contribute to lifestyle interventions. This paper presents a novel, explainable, and individualized approach for stress prediction using physiological data from consumer-grade smartwatches. We develop a time series forecasting model that leverages multivariate features, including heart rate variability, activity patterns, and sleep metrics, to predict stress levels across 16 temporal horizons (History window: 3, 5, 7, 9 days; forecasting window: 1, 3, 5, 7 days). Our evaluation involves 16 participants monitored for 10-15 weeks. We evaluate our approach across 16 participants, comparing against state-of-the-art time series models (Informer, TimesNet, PatchTST) and traditional baselines (CNN, LSTM, CNN-LSTM) across multiple temporal horizons. Our model achieved performance with an MSE of 0.053, MAE of 0.190, and RMSE of 0.226 in optimal settings (5-day input, 1-day prediction). A comparison with the baseline models shows that our model outperforms TimesNet, PatchTST, CNN-LSTM, LSTM, and CNN under all conditions, representing improvements of 36.9%, 25.5%, and 21.5% over the best baseline. According to the explanability analysis, sleep metrics are the most dominant and consistent stress predictors (importance: 1.1, consistency: 0.9-1.0), while activity features exhibit high inter-participant variability (0.1-0.2). Most notably, the model captures individual-specific patterns where identical features can have opposing effects across users, validating its personalization capabilities. These findings establish that consumer wearables, combined with adaptive and interpretable deep learning, can deliver relevant stress assessment adapted to individual physiological responses, providing a foundation for scalable, continuous, explainable mental health monitoring in real-world settings.
Abstract (translated)
持续的压力预测有可能为生活方式的干预提供帮助。本文提出了一种新颖、可解释且个性化的压力预测方法，该方法利用来自消费级智能手表的生理数据来预测压力水平。我们开发了一个时间序列预测模型，该模型利用多变量特征（包括心率变异性、活动模式和睡眠指标），在16个不同的时间窗口内进行预测（历史窗口：3天、5天、7天、9天；预测窗口：1天、3天、5天、7天）。我们的评估涵盖了10到15周监控下的16名参与者。 我们从多个时间窗口角度对这种方法进行了跨16位参与者的评估，并将其与最先进的时间序列模型（包括Informer、TimesNet和PatchTST）以及传统的基础线模型（如CNN、LSTM、CNN-LSTM）进行了对比。在最优设置下（5天输入，1天预测），我们的模型取得了MSE为0.053、MAE为0.190和RMSE为0.226的性能指标。与基线模型相比，在所有条件下，我们的模型都优于TimesNet、PatchTST、CNN-LSTM、LSTM和CNN，分别提高了36.9%、25.5%和21.5%，表明了相对于最佳基础模型的进步。 根据可解释性分析，睡眠指标是最重要的且最一致的压力预测因素（重要度：1.1，一致性：0.9-1.0），而活动特征在不同参与者之间表现出较高的变异性（0.1-0.2）。值得注意的是，该模型能够捕捉到个体特有的模式，在这些模式中相同的特征可以对不同的用户产生相反的影响，从而验证了其个性化能力。 这些发现表明消费级可穿戴设备结合适应性和解释性的深度学习技术可以在现实世界环境中提供与个人生理反应相匹配的相关压力评估。这为大规模、连续且具有解释性的心理健康监测奠定了基础。
URL
https://arxiv.org/abs/2602.18521
PDF
-
Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
2026-02-19 18:48:08Xinghong Fu, Yanhong Li, Georgios Papaioannou, Yoon KimAbstract
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.
Abstract (translated)
URL
https://arxiv.org/abs/2602.17634
PDF
-
DeepContext: Stateful Real-Time Detection of Multi-Turn Adversarial Intent Drift in LLMs
2026-02-18 22:57:43Justin Albrethsen, Yash Datta, Kunal Kumar, Sharath RajasekarAbstract
While Large Language Model (LLM) capabilities have scaled, safety guardrails remain largely stateless, treating multi-turn dialogues as a series of disconnected events. This lack of temporal awareness facilitates a "Safety Gap" where adversarial tactics, like Crescendo and ActorAttack, slowly bleed malicious intent across turn boundaries to bypass stateless filters. We introduce DeepContext, a stateful monitoring framework designed to map the temporal trajectory of user intent. DeepContext discards the isolated evaluation model in favor of a Recurrent Neural Network (RNN) architecture that ingests a sequence of fine-tuned turn-level embeddings. By propagating a hidden state across the conversation, DeepContext captures the incremental accumulation of risk that stateless models overlook. Our evaluation demonstrates that DeepContext significantly outperforms existing baselines in multi-turn jailbreak detection, achieving a state-of-the-art F1 score of 0.84, which represents a substantial improvement over both hyperscaler cloud-provider guardrails and leading open-weight models such as Llama-Prompt-Guard-2 (0.67) and Granite-Guardian (0.67). Furthermore, DeepContext maintains a sub-20ms inference overhead on a T4 GPU, ensuring viability for real-time applications. These results suggest that modeling the sequential evolution of intent is a more effective and computationally efficient alternative to deploying massive, stateless models.
Abstract (translated)
URL
https://arxiv.org/abs/2602.16935
PDF
-
AIFL: A Global Daily Streamflow Forecasting Model Using Deterministic LSTM Pre-trained on ERA5-Land and Fine-tuned on IFS
2026-02-18 16:26:36Maria Luisa Taccari, Kenza Tazi, Ois\'in M. Morrison, Andreas Grafberger, Juan Colonese, Corentin Carton de Wiart, Christel Prudhomme, Cinzia Mazzetti, Matthew Chantry, Florian PappenbergerAbstract
Reliable global streamflow forecasting is essential for flood preparedness and water resource management, yet data-driven models often suffer from a performance gap when transitioning from historical reanalysis to operational forecast products. This paper introduces AIFL (Artificial Intelligence for Floods), a deterministic LSTM-based model designed for global daily streamflow forecasting. Trained on 18,588 basins curated from the CARAVAN dataset, AIFL utilises a novel two-stage training strategy to bridge the reanalysis-to-forecast domain shift. The model is first pre-trained on 40 years of ERA5-Land reanalysis (1980-2019) to capture robust hydrological processes, then fine-tuned on operational Integrated Forecasting System (IFS) control forecasts (2016-2019) to adapt to the specific error structures and biases of operational numerical weather prediction. To our knowledge, this is the first global model trained end-to-end within the CARAVAN ecosystem. On an independent temporal test set (2021-2024), AIFL achieves high predictive skill with a median modified Kling-Gupta Efficiency (KGE') of 0.66 and a median Nash-Sutcliffe Efficiency (NSE) of 0.53. Benchmarking results show that AIFL is highly competitive with current state-of-the-art global systems, achieving comparable accuracy while maintaining a transparent and reproducible forcing pipeline. The model demonstrates exceptional reliability in extreme-event detection, providing a streamlined and operationally robust baseline for the global hydrological community.
Abstract (translated)
URL
https://arxiv.org/abs/2602.16579
PDF
-
UCTECG-Net: Uncertainty-aware Convolution Transformer ECG Network for Arrhythmia Detection
2026-02-18 06:39:19Hamzeh Asgharnezhad, Pegah Tabarisaadi, Abbas Khosravi, Roohallah Alizadehsani, U. Rajendra AcharyaAbstract
Deep learning has improved automated electrocardiogram (ECG) classification, but limited insight into prediction reliability hinders its use in safety-critical settings. This paper proposes UCTECG-Net, an uncertainty-aware hybrid architecture that combines one-dimensional convolutions and Transformer encoders to process raw ECG signals and their spectrograms jointly. Evaluated on the MIT-BIH Arrhythmia and PTB Diagnostic datasets, UCTECG-Net outperforms LSTM, CNN1D, and Transformer baselines in terms of accuracy, precision, recall and F1 score, achieving up to 98.58% accuracy on MIT-BIH and 99.14% on PTB. To assess predictive reliability, we integrate three uncertainty quantification methods (Monte Carlo Dropout, Deep Ensembles, and Ensemble Monte Carlo Dropout) into all models and analyze their behavior using an uncertainty-aware confusion matrix and derived metrics. The results show that UCTECG-Net, particularly with Ensemble or EMCD, provides more reliable and better-aligned uncertainty estimates than competing architectures, offering a stronger basis for risk-aware ECG decision support.
Abstract (translated)
URL
https://arxiv.org/abs/2602.16216
PDF
-
Under-resourced studies of under-resourced languages: lemmatization and POS-tagging with LLM annotators for historical Armenian, Georgian, Greek and Syriac
2026-02-17 17:34:32Chahan Vidal-Gor\`ene (CJM, LIPN), Bastien Kindt (UCL), Florian Cafiero (PSL, CJM)Abstract
Low-resource languages pose persistent challenges for Natural Language Processing tasks such as lemmatization and part-of-speech (POS) tagging. This paper investigates the capacity of recent large language models (LLMs), including GPT-4 variants and open-weight Mistral models, to address these tasks in few-shot and zero-shot settings for four historically and linguistically diverse under-resourced languages: Ancient Greek, Classical Armenian, Old Georgian, and Syriac. Using a novel benchmark comprising aligned training and out-of-domain test corpora, we evaluate the performance of foundation models across lemmatization and POS-tagging, and compare them with PIE, a task-specific RNN baseline. Our results demonstrate that LLMs, even without fine-tuning, achieve competitive or superior performance in POS-tagging and lemmatization across most languages in few-shot settings. Significant challenges persist for languages characterized by complex morphology and non-Latin scripts, but we demonstrate that LLMs are a credible and relevant option for initiating linguistic annotation tasks in the absence of data, serving as an effective aid for annotation.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15753
PDF
-
Relative Geometry of Neural Forecasters: Linking Accuracy and Alignment in Learned Latent Geometry
2026-02-17 16:00:08Deniz Kucukahmetler, Maximilian Jean Hemmann, Julian Mosig von Aehrenfeld, Maximilian Amthor, Christian Deubel, Nico Scherf, Diaaeldin TahaAbstract
Neural networks can accurately forecast complex dynamical systems, yet how they internally represent underlying latent geometry remains poorly understood. We study neural forecasters through the lens of representational alignment, introducing anchor-based, geometry-agnostic relative embeddings that remove rotational and scaling ambiguities in latent spaces. Applying this framework across seven canonical dynamical systems - ranging from periodic to chaotic - we reveal reproducible family-level structure: multilayer perceptrons align with other MLPs, recurrent networks with RNNs, while transformers and echo-state networks achieve strong forecasts despite weaker alignment. Alignment generally correlates with forecasting accuracy, yet high accuracy can coexist with low alignment. Relative geometry thus provides a simple, reproducible foundation for comparing how model families internalize and represent dynamical structure.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15676
PDF
-
BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training
2026-02-17 13:52:05Luca Colombo, Fabrizio Pittorino, Daniele Zambon, Carlo Baldassi, Manuel Roveri, Cesare AlippiAbstract
Binary Neural Networks (BNNs), which constrain both weights and activations to binary values, offer substantial reductions in computational complexity, memory footprint, and energy consumption. These advantages make them particularly well suited for deployment on resource-constrained devices. However, training BNNs via gradient-based optimization remains challenging due to the discrete nature of their variables. The dominant approach, quantization-aware training, circumvents this issue by employing surrogate gradients. Yet, this method requires maintaining latent full-precision parameters and performing the backward pass with floating-point arithmetic, thereby forfeiting the efficiency of binary operations during training. While alternative approaches based on local learning rules exist, they are unsuitable for global credit assignment and for back-propagating errors in multi-layer architectures. This paper introduces Binary Error Propagation (BEP), the first learning algorithm to establish a principled, discrete analog of the backpropagation chain rule. This mechanism enables error signals, represented as binary vectors, to be propagated backward through multiple layers of a neural network. BEP operates entirely on binary variables, with all forward and backward computations performed using only bitwise operations. Crucially, this makes BEP the first solution to enable end-to-end binary training for recurrent neural network architectures. We validate the effectiveness of BEP on both multi-layer perceptrons and recurrent neural networks, demonstrating gains of up to +6.89% and +10.57% in test accuracy, respectively. The proposed algorithm is released as an open-source repository.
Abstract (translated)
URL
https://arxiv.org/abs/2512.04189
PDF
