Text_Classification
-
Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM
2026-03-25 14:56:34Mohammad AL-SmadiAbstract
Clinical trials require strict adherence to medication protocols, yet dosing errors remain a persistent challenge affecting patient safety and trial integrity. We present an automated system for detecting dosing errors in unstructured clinical trial narratives using gradient boosting with comprehensive multi-modal feature engineering. Our approach combines 3,451 features spanning traditional NLP (TF-IDF, character n-grams), dense semantic embeddings (all-MiniLM-L6v2), domain-specific medical patterns, and transformer-based scores (BiomedBERT, DeBERTa-v3), used to train a LightGBM model. Features are extracted from nine complementary text fields (median 5,400 characters per sample) ensuring complete coverage across all 42,112 clinical trial narratives. On the CT-DEB benchmark dataset with severe class imbalance (4.9% positive rate), we achieve 0.8725 test ROC-AUC through 5-fold ensemble averaging (cross-validation: 0.8833 + 0.0091 AUC). Systematic ablation studies reveal that removing sentence embeddings causes the largest performance degradation (2.39%), demonstrating their critical role despite contributing only 37.07% of total feature importance. Feature efficiency analysis demonstrates that selecting the top 500-1000 features yields optimal performance (0.886-0.887 AUC), outperforming the full 3,451-feature set (0.879 AUC) through effective noise reduction. Our findings highlight the importance of feature selection as a regularization technique and demonstrate that sparse lexical features remain complementary to dense representations for specialized clinical text classification under severe class imbalance.
Abstract (translated)
URL
https://arxiv.org/abs/2604.19759
PDF
-
LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation
2026-03-23 23:07:16Hailay Teklehaymanot, Dren Fazlija, Wolfgang NejdlAbstract
Adapting pretrained language models to low-resource, morphologically rich languages remains a significant challenge. Existing vocabulary expansion methods typically rely on arbitrarily segmented subword units, resulting in fragmented lexical representations and loss of critical morphological information. To address this limitation, we propose the Lexically Grounded Subword Embedding Initialization (LGSE) framework, which introduces morphologically informed segmentation for initializing embeddings of novel tokens. Instead of using random vectors or arbitrary subwords, LGSE decomposes words into their constituent morphemes and constructs semantically coherent embeddings by averaging pretrained subword or FastText-based morpheme representations. When a token cannot be segmented into meaningful morphemes, its embedding is constructed using character n-gram representations to capture structural information. During Language-Adaptive Pretraining, we apply a regularization term that penalizes large deviations of newly introduced embeddings from their initialized values, preserving alignment with the original pretrained embedding space while enabling adaptation to the target language. To isolate the effect of initialization, we retain the original pre-trained model vocabulary and tokenizer and update only the new embeddings during adaptation. We evaluate LGSE on three NLP tasks: Question Answering, Named Entity Recognition, and Text Classification, in two morphologically rich, low-resource languages: Amharic and Tigrinya, where morphological segmentation resources are available. Experimental results show that LGSE consistently outperforms baseline methods across all tasks, demonstrating the effectiveness of morphologically grounded embedding initialization for improving representation quality in underrepresented languages. Project resources are available in the GitHub link.
Abstract (translated)
将预训练语言模型适配到低资源、形态丰富的语言仍是一项重大挑战。现有词汇扩展方法通常依赖任意分割的子词单元，导致词汇表示碎片化并丢失关键的形态学信息。为应对这一局限，我们提出了词汇锚定子词嵌入初始化（LGSE）框架，该框架引入基于形态学的分割来初始化新词元的嵌入。LGSE不使用随机向量或任意子词，而是将单词分解为构成语素，并通过平均预训练的子词或基于FastText的语素表示来构建语义连贯的嵌入。当词元无法分割为有意义的语素时，则使用字符n-gram表示来捕获结构信息以构建其嵌入。在语言自适应预训练过程中，我们应用一个正则化项来惩罚新引入嵌入与初始化值之间的较大偏差，从而在保持与原始预训练嵌入空间对齐的同时实现目标语言适配。为隔离初始化效果，我们保留原始预训练模型的词汇表和分词器，并在适配过程中仅更新新嵌入。我们在两种形态丰富且资源稀缺的语言（阿姆哈拉语和提格里尼亚语，其形态分割资源可用）上，于三个NLP任务（问答、命名实体识别和文本分类）中评估了LGSE。实验结果表明，LGSE在所有任务上均持续优于基线方法，证明了基于形态学的嵌入初始化对提升低资源语言表示质量的有效性。项目资源可在GitHub链接获取。
URL
https://arxiv.org/abs/2603.22629
PDF
-
Exploring Data Augmentation and Resampling Strategies for Transformer-Based Models to Address Class Imbalance in AI Scoring of Scientific Explanations in NGSS Classroom
2026-03-21 07:12:10Prudence Djagba, Kevin Haudek, Clare G. C. Franovic, Leonora KaldarasAbstract
Automated scoring of students' scientific explanations offers the potential for immediate, accurate feedback, yet class imbalance in rubric categories particularly those capturing advanced reasoning remains a challenge. This study investigates augmentation strategies to improve transformer-based text classification of student responses to a physical science assessment based on an NGSS-aligned learning progression. The dataset consists of 1,466 high school responses scored on 11 binary-coded analytic categories. This rubric identifies six important components including scientific ideas needed for a complete explanation along with five common incomplete or inaccurate ideas. Using SciBERT as a baseline, we applied fine-tuning and test these augmentation strategies: (1) GPT-4--generated synthetic responses, (2) EASE, a word-level extraction and filtering approach, and (3) ALP (Augmentation using Lexicalized Probabilistic context-free grammar) phrase-level extraction. While fine-tuning SciBERT improved recall over baseline, augmentation substantially enhanced performance, with GPT data boosting both precision and recall, and ALP achieving perfect precision, recall, and F1 scores across most severe imbalanced categories (5,6,7 and 9). Across all rubric categories EASE augmentation substantially increased alignment with human scoring for both scientific ideas (Categories 1--6) and inaccurate ideas (Categories 7--11). We compared different augmentation strategies to a traditional oversampling method (SMOTE) in an effort to avoid overfitting and retain novice-level data critical for learning progression alignment. Findings demonstrate that targeted augmentation can address severe imbalance while preserving conceptual coverage, offering a scalable solution for automated learning progression-aligned scoring in science education.
Abstract (translated)
URL
https://arxiv.org/abs/2604.19754
PDF
-
Enhancing Hyperspace Analogue to Language Representations via Attention-Based Pooling for Text Classification
2026-03-20 17:25:09Ali Sakour, Zoalfekar SakourAbstract
The Hyperspace Analogue to Language (HAL) model relies on global word co-occurrence matrices to construct distributional semantic representations. While these representations capture lexical relationships effectively, aggregating them into sentence-level embeddings via standard mean pooling often results in information loss. Mean pooling assigns equal weight to all tokens, thereby diluting the impact of contextually salient words with uninformative structural tokens. In this paper, we address this limitation by integrating a learnable, temperature-scaled additive attention mechanism into the HAL representation pipeline. To mitigate the sparsity and high dimensionality of the raw co-occurrence matrices, we apply Truncated Singular Value Decomposition (SVD) to project the vectors into a dense latent space prior to the attention layer. We evaluate the proposed architecture on the IMDB sentiment analysis dataset. Empirical results demonstrate that the attention-based pooling approach achieves a test accuracy of 82.38%, yielding an absolute improvement of 6.74 percentage points over the traditional mean pooling baseline (75.64%). Furthermore, qualitative analysis of the attention weights indicates that the mechanism successfully suppresses stop-words and selectively attends to sentiment-bearing tokens, improving both classification performance and model interpretability.
Abstract (translated)
超空间语言类比（HAL）模型依赖全局词共现矩阵构建分布语义表示。尽管这些表示能有效捕捉词汇关系，但通过标准均值池化聚合为句子级嵌入时往往导致信息损失。均值池化对所有词元赋予相同权重，从而稀释了语境显著词汇的影响，同时放大了无信息量的结构词元的作用。本文针对这一局限，在HAL表示流程中集成了可学习的温度缩放可加注意力机制。为缓解原始共现矩阵的稀疏性与高维问题，我们在注意力层前采用截断奇异值分解（SVD）将向量投影至稠密潜在空间。所提架构在IMDB情感分析数据集上进行了评估。实证结果表明，基于注意力池化的方法达到82.38%的测试准确率，相较于传统均值池化基线（75.64%）实现了6.74个百分点的绝对提升。此外，对注意力权重的定性分析显示，该机制成功抑制了停用词，并选择性地聚焦于携带情感信息的词元，从而提升了分类性能与模型可解释性。
URL
https://arxiv.org/abs/2603.20149
PDF
-
TextReasoningBench: Does Reasoning Really Improve Text Classification in Large Language Models?
2026-03-20 01:53:12Xinyu Guo, Yazhou Zhang, Jing QinAbstract
Eliciting explicit, step-by-step reasoning traces from large language models (LLMs) has emerged as a dominant paradigm for enhancing model capabilities. Although such reasoning strategies were originally designed for problems requiring explicit multi-step reasoning, they have increasingly been applied to a broad range of NLP tasks. This expansion implicitly assumes that deliberative reasoning uniformly benefits heterogeneous tasks. However, whether such reasoning mechanisms truly benefit classification tasks remains largely underexplored, especially considering their substantial token and time costs. To fill this gap, we introduce TextReasoningBench, a systematic benchmark designed to evaluate the effectiveness and efficiency of reasoning strategies for text classification with LLMs. We compare seven reasoning strategies, namely IO, CoT, SC-CoT, ToT, GoT, BoC, and long-CoT across ten LLMs on five text classification datasets. Beyond traditional metrics such as accuracy and macro-F1, we introduce two cost-aware evaluation metrics that quantify the performance gain per reasoning token and the efficiency of performance improvement relative to token cost growth. Experimental results reveal three notable findings: (1) Reasoning does not universally improve classification performance: while moderate strategies such as CoT and SC-CoT yield consistent but limited gains (typically +1% to +3% on big models), more complex methods (e.g., ToT and GoT) often fail to outperform simpler baselines and can even degrade performance, especially on small models; (2) Reasoning is often inefficient: many reasoning strategies increase token consumption by 10$\times$ to 100$\times$ (e.g., SC-CoT and ToT) while providing only marginal performance improvements.
Abstract (translated)
从大语言模型（LLMs）中提取显式的逐步推理痕迹，已成为提升模型能力的主流范式。尽管此类推理策略最初专为需要显式多步推理的问题设计，但现已日益广泛应用于各类自然语言处理任务。这种扩展隐含地假设审慎推理能普遍惠及异构任务。然而，此类推理机制是否真正有益于分类任务，尤其是考虑到其高昂的令牌与时间成本，仍 largely 未被充分探索。为填补这一空白，我们推出 TextReasoningBench——一个系统性基准测试，旨在评估 LLM 文本分类中推理策略的有效性与效率。我们在五个文本分类数据集上，对比了七种推理策略（IO、CoT、SC-CoT、ToT、GoT、BoC 及 long-CoT）在十种 LLM 上的表现。除准确率与宏 F1 等传统指标外，我们引入两项成本感知评估指标：量化每推理令牌的性能增益，以及性能提升效率相对于令牌成本增长的比率。实验结果显示三项关键发现：（1）推理并非普遍提升分类性能：适度策略（如 CoT 与 SC-CoT）在大模型上仅带来稳定但有限的增益（通常为 +1% 至 +3%），而复杂方法（如 ToT 与 GoT）常无法超越简单基线，甚至可能损害性能，在小模型上尤为明显；（2）推理往往效率低下：多数策略使令牌消耗增加 10 倍至 100 倍（如 SC-CoT 与 ToT），却仅换取微小的性能提升。
URL
https://arxiv.org/abs/2603.19558
PDF
-
AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba
2026-03-19 03:47:21Yan Li, Yifei Xing, Xiangyuan Lan, Xin Li, Haifeng Chen, Dongmei JiangAbstract
In the era of large-scale pre-trained models, effectively adapting general knowledge to specific affective computing tasks remains a challenge, particularly regarding computational efficiency and multimodal heterogeneity. While Transformer-based methods have excelled at modeling inter-modal dependencies, their quadratic computational complexity limits their use with long-sequence data. Mamba-based models have emerged as a computationally efficient alternative; however, their inherent sequential scanning mechanism struggles to capture the global, non-sequential relationships that are crucial for effective cross-modal alignment. To address these limitations, we propose \textbf{AlignMamba-2}, an effective and efficient framework for multimodal fusion and sentiment analysis. Our approach introduces a dual alignment strategy that regularizes the model using both Optimal Transport distance and Maximum Mean Discrepancy, promoting geometric and statistical consistency between modalities without incurring any inference-time overhead. More importantly, we design a Modality-Aware Mamba layer, which employs a Mixture-of-Experts architecture with modality-specific and modality-shared experts to explicitly handle data heterogeneity during the fusion process. Extensive experiments on four challenging benchmarks, including dynamic time-series (on the CMU-MOSI and CMU-MOSEI datasets) and static image-related tasks (on the NYU-Depth V2 and MVSA-Single datasets), demonstrate that AlignMamba-2 establishes a new state-of-the-art in both effectiveness and efficiency across diverse pattern recognition tasks, ranging from dynamic time-series analysis to static image-text classification.
Abstract (translated)
URL
https://arxiv.org/abs/2603.18462
PDF
-
Uncertainty Estimation for the Open-Set Text Classification systems
2026-03-17 12:25:34Leonid Erlygin, Alexey ZaytsevAbstract
Accurate uncertainty estimation is essential for building robust and trustworthy recognition systems. In this paper, we consider the open-set text classification (OSTC) task - and uncertainty estimation for it. For OSTC a text sample should be classified as one of the existing classes or rejected as unknown. To account for the different uncertainty types encountered in OSTC, we adapt the Holistic Uncertainty Estimation (HolUE) method for the text domain. Our approach addresses two major causes of prediction errors in text recognition systems: text uncertainty that stems from ill formulated queries and gallery uncertainty that is related the ambiguity of data distribution. By capturing these sources, it becomes possible to predict when the system will make a recognition error. We propose a new OSTC benchmark and conduct extensive experiments on a wide range of data, utilizing the authorship attribution, intent and topic classification datasets. HolUE achieves 40-365% improvement in Prediction Rejection Ratio (PRR) over the quality-based SCF baseline across datasets: 365% on Yahoo Answers (0.79 vs 0.17 at FPIR 0.1), 347% on DBPedia (0.85 vs 0.19), 240% on PAN authorship attribution (0.51 vs 0.15 at FPIR 0.5), and 40% on CLINC150 intent classification (0.73 vs~0.52). We make public our code and protocols this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2604.08560
PDF
-
Mastering the Minority: An Uncertainty-guided Multi-Expert Framework for Challenging-tailed Sequence Learning
2026-03-16 13:34:56Ye Wang, Zixuan Wu, Lifeng Shen, Jiang Xie, Xiaoling Wang, Hong Yu, Guoyin WangAbstract
Imbalanced data distribution remains a critical challenge in sequential learning, leading models to easily recognize frequent categories while failing to detect minority classes adequately. The Mixture-of-Experts model offers a scalable solution, yet its application is often hindered by parameter inefficiency, poor expert specialization, and difficulty in resolving prediction conflicts. To Master the Minority classes effectively, we propose the Uncertainty-based Multi-Expert fusion network (UME) framework. UME is designed with three core innovations: First, we employ Ensemble LoRA for parameter-efficient modeling, significantly reducing the trainable parameter count. Second, we introduce Sequential Specialization guided by Dempster-Shafer Theory (DST), which ensures effective specialization on the challenging-tailed classes. Finally, an Uncertainty-Guided Fusion mechanism uses DST's certainty measures to dynamically weigh expert opinions, resolving conflicts by prioritizing the most confident expert for reliable final predictions. Extensive experiments across four public hierarchical text classification datasets demonstrate that UME achieves state-of-the-art performance. We achieve a performance gain of up to 17.97\% over the best baseline on individual categories, while reducing trainable parameters by up to 10.32\%. The findings highlight that uncertainty-guided expert coordination is a principled strategy for addressing challenging-tailed sequence learning. Our code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2603.15708
PDF
-
Towards Privacy-Preserving Machine Translation at the Inference Stage: A New Task and Benchmark
2026-03-16 02:41:06Wei Shao, Lemao Liu, Yinqiao Li, Guoping Huang, Shuming Shi, Linqi SongAbstract
Current online translation services require sending user text to cloud servers, posing a risk of privacy leakage when the text contains sensitive information. This risk hinders the application of online translation services in privacy-sensitive scenarios. One way to mitigate this risk for online translation services is introducing privacy protection mechanisms targeting the inference stage of translation models. However, compared to subfields of NLP like text classification and summarization, the machine translation research community has limited exploration of privacy protection during the inference stage. There is no clearly defined privacy protection task for the inference stage, dedicated evaluation datasets and metrics, and reference benchmark methods. The absence of these elements has seriously constrained researchers' in-depth exploration of this direction. To bridge this gap, this paper proposes a novel "Privacy-Preserving Machine Translation" (PPMT) task, aiming to protect the private information in text during the model inference stage. For this task, we constructed three benchmark test datasets, designed corresponding evaluation metrics, and proposed a series of benchmark methods as a starting point for this task. The definition of privacy is complex and diverse. Considering that named entities often contain a large amount of personal privacy and commercial secrets, we have focused our research on protecting only the named entity's privacy in the text. We expect this research work will provide a new perspective and a solid foundation for the privacy protection problem in machine translation.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14756
PDF
-
Selective Fine-Tuning of GPT Architectures for Parameter-Efficient Clinical Text Classification
2026-03-15 02:16:06Fariba Afrin Irany, Sampson AkwafuoAbstract
The rapid expansion of electronic health record (EHR) systems has generated large volumes of unstructured clinical narratives that contain valuable information for disease identification, patient cohort discovery, and clinical decision support. Extracting structured knowledge from these free-text documents remains challenging because clinical language is highly specialized, labeled datasets are limited, and full fine-tuning of large pretrained language models can require substantial computational resources. Efficient adaptation strategies are therefore essential for practical clinical natural language processing applications. This study proposes a parameter-efficient selective fine-tuning framework for adapting GPT-2 to clinical text classification tasks. Instead of updating the entire pretrained model, the majority of network parameters are frozen, and only the final Transformer block, the final layer normalization module, and a lightweight classification head are updated during training. This design substantially reduces the number of trainable parameters while preserving the contextual representation capabilities learned during pretraining. The proposed approach is evaluated using radiology reports from the MIMIC-IV-Note dataset with automatically derived CheXpert-style labels. Experiments on 50,000 radiology reports demonstrate that selective fine-tuning achieves approximately 91% classification accuracy while updating fewer than 6% of the model parameters. Comparative experiments with head-only training and full-model fine-tuning show that the proposed method provides a favorable balance between predictive performance and computational efficiency. These results indicate that selective fine-tuning offers an efficient and scalable framework for clinical text classification.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14183
PDF
-
CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classification
2026-03-14 18:47:44Menna Elgabry, Ali Hamdi, Khaled ShabanAbstract
Textual Emotion Classification (TEC) is one of the most difficult NLP tasks. State of the art approaches rely on Large language models (LLMs) and multi-model ensembles. In this study, we challenge the assumption that larger scale or more complex models are necessary for improved performance. In order to improve logical consistency, We introduce CMHL, a novel single-model architecture that explicitly models the logical structure of emotions through three key innovations: (1) multi-task learning that jointly predicts primary emotions, valence, and intensity, (2) psychologically-grounded auxiliary supervision derived from Russell's circumplex model, and (3) a novel contrastive contradiction loss that enforces emotional consistency by penalizing mutually incompatible predictions (e.g., simultaneous high confidence in joy and anger). With just 125M parameters, our model outperforms 56x larger LLMs and sLM ensembles with a new state-of-the-art F1 score of 93.75\% compared to (86.13\%-93.2\%) on the dair-ai Emotion dataset. We further show cross domain generalization on the Reddit Suicide Watch and Mental Health Collection dataset (SWMH), outperforming domain-specific models like MentalBERT and MentalRoBERTa with an F1 score of 72.50\% compared to (68.16\%-72.16\%) + a 73.30\% recall compared to (67.05\%-70.89\%) that translates to enhanced sensitivity for detecting mental health distress. Our work establishes that architectural intelligence (not parameter count) drives progress in TEC. By embedding psychological priors and explicit consistency constraints, a well-designed single model can outperform both massive LLMs and complex ensembles, offering a efficient, interpretable, and clinically-relevant paradigm for affective computing.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14078
PDF
-
98$times$ Faster LLM Routing Without a Dedicated GPU: Flash Attention, Prompt Compression, and Near-Streaming for the vLLM Semantic Router
2026-03-13 04:33:53Xunzhuo Liu, Bowei He, Xue Liu, Andy Luo, Haichen Zhang, Huamin ChenAbstract
System-level routers that intercept LLM requests for safety classification, domain routing, and PII detection must be both fast and operationally lightweight: they should add minimal latency to every request, yet not require a dedicated GPU -- an expensive resource better used for LLM inference itself. When the router co-locates on the same GPU as vLLM serving instances, standard attention's $O(n^2)$ memory makes long-context classification (8K--32K tokens) impossible: at 8K tokens, three concurrent classifiers need ${\sim}$4.5\,GB for attention masks alone, far exceeding the memory left by vLLM. We present three staged optimizations for the vLLM Semantic Router, benchmarked on AMD Instinct MI300X, that solve both the latency and the memory problem. \emph{Stage~1}: a custom CK Flash Attention operator for ONNX Runtime on ROCm reduces attention memory from $O(n^2)$ to $O(n)$ and end-to-end (E2E) latency from 4{,}918\,ms to 127\,ms (\textbf{38.7$\times$}), enabling 8K--32K tokens where SDPA OOMs. \emph{Stage~2}: classical NLP prompt compression (TextRank, position weighting, TF-IDF, and novelty scoring) reduces all inputs to ${\sim}$512 tokens without neural inference, capping both latency and GPU memory at a constant regardless of original prompt length (E2E 127$\to$62\,ms, \textbf{2.0$\times$}). \emph{Stage~3}: near-streaming body processing with adaptive chunking and zero-copy JSON eliminates serialization overhead (E2E 62$\to$50\,ms, \textbf{1.2$\times$}). Cumulatively: \textbf{98$\times$} improvement (4{,}918\,ms to 50\,ms), 16K-token routing in 108\,ms, and a total router GPU footprint under 800\,MB -- small enough to share a GPU with LLM serving and removing the need for a dedicated accelerator. Stage~1 targets AMD ROCm (NVIDIA GPUs already have FlashAttention via cuDNN); Stages~2 and~3 are hardware-agnostic.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12646
PDF
-
BTZSC: A Benchmark for Zero-Shot Text Classification Across Cross-Encoders, Embedding Models, Rerankers and LLMs
2026-03-12 14:43:20Ilias AarabAbstract
Zero-shot text classification (ZSC) offers the promise of eliminating costly task-specific annotation by matching texts directly to human-readable label descriptions. While early approaches have predominantly relied on cross-encoder models fine-tuned for natural language inference (NLI), recent advances in text-embedding models, rerankers, and instruction-tuned large language models (LLMs) have challenged the dominance of NLI-based architectures. Yet, systematically comparing these diverse approaches remains difficult. Existing evaluations, such as MTEB, often incorporate labeled examples through supervised probes or fine-tuning, leaving genuine zero-shot capabilities underexplored. To address this, we introduce BTZSC, a comprehensive benchmark of 22 public datasets spanning sentiment, topic, intent, and emotion classification, capturing diverse domains, class cardinalities, and document lengths. Leveraging BTZSC, we conduct a systematic comparison across four major model families, NLI cross-encoders, embedding models, rerankers and instruction-tuned LLMs, encompassing 38 public and custom checkpoints. Our results show that: (i) modern rerankers, exemplified by Qwen3-Reranker-8B, set a new state-of-the-art with macro F1 = 0.72; (ii) strong embedding models such as GTE-large-en-v1.5 substantially close the accuracy gap while offering the best trade-off between accuracy and latency; (iii) instruction-tuned LLMs at 4--12B parameters achieve competitive performance (macro F1 up to 0.67), excelling particularly on topic classification but trailing specialized rerankers; (iv) NLI cross-encoders plateau even as backbone size increases; and (v) scaling primarily benefits rerankers and LLMs over embedding models. BTZSC and accompanying evaluation code are publicly released to support fair and reproducible progress in zero-shot text understanding.
Abstract (translated)
零样本文本分类（ZSC）提供了一种希望，即通过将文本直接匹配到可读的标签描述来消除耗资的任务特定标注。虽然早期的方法主要依赖于为自然语言推理（NLI）微调的交叉编码器模型，但最近在文本嵌入模型、重排序器和指令调整的大规模语言模型（LLM）方面的进步挑战了基于NLI架构的主导地位。然而，系统地比较这些多样化的方法仍然困难重重。现有的评估方法，如MTEB，通常通过监督探针或微调来整合带标签的例子，这使得真正意义上的零样本能力被忽视了。为了解决这一问题，我们介绍了BTZSC，这是一个全面的基准测试集合，包括22个公共数据集，涵盖了情绪、主题、意图和情感分类，覆盖了多样化的领域、类别数量以及文档长度。利用BTZSC，我们在四种主要模型家族之间进行了系统比较：NLI交叉编码器、嵌入模型、重排序器以及指令调整的LLM，包括38个公共和自定义检查点。 我们的结果表明： (i) 现代重排序器，如Qwen3-Reranker-8B，在宏观F1分数上达到了0.72的新最高水平； (ii) 强大的嵌入模型，例如GTE-large-en-v1.5，显著缩小了准确性的差距，并且在精度和延迟之间提供了最佳的权衡； (iii) 参数量为4至12B的指令调整LLM取得了竞争性性能（宏观F1分数最高达0.67），尤其擅长主题分类，但不及专门化的重排序器的表现； (iv) NLI交叉编码器即使随着骨干模型规模的增长也表现停滞； (v) 规模化主要有利于重排序器和LLM超过嵌入模型。 BTZSC及其伴随的评估代码已公开发布，以支持零样本文本理解中的公正且可复现的进步。
URL
https://arxiv.org/abs/2603.11991
PDF
-
An Automatic Text Classification Method Based on Hierarchical Taxonomies, Neural Networks and Document Embedding: The NETHIC Tool
2026-03-12 10:24:14Luigi Lomasto, Rosario Di Florio, Andrea Ciapetti, Giuseppe Miscione, Giulia Ruggiero, Daniele TotiAbstract
This work describes an automatic text classification method implemented in a software tool called NETHIC, which takes advantage of the inner capabilities of highly-scalable neural networks combined with the expressiveness of hierarchical taxonomies. As such, NETHIC succeeds in bringing about a mechanism for text classification that proves to be significantly effective as well as efficient. The tool had undergone an experimentation process against both a generic and a domain-specific corpus, outputting promising results. On the basis of this experimentation, NETHIC has been now further refined and extended by adding a document embedding mechanism, which has shown improvements in terms of performance on the individual networks and on the whole hierarchical model.
Abstract (translated)
这项工作描述了一种自动文本分类方法，该方法被实现在名为NETHIC的软件工具中。NETHIC利用了高度可扩展神经网络的内在能力，并结合了分层分类法的表现力，从而成功地创建了一个既有效又高效的文本分类机制。该工具针对通用语料库和特定领域语料库进行了实验测试，取得了令人鼓舞的结果。基于这些实验结果，NETHIC现已进一步改进并扩充，添加了一种文档嵌入机制，在各个单独网络以及整个分层模型上都显示出性能的提升。
URL
https://arxiv.org/abs/2603.11770
PDF
-
Temporal Text Classification with Large Language Models
2026-03-11 20:49:20Nishat Raihan, Marcos ZampieriAbstract
Languages change over time. Computational models can be trained to recognize such changes enabling them to estimate the publication date of texts. Despite recent advancements in Large Language Models (LLMs), their performance on automatic dating of texts, also known as Temporal Text Classification (TTC), has not been explored. This study provides the first systematic evaluation of leading proprietary (Claude 3.5, GPT-4o, Gemini 1.5) and open-source (LLaMA 3.2, Gemma 2, Mistral, Nemotron 4) LLMs on TTC using three historical corpora, two in English and one in Portuguese. We test zero-shot and few-shot prompting, and fine-tuning settings. Our results indicate that proprietary models perform well, especially with few-shot prompting. They also indicate that fine-tuning substantially improves open-source models but that they still fail to match the performance delivered by proprietary LLMs.
Abstract (translated)
语言随着时间的推移而发生变化。计算模型可以被训练来识别这些变化，从而能够估计文本的出版日期。尽管近年来大型语言模型（LLM）取得了显著进展，但它们在自动确定文本年代上的表现——即时间文本分类（TTC）方面尚未得到充分研究。本研究首次系统地评估了领先的专有模型（Claude 3.5、GPT-4o、Gemini 1.5）和开源模型（LLaMA 3.2、Gemma 2、Mistral、Nemotron 4）在TTC上的表现，使用三个历史语料库进行评估，其中包括两个英语语料库和一个葡萄牙语文料库。我们测试了零样本和少量样本提示以及微调设置的表现。我们的结果显示，专有模型表现良好，特别是在少量样本提示下。此外，结果还表明开源模型在经过微调后性能显著提升，但仍然无法达到专有LLM的水平。
URL
https://arxiv.org/abs/2603.11295
PDF
-
An Extreme Multi-label Text Classification Library Dataset: What if we took 'Use of Practical AI in Digital Libraries' seriously?
2026-03-11 15:24:20Jennifer D'Souza, Sameer Sadruddin, Maximilian K\"ahler, Andrea Salfinger, Luca Zaccagna, Francesca Incitti, Lauro Snidaro, Osma SuominenAbstract
Subject indexing is vital for discovery but hard to sustain at scale and across languages. We release a large bilingual (English/German) corpus of catalog records annotated with the Integrated Authority File (GND), plus a machine-actionable GND taxonomy. The resource enables ontology-aware multi-label classification, mapping text to authority terms, and agent-assisted cataloging with reproducible, authority-grounded evaluation. We provide a brief statistical profile and qualitative error analyses of three systems. We invite the community to assess not only accuracy but usefulness and transparency, toward authority-anchored AI co-pilots that amplify catalogers' work.
Abstract (translated)
主题索引对于发现至关重要，但在大规模和跨语言环境中维护却颇具挑战。我们发布了一个大型的双语（英语/德语）目录记录语料库，并且这些记录使用综合权威文件（GND）进行了注释，同时还提供了一个机器可操作的GND分类学。该资源支持基于本体的知识多标签分类、文本到权威术语的映射以及代理辅助编目的工作，具备可重复性和以权威为基础的评估能力。我们提供了三个系统的简要统计概况和定性错误分析。我们邀请社区不仅从准确性方面，还从实用性及透明度方面对这些系统进行评价，以便共同开发出锚定于权威、能够增强目录员工作的AI辅助工具。
URL
https://arxiv.org/abs/2603.10876
PDF
-
Detecting Abnormal User Feedback Patterns through Temporal Sentiment Aggregation
2026-03-11 03:07:31Yalun Qi, Sichen Zhao, Zhiming Xue, Xianling Zeng, Zihan YuAbstract
In many real-world applications, such as customer feedback monitoring, brand reputation management, and product health tracking, understanding the temporal dynamics of user sentiment is crucial for early detection of anomalous events such as malicious review campaigns or sudden declines in user satisfaction. Traditional sentiment analysis methods focus on individual text classification, which is insufficient to capture collective behavioral shifts over time due to inherent noise and class imbalance in short user comments. In this work, we propose a temporal sentiment aggregation framework that leverages pretrained transformer-based language models to extract per-comment sentiment signals and aggregates them into time-window-level scores. Significant downward shifts in these aggregated scores are interpreted as potential anomalies in user feedback patterns. We adopt RoBERTa as our core semantic feature extractor and demonstrate, through empirical evaluation on real social media data, that the aggregated sentiment scores reveal meaningful trends and support effective anomaly detection. Experiments on real-world social media data demonstrate that our method successfully identifies statistically significant sentiment drops that correspond to coherent complaint patterns, providing an effective and interpretable solution for feedback anomaly monitoring.
Abstract (translated)
在许多实际应用中，如客户反馈监控、品牌声誉管理和产品健康追踪，理解用户情感的时间动态对于早期检测异常事件（例如恶意刷评活动或用户满意度骤降）至关重要。传统情感分析方法侧重于单条文本分类，但因其固有的噪声和短用户评论中的类别不平衡问题，难以捕捉随时间变化的集体行为趋势。本研究提出一种时序情感聚合框架，该框架利用预训练的Transformer语言模型提取每条评论的情感信号，并将其聚合为时间窗口级别的评分。这些聚合评分的显著下降被解读为用户反馈模式中的潜在异常。我们采用RoBERTa作为核心语义特征提取器，并通过在真实社交媒体数据上的实证评估表明，聚合情感评分能揭示有意义的趋势并支持有效的异常检测。在真实社交媒体数据上的实验证明，我们的方法能够成功识别与连贯投诉模式对应的统计显著情感下降，为反馈异常监控提供了有效且可解释的解决方案。
URL
https://arxiv.org/abs/2604.00020
PDF
-
Correction of Transformer-Based Models with Smoothing Pseudo-Projector
2026-03-10 15:42:46Vitaly BulgakovAbstract
The pseudo-projector is a lightweight modification that can be integrated into existing language models and other neural networks without altering their core architecture. It can be viewed as a hidden-representation corrector that reduces sensitivity to noise by suppressing directions induced by label-irrelevant input content. The design is inspired by the multigrid (MG) paradigm, originally developed to accelerate the convergence of iterative solvers for partial differential equations and boundary value problems, and later extended to more general linear systems through algebraic multigrid methods. We refer to the method as a pseudo-projector because its linear prototype corresponds to a strictly idempotent orthogonal projector, whereas the practical formulation employs learnable restriction and prolongation operators and therefore does not, in general, satisfy the properties of an exact orthogonal projection. We evaluate the proposed approach on transformer-based text classification tasks, as well as controlled synthetic benchmarks, demonstrating its effectiveness in improving training dynamics and robustness. Experimental results, together with supporting theoretical heuristics, indicate consistent improvements in training behavior across a range of settings, with no adverse effects observed otherwise. Our next step will be to extend this approach to language models.
Abstract (translated)
伪投影器是一种轻量级的修改，可以集成到现有的语言模型和其他神经网络中而不改变其核心架构。它可以被视为一种隐藏表示纠正器，通过抑制由标签无关输入内容引起的导向来减少对噪声的敏感性。该设计受到了多重网格（MG）范式的启发，这种范式最初是为了加速偏微分方程和边值问题迭代求解器的收敛速度而开发，并且后来通过代数多重网格方法被扩展到更一般的线性系统中。我们将其称为伪投影器是因为其线性原型对应于严格的幂等正交投影器，而在实际实施方案中使用了可学习的限制操作符和延拓操作符，因此通常不满足精确正交投影所具备的性质。 我们在基于变压器的文本分类任务以及受控合成基准测试上评估了该方法，展示了其在改进训练动态性和提高鲁棒性方面的有效性。实验结果与支持性的理论依据表明，在各种设置下训练行为的一致改善，并且没有观察到其他不良影响。下一步我们将扩展此方法以应用于语言模型。
URL
https://arxiv.org/abs/2603.09815
PDF
-
To Predict or Not to Predict? Towards reliable uncertainty estimation in the presence of noise
2026-03-07 20:22:20Nouran Khallaf, Serge SharoffAbstract
This study examines the role of uncertainty estimation (UE) methods in multilingual text classification under noisy and non-topical conditions. Using a complex-vs-simple sentence classification task across several languages, we evaluate a range of UE techniques against a range of metrics to assess their contribution to making more robust predictions. Results indicate that while methods relying on softmax outputs remain competitive in high-resource in-domain settings, their reliability declines in low-resource or domain-shift scenarios. In contrast, Monte Carlo dropout approaches demonstrate consistently strong performance across all languages, offering more robust calibration, stable decision thresholds, and greater discriminative power even under adverse conditions. We further demonstrate the positive impact of UE on non-topical classification: abstaining from predicting the 10\% most uncertain instances increases the macro F1 score from 0.81 to 0.85 in the Readme task. By integrating UE with trustworthiness metrics, this study provides actionable insights for developing more reliable NLP systems in real-world multilingual environments. See this https URL
Abstract (translated)
这项研究探讨了不确定性估计（UE）方法在多语言文本分类中噪声和非主题条件下的作用。通过跨几种语言的复杂句与简单句分类任务，我们评估了一系列不确定性估计技术，并使用一系列度量标准来评估它们对提高预测稳健性方面的贡献。结果显示，在高资源且领域内设置的情况下，依赖于softmax输出的方法仍然具有竞争力；但在低资源或领域迁移场景中，其可靠性会下降。相比之下，蒙特卡洛 dropout 方法在所有语言中的表现始终强劲，提供了更稳健的校准、稳定的决策阈值和更高的判别力，即使是在不利条件下也是如此。 我们还展示了 UE 对非主题分类的积极影响：放弃预测 10% 最不确定的实例后，在 Readme 任务中宏平均 F1 分数从 0.81 提升到了 0.85。通过将不确定性估计与可信度指标相结合，这项研究为在真实世界的多语言环境中开发更可靠的 NLP 系统提供了实用见解。 参见此 [链接](https://example.com)。
URL
https://arxiv.org/abs/2603.07330
PDF
-
Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Means
2026-03-04 15:48:44Rapha\"el Razafindralambo, R\'emy Sun, Fr\'ed\'eric Precioso, Damien Garreau, Pierre-Alexandre MatteiAbstract
Density aggregation is a central problem in machine learning, for instance when combining predictions from a Deep Ensemble. The choice of aggregation remains an open question with two commonly proposed approaches being linear pooling (probability averaging) and geometric pooling (logit averaging). In this work, we address this question by studying the normalized generalized mean of order $r \in \mathbb{R} \cup \{-\infty,+\infty\}$ through the lens of log-likelihood, the standard evaluation criterion in machine learning. This provides a unifying aggregation formalism and shows different optimal configurations for different situations. We show that the regime $r \in [0,1]$ is the only range ensuring systematic improvements relative to individual distributions, thereby providing a principled justification for the reliability and widespread practical use of linear ($r=1$) and geometric ($r=0$) pooling. In contrast, we show that aggregation rules with $r \notin [0,1]$ may fail to provide consistent gains with explicit counterexamples. Finally, we corroborate our theoretical findings with empirical evaluations using Deep Ensembles on image and text classification benchmarks.
Abstract (translated)
密度聚合是机器学习中的一个核心问题，尤其是在结合深度集成模型的预测时。选择哪种聚合方法仍然是一个开放性的问题，其中两种常用的方法分别是线性池化（概率平均）和几何池化（对数几率平均）。在这项工作中，我们通过从似然度的角度来研究归一化的广义均值$r \in \mathbb{R} \cup \{-\infty,+\infty\}$，为这一问题提供了新的视角。这提供了一个统一的聚合形式，并展示了在不同情况下不同的最优配置。我们证明了当$r \in [0,1]$时是唯一一个能够确保相对于单个分布系统性改进的区间，从而从理论上证明了线性（$r=1$）和几何（$r=0$）池化方法的有效性和广泛实用性。相比之下，我们展示了当$r \notin [0,1]$时聚合规则可能无法提供一致的好处，并通过具体的反例进行了说明。最后，我们使用深度集成模型在图像分类和文本分类基准上进行实验评估来验证我们的理论发现。 这段翻译介绍了研究中探讨的密度聚合问题以及如何通过归一化的广义均值$r \in [0,1]$区间提供了对线性池化和几何池化方法的有效性的解释。同时也指出，当$r$不在这个区间内时，可能存在其他聚合规则不能提供一致改进的风险，并且以具体的反例加以说明。最后提到该研究通过实验验证了理论上的发现，在深度集成模型的图像分类与文本分类任务中进行了实证评估。
URL
https://arxiv.org/abs/2603.04204
PDF
