Abstract
Classification is one of the most widespread tasks in AI applications, serving often as the first step in filtering, sorting, and categorizing data. Since modern AI systems must handle large volumes of input data and early pipeline stages can propagate errors downstream, achieving high efficiency and accuracy is critical. Moreover, classification requirements can change dynamically based on user needs, necessitating models with strong zero-shot capabilities. While generative LLMs have become mainstream for zero-shot classification due to their versatility, they suffer from inconsistent instruction following and computational inefficiency. Cross-encoders, commonly used as rerankers in RAG pipelines, face a different bottleneck: they must process text-label pairs sequentially, significantly reducing efficiency with large label sets. Embedding-based approaches offer good efficiency but struggle with complex scenarios involving logical and semantic constraints. We propose GLiClass, a novel method that adapts the GLiNER architecture for sequence classification tasks. Our approach achieves strong accuracy and efficiency comparable to embedding-based methods, while maintaining the flexibility needed for zero-shot and few-shot learning scenarios. Additionally, we adapted proximal policy optimization (PPO) for multi-label text classification, enabling training classifiers in data-sparse conditions or from human feedback.
Abstract (translated)
分类是人工智能应用中最常见的任务之一，通常作为过滤、排序和对数据进行分类的第一步。鉴于现代AI系统必须处理大量输入数据，并且早期管道阶段的错误可能会向下游传播，因此实现高效率和准确性至关重要。此外，由于用户需求的变化，分类要求可能动态变化，这需要具备强大零样本学习能力的模型。虽然基于生成的大语言模型（LLM）因其多功能性而在零样本分类中变得主流，但它们在遵循指令的一致性和计算效率方面存在问题。交叉编码器通常用于检索增强生成（RAG）管道中的重排序阶段，面临不同的瓶颈：它们必须按顺序处理文本-标签对，在面对大型标签集时会显著降低效率。基于嵌入的方法提供了较好的效率，但在涉及逻辑和语义约束的复杂场景中表现不佳。 我们提出了一种新的方法GLiClass，它将GLiNER架构应用于序列分类任务。我们的方法在准确性方面表现出色，并且与基于嵌入的方法相比，在效率方面也具有可比性，同时保持了零样本学习和少样本学习场景所需的灵活性。此外，我们将近端策略优化（PPO）用于多标签文本分类，从而能够在数据稀疏条件下或从人类反馈中训练分类器。
URL
https://arxiv.org/abs/2508.07662
