Abstract
Zero-shot text classification remains a difficult task in domains with evolving knowledge and ambiguous category boundaries, such as ticketing systems. Large language models (LLMs) often struggle to generalize in these scenarios due to limited topic separability, while few-shot methods are constrained by insufficient data diversity. We propose a classification framework that combines iterative topic refinement, contrastive prompting, and active learning. Starting with a small set of labeled samples, the model generates initial topic labels. Misclassified or ambiguous samples are then used in an iterative contrastive prompting process to refine category distinctions by explicitly teaching the model to differentiate between closely related classes. The framework features a human-in-the-loop component, allowing users to introduce or revise category definitions in natural language. This enables seamless integration of new, unseen categories without retraining, making the system well-suited for real-world, dynamic environments. The evaluations on AGNews and DBpedia demonstrate strong performance: 91% accuracy on AGNews (3 seen, 1 unseen class) and 84% on DBpedia (8 seen, 1 unseen), with minimal accuracy shift after introducing unseen classes (82% and 87%, respectively). The results highlight the effectiveness of prompt-based semantic reasoning for fine-grained classification with limited supervision.
Abstract (translated)
在知识不断演变且类别边界模糊的领域（如票务系统）中，零样本文本分类仍然是一项具有挑战性的任务。大型语言模型（LLM）往往由于话题分离度有限而难以在这种场景下泛化，而少样本方法则受限于数据多样性不足的问题。为此，我们提出了一种结合迭代主题细化、对比提示和主动学习的分类框架。该框架从一组小规模标记样本开始，生成初始的主题标签。之后，通过将误分类或模棱两可的样本用于迭代对比提示过程来进一步优化类别区分度，使模型能够明确地区分出紧密相关的类。 此框架的一个关键特点是包含了一个“人机交互”组件，允许用户以自然语言形式引入或修订类别定义。这使得系统能够在无需重新训练的情况下无缝集成新的、未见过的类别，使其非常适合于现实世界中不断变化的环境。 在AGNews和DBpedia数据集上的评估结果表明了该方法的强大性能：在包含3个已知类和1个未知类的AGNews上达到了91%的准确率，在包含8个已知类和1个未知类的DBpedia上达到了84%的准确率。值得注意的是，引入新的未见过类别后，模型表现基本没有显著下降（分别为82%和87%），这进一步证明了提示语驱动的语义推理在有限监督下的细粒度分类任务中的有效性。
URL
https://arxiv.org/abs/2508.00957
