Abstract
Large Language Models (LLMs) demonstrate strong in-context learning abilities, yet their effectiveness in text classification depends heavily on prompt design and incurs substantial computational cost. Conformal In-Context Learning (CICLe) has been proposed as a resource-efficient framework that integrates a lightweight base classifier with Conformal Prediction to guide LLM prompting by adaptively reducing the set of candidate classes. However, its broader applicability and efficiency benefits beyond a single domain have not yet been systematically explored. In this paper, we present a comprehensive evaluation of CICLe across diverse NLP classification benchmarks. The results show that CICLe consistently improves over its base classifier and outperforms few-shot prompting baselines when the sample size is sufficient for training the base classifier, and performs comparably in low-data regimes. In terms of efficiency, CICLe reduces the number of shots and prompt length by up to 34.45% and 25.16%, respectively, and enables the use of smaller models with competitive performance. CICLe is furthermore particularly advantageous for text classification tasks with high class imbalance. These findings highlight CICLe as a practical and scalable approach for efficient text classification, combining the robustness of traditional classifiers with the adaptability of LLMs, and achieving substantial gains in data and computational efficiency.
Abstract (translated)
大型语言模型（LLMs）展现了强大的上下文学习能力，但它们在文本分类中的效果严重依赖于提示设计，并且计算成本高。为了解决资源效率问题，提出了符合上下文学习（CICLe）框架，该框架将轻量级的基础分类器与符合预测结合，通过自适应减少候选类别集来指导LLM的提示生成。然而，其在单个领域之外的应用广度和效率优势尚未系统地探索。 本文中，我们对CICLe进行了全面评估，覆盖了多样化的NLP分类基准测试。结果显示，在基础分类器训练所需的样本量充足时，CICLe始终优于它的基础分类器，并且在少量提示基线的条件下表现出色；而在低数据环境中表现相当。 从效率角度来看，CICLe将示例数量和提示长度分别减少了最多34.45%和25.16%，并且允许使用性能竞争的小型模型。此外，在高类别不平衡的文本分类任务中，CICLe尤其具有优势。 这些发现突显了CICLe作为一种实用且可扩展的方法，用于高效的文本分类，结合传统分类器的稳健性与LLM的适应性，并在数据和计算效率方面取得了显著改进。
URL
https://arxiv.org/abs/2512.05732
