Abstract
Few-shot text classification has important application value in low-resource environments. This paper proposes a strategy that combines adaptive fine-tuning, contrastive learning, and regularization optimization to improve the classification performance of Transformer-based models. Experiments on the FewRel 2.0 dataset show that T5-small, DeBERTa-v3, and RoBERTa-base perform well in few-shot tasks, especially in the 5-shot setting, which can more effectively capture text features and improve classification accuracy. The experiment also found that there are significant differences in the classification difficulty of different relationship categories. Some categories have fuzzy semantic boundaries or complex feature distributions, making it difficult for the standard cross entropy loss to learn the discriminative information required to distinguish categories. By introducing contrastive loss and regularization loss, the generalization ability of the model is enhanced, effectively alleviating the overfitting problem in few-shot environments. In addition, the research results show that the use of Transformer models or generative architectures with stronger self-attention mechanisms can help improve the stability and accuracy of few-shot classification.
Abstract (translated)
少量样本文本分类在资源匮乏的环境中具有重要的应用价值。本文提出了一种结合自适应微调、对比学习和正则化优化的策略，以提高基于Transformer模型的分类性能。在FewRel 2.0数据集上的实验表明，T5-small、DeBERTa-v3和RoBERTa-base在少量样本任务中表现良好，尤其是在五次训练设置下，能够更有效地捕捉文本特征并提升分类准确性。实验还发现不同关系类别的分类难度存在显著差异。某些类别具有模糊的语义边界或复杂的特征分布，使得标准交叉熵损失难以学习区分所需的信息。通过引入对比损失和正则化损失，模型的泛化能力得到增强，有效缓解了少量样本环境中过拟合的问题。此外，研究结果表明使用Transformer模型或具有更强自注意力机制的生成架构可以帮助提高少量样本分类的稳定性和准确性。
URL
https://arxiv.org/abs/2505.06145
