Abstract
Prompt-based methods have achieved promising results in most few-shot text classification tasks. However, for readability assessment tasks, traditional prompt methods lackcrucial linguistic knowledge, which has already been proven to be essential. Moreover, previous studies on utilizing linguistic features have shown non-robust performance in few-shot settings and may even impair model this http URL address these issues, we propose a novel prompt-based tuning framework that incorporates rich linguistic knowledge, called Feature Prompt Tuning (FPT). Specifically, we extract linguistic features from the text and embed them into trainable soft prompts. Further, we devise a new loss function to calibrate the similarity ranking order between categories. Experimental results demonstrate that our proposed method FTP not only exhibits a significant performance improvement over the prior best prompt-based tuning approaches, but also surpasses the previous leading methods that incorporate linguistic features. Also, our proposed model significantly outperforms the large language model gpt-3.5-turbo-16k in most cases. Our proposed method establishes a new architecture for prompt tuning that sheds light on how linguistic features can be easily adapted to linguistic-related tasks.
Abstract (translated)
基于提示的方法在大多数几条文本分类任务中取得了良好的效果。然而，对于可读性评估任务，传统提示方法缺乏关键的语言知识，这已经被证明是必要的。此外，之前关于使用语言特征的研究表明，在几条设置中表现非鲁棒，甚至可能损害模型，我们提出了一种新颖的提示基于调整框架，称为特征提示调整（FPT）。具体来说，我们从文本中提取语言特征并将它们嵌入到可训练的轻量级提示中。进一步，我们设计了一个新的损失函数来调整类别之间的相似度排名顺序。实验结果表明，与之前最好的基于提示的调整方法相比，我们的FPT方法不仅表现出显著的性能提升，而且超过了之前采用语言特征的主要方法。此外，在大多数情况下，我们的方法显著优于大型语言模型gpt-3.5-turbo-16k。我们提出的方法建立了一个新的提示调整架构，阐明了语言特征如何轻松适应与语言相关的任务。
URL
https://arxiv.org/abs/2404.02772
