Abstract
Intent classification (IC) plays an important role in task-oriented dialogue systems as it identifies user intents from given utterances. However, models trained on limited annotations for IC often suffer from a lack of generalization to unseen intent classes. We propose a novel pre-training method for text encoders that uses contrastive learning with intent psuedo-labels to produce embeddings that are well-suited for IC tasks. By applying this pre-training strategy, we also introduce the pre-trained intent-aware encoder (PIE). Specifically, we first train a tagger to identify key phrases within utterances that are crucial for interpreting intents. We then use these extracted phrases to create examples for pre-training a text encoder in a contrastive manner. As a result, our PIE model achieves up to 5.4% and 4.0% higher accuracy than the previous state-of-the-art pre-trained sentence encoder for the N-way zero- and one-shot settings on four IC datasets.
Abstract (translated)
意图分类(IC)在任务导向对话系统中发挥着重要作用，因为它从给定的对话表达中识别用户的意图。然而，训练基于有限意图分类标注模型通常缺乏对 unseen intent 类的泛化能力。我们提出了一种新的意图编码器预训练方法，该方法使用意图伪标签进行 contrastive 学习，以产生适合 IC 任务的嵌入。通过应用这种方法预训练策略，我们还引入了预训练意图意识到编码器(PIE)。具体而言，我们首先训练一个分词器，以识别对话中的关键短语，这些短语对于解释意图至关重要。然后我们使用这些提取的短语创建用于预训练意图编码器的示例，以进行 contrastive 训练。因此，我们的 PIE 模型在 four IC 数据集上的 N-way 零和一次性设置中实现高达 5.4% 和 4.0% 的准确度提高了先前最先进的意图编码器在四个 IC 数据集上的精度。
URL
https://arxiv.org/abs/2305.14827
