Abstract
Multi-intent natural language understanding (NLU) presents a formidable challenge due to the model confusion arising from multiple intents within a single utterance. While previous works train the model contrastively to increase the margin between different multi-intent labels, they are less suited to the nuances of multi-intent NLU. They ignore the rich information between the shared intents, which is beneficial to constructing a better embedding space, especially in low-data scenarios. We introduce a two-stage Prediction-Aware Contrastive Learning (PACL) framework for multi-intent NLU to harness this valuable knowledge. Our approach capitalizes on shared intent information by integrating word-level pre-training and prediction-aware contrastive fine-tuning. We construct a pre-training dataset using a word-level data augmentation strategy. Subsequently, our framework dynamically assigns roles to instances during contrastive fine-tuning while introducing a prediction-aware contrastive loss to maximize the impact of contrastive learning. We present experimental results and empirical analysis conducted on three widely used datasets, demonstrating that our method surpasses the performance of three prominent baselines on both low-data and full-data scenarios.
Abstract (translated)
由于在一个句子中存在多个意图，多意图自然语言理解（NLU）面临着巨大的挑战。虽然以前的工作通过对比训练来增加不同多意图标签之间的间隔，但他们并不适合多意图NLU的细微差别。他们忽略了共享意图之间的丰富信息，这对于构建更好的嵌入空间尤其在低数据场景中是很有益的。我们提出了一个两阶段预测意识对比学习（PACL）框架，用于多意图NLU，以充分利用这种有价值的信息。我们的方法通过将词级预训练和预测意识对比微调相结合，利用共享意图信息，构建了一个预训练数据集。在对比微调期间，我们的框架动态地为实例分配角色，并引入预测意识对比损失以最大化对比学习的影响。我们在三个广泛使用数据集上进行了实验和实证分析，结果表明，我们的方法在低数据和全数据场景上都超越了三个显著的基本方法。
URL
https://arxiv.org/abs/2405.02925
