Abstract
The category of figurative language contains many varieties, some of which are non-compositional in nature. This type of phrase or multi-word expression (MWE) includes idioms, which represent a single meaning that does not consist of the sum of its words. For language models, this presents a unique problem due to tokenization and adjacent contextual embeddings. Many large language models have overcome this issue with large phrase vocabulary, though immediate recognition frequently fails without one- or few-shot prompting or instruction finetuning. The best results have been achieved with BERT-based or LSTM finetuning approaches. The model in this paper contains one such variety. We propose BERT- and RoBERTa-based models finetuned with a combination of slot loss and span contrastive loss (SCL) with hard negative reweighting to improve idiomaticity detection, attaining state of the art sequence accuracy performance on existing datasets. Comparative ablation studies show the effectiveness of SCL and its generalizability. The geometric mean of F1 and sequence accuracy (SA) is also proposed to assess a model's span awareness and general performance together.
Abstract (translated)
比喻性语言包含多种变体，其中部分具有非组合性本质。这类短语或多词表达（MWE）包含习语，其整体含义并非词汇意义的简单叠加。对于语言模型而言，由于分词机制和相邻上下文嵌入的影响，这构成了独特挑战。许多大语言模型通过扩充短语词汇库克服了该问题，但若无小样本提示或指令微调，即时识别仍常会失败。目前最佳成果多基于BERT或LSTM微调方法实现。本文模型即属此类变体之一。我们提出基于BERT和RoBERTa的模型，通过结合槽位损失与带困难负样本重加权的跨度对比损失（SCL）进行微调，以提升习语性检测能力，并在现有数据集上达到最优序列准确率性能。对比消融实验验证了SCL的有效性及其泛化能力。此外，我们提出采用F1值与序列准确率（SA）的几何平均数，以综合评估模型的跨度感知能力与整体性能。
URL
https://arxiv.org/abs/2603.22799
