Abstract
Idiom detection using Natural Language Processing (NLP) is the computerized process of recognizing figurative expressions within a text that convey meanings beyond the literal interpretation of the words. While idiom detection has seen significant progress across various languages, the Kurdish language faces a considerable research gap in this area despite the importance of idioms in tasks like machine translation and sentiment analysis. This study addresses idiom detection in Sorani Kurdish by approaching it as a text classification task using deep learning techniques. To tackle this, we developed a dataset containing 10,580 sentences embedding 101 Sorani Kurdish idioms across diverse contexts. Using this dataset, we developed and evaluated three deep learning models: KuBERT-based transformer sequence classification, a Recurrent Convolutional Neural Network (RCNN), and a BiLSTM model with an attention mechanism. The evaluations revealed that the transformer model, the fine-tuned BERT, consistently outperformed the others, achieving nearly 99% accuracy while the RCNN achieved 96.5% and the BiLSTM 80%. These results highlight the effectiveness of Transformer-based architectures in low-resource languages like Kurdish. This research provides a dataset, three optimized models, and insights into idiom detection, laying a foundation for advancing Kurdish NLP.
Abstract (translated)
使用自然语言处理（NLP）进行成语检测是通过计算机识别文本中表达超出字面意义的比喻性表达的过程。尽管在多种语言上已经取得了显著进展，但库尔德语（尤其是索拉尼库尔德语）在这个领域的研究仍然存在较大空白，这不利于机器翻译和情感分析等任务的发展。这项研究表明如何使用深度学习技术将索拉尼库尔德语的成语检测转化为文本分类问题。 为了实现这一目标，我们创建了一个包含10,580个句子的数据集，这些句子中嵌入了101条索拉尼库尔德语成语，并涵盖了不同的上下文。利用这个数据集，我们开发并评估了三种深度学习模型：基于KuBERT的Transformer序列分类、递归卷积神经网络（RCNN）以及带有注意力机制的双层长短期记忆网络（BiLSTM）。在实验中，变压器模型（微调后的BERT模型）的表现始终优于其他两种模型，在准确性上接近99%，而RCNN和BiLSTM分别达到了96.5% 和80%。 这些结果突显了基于Transformer架构的有效性，尤其是在像库尔德语这样的低资源语言环境中。这项研究不仅提供了一个数据集、三种优化后的模型以及有关成语检测的见解，并为推动库尔德语自然语言处理的进步奠定了基础。
URL
https://arxiv.org/abs/2501.14528
