Abstract
The widespread use of text-based communication on social media-through chats, comments, and microblogs-has improved user interaction but has also led to an increase in offensive content, including hate speech, racism, and other forms of abuse. Due to the enormous volume of user-generated content, manual moderation is impractical, which creates a need for automated systems that can detect offensive language. Deep learning models, particularly those using transfer learning, have demonstrated significant success in understanding natural language through large-scale pretraining. In this study, we propose an automatic offensive language detection model based on XLNet, a generalized autoregressive pretraining method, and compare its performance with BERT (Bidirectional Encoder Representations from Transformers), which is a widely used baseline in natural language processing (NLP). Both models are evaluated using the Offensive Language Identification Dataset (OLID), a benchmark Twitter dataset that includes hierarchical annotations. Our experimental results show that XLNet outperforms BERT in detecting offensive content and in categorizing the types of offenses, while BERT performs slightly better in identifying the targets of the offenses. Additionally, we find that oversampling and undersampling strategies are effective in addressing class imbalance and improving classification performance. These findings highlight the potential of transfer learning and XLNet-based architectures to create robust systems for detecting offensive language on social media platforms.
Abstract (translated)
社交媒体上基于文本的通信——通过聊天、评论和微博等形式——虽然增强了用户间的互动，但也导致了包括仇恨言论、种族主义和其他形式滥用在内的攻击性内容增多。由于用户生成的内容数量庞大，人工审核变得不切实际，因此需要能够自动检测攻击性语言的系统。深度学习模型，特别是那些使用迁移学习的方法，在通过大规模预训练来理解自然语言方面表现出显著的成功。在这项研究中，我们提出了一种基于XLNet（一种广义自回归预训练方法）的自动检测攻击性语言的模型，并将其性能与广泛应用于自然语言处理（NLP）领域的基准BERT模型进行了比较。这两种模型均使用了带有层次化注释的 Offensive Language Identification Dataset (OLID) 进行评估，这是一个基准推特数据集。 实验结果显示，XLNet在检测和分类攻击内容方面优于BERT，而BERT在识别攻击目标时略胜一筹。此外，我们发现过采样和欠采样策略对于解决类别不平衡问题并提高分类性能有效。这些发现强调了迁移学习以及基于XLNet架构的潜力，在创建能够探测社交媒体平台上攻击性语言的强大系统中具有重要意义。
URL
https://arxiv.org/abs/2506.21795
