Abstract
The prevalence of digital media and evolving sociopolitical dynamics have significantly amplified the dissemination of hateful content. Existing studies mainly focus on classifying texts into binary categories, often overlooking the continuous spectrum of offensiveness and hatefulness inherent in the text. In this research, we present an extensive benchmark dataset for Amharic, comprising 8,258 tweets annotated for three distinct tasks: category classification, identification of hate targets, and rating offensiveness and hatefulness intensities. Our study highlights that a considerable majority of tweets belong to the less offensive and less hate intensity levels, underscoring the need for early interventions by stakeholders. The prevalence of ethnic and political hatred targets, with significant overlaps in our dataset, emphasizes the complex relationships within Ethiopia's sociopolitical landscape. We build classification and regression models and investigate the efficacy of models in handling these tasks. Our results reveal that hate and offensive speech can not be addressed by a simplistic binary classification, instead manifesting as variables across a continuous range of values. The Afro-XLMR-large model exhibits the best performances achieving F1-scores of 75.30%, 70.59%, and 29.42% for the category, target, and regression tasks, respectively. The 80.22% correlation coefficient of the Afro-XLMR-large model indicates strong alignments.
Abstract (translated)
数字媒体和不断变化的社会政治动态显著增强了仇恨内容的传播。现有的研究主要集中在将文本分类为二元类别，往往忽视了文本中存在的连续的冒犯和仇恨程度。在这项研究中，我们提出了一个广泛的哈马斯语 benchmark 数据集，包括 8,258 条推特，分别用于三个不同的任务：分类、识别仇恨目标和评分冒犯力和仇恨程度。我们的研究强调，绝大多数推特属于不太冒犯和不太仇恨的程度，这需要利益相关者的早期干预。民族和政治仇恨目标的普遍存在，在我们的数据集中具有显著的重叠，强调了 Ethiopia 社会政治格局中复杂的关系。我们构建了分类和回归模型，并研究了这些任务中模型的效果。我们的结果表明，简单的二元分类无法解决仇恨和冒犯性言论的问题，反而表现为一个连续范围内的变量。Afro-XLMR-large 模型在分类、目标和回归任务上都取得了最佳性能，分别达到 F1 分数为 75.30%、70.59% 和 29.42%。Afro-XLMR-large 模型的 80.22% 相关系数表明很强的 alignments。
URL
https://arxiv.org/abs/2404.12042
