Abstract
The advancement of Large Language Models (LLMs) has transformed Natural Language Processing (NLP), enabling performance across diverse tasks with little task-specific training. However, LLMs remain susceptible to social biases, particularly reflecting harmful stereotypes from training data, which can disproportionately affect marginalised communities. We measure gender bias in Maltese LMs, arguing that such bias is harmful as it reinforces societal stereotypes and fails to account for gender diversity, which is especially problematic in gendered, low-resource languages. While bias evaluation and mitigation efforts have progressed for English-centric models, research on low-resourced and morphologically rich languages remains limited. This research investigates the transferability of debiasing methods to Maltese language models, focusing on BERTu and mBERTu, BERT-based monolingual and multilingual models respectively. Bias measurement and mitigation techniques from English are adapted to Maltese, using benchmarks such as CrowS-Pairs and SEAT, alongside debiasing methods Counterfactual Data Augmentation, Dropout Regularization, Auto-Debias, and GuiDebias. We also contribute to future work in the study of gender bias in Maltese by creating evaluation datasets. Our findings highlight the challenges of applying existing bias mitigation methods to linguistically complex languages, underscoring the need for more inclusive approaches in the development of multilingual NLP.
Abstract (translated)
大型语言模型（LLMs）的发展已经革新了自然语言处理（NLP），使其能够在几乎不需要特定任务训练的情况下完成多样化的任务。然而，LLMs仍然容易受到社会偏见的影响，特别是从训练数据中反映出有害的刻板印象，这会对边缘化社区产生不成比例的影响。我们测量马耳他语模型中的性别偏见，并主张这种偏见是有害的，因为它强化了社会上的刻板印象并且未能考虑到性别多样性，这对性别化且资源较少的语言来说尤其成问题。 尽管针对以英语为中心的模型的偏见评估和缓解工作已经有所进展，但对于低资源和形态丰富的语言的研究仍然有限。这项研究探讨了将去偏方法转移到马耳他语模型上的可行性，重点分析了基于BERT的单语模型BERTu和多语种模型mBERTu。从英语中借鉴的偏见测量与缓解技术被应用于马耳他语，并使用CrowS-Pairs和SEAT等基准进行测试，同时应用Counterfactual Data Augmentation（反事实数据增强）、Dropout Regularization（dropout正则化）、Auto-Debias（自动去偏）以及GuiDebias等方法。此外，我们还通过创建评估数据集为未来研究马耳他语中的性别偏见工作做出了贡献。 我们的发现强调了将现有的偏见缓解方法应用于语言复杂性较高的语言所面临的挑战，并突显了在多语言NLP开发中采用更具包容性的方法的必要性。
URL
https://arxiv.org/abs/2507.03142
