Abstract
Digital platforms have an ever-expanding user base, and act as a hub for communication, business, and connectivity. However, this has also allowed for the spread of hate speech and misogyny. Artificial intelligence models have emerged as an effective solution for countering online hate speech but are under explored for low resource and code-mixed languages and suffer from a lack of interpretability. Explainable Artificial Intelligence (XAI) can enhance transparency in the decisions of deep learning models, which is crucial for a sensitive domain such as hate speech detection. In this paper, we present a multi-modal and explainable web application for detecting misogyny in text and memes in code-mixed Hindi and English. The system leverages state-of-the-art transformer-based models that support multilingual and multimodal settings. For text-based misogyny identification, the system utilizes XLM-RoBERTa (XLM-R) and multilingual Bidirectional Encoder Representations from Transformers (mBERT) on a dataset of approximately 4,193 comments. For multimodal misogyny identification from memes, the system utilizes mBERT + EfficientNet, and mBERT + ResNET trained on a dataset of approximately 4,218 memes. It also provides feature importance scores using explainability techniques including Shapley Additive Values (SHAP) and Local Interpretable Model Agnostic Explanations (LIME). The application aims to serve as a tool for both researchers and content moderators, to promote further research in the field, combat gender based digital violence, and ensure a safe digital space. The system has been evaluated using human evaluators who provided their responses on Chatbot Usability Questionnaire (CUQ) and User Experience Questionnaire (UEQ) to determine overall usability.
Abstract (translated)
数字平台的用户群体不断扩张，这些平台已成为沟通、商务和连接的核心枢纽。然而，这也导致了仇恨言论和性别歧视言论在网络上的传播。人工智能模型已被证明是应对在线仇恨言论的有效解决方案，但它们在低资源语言和代码混合语言中的应用尚不充分，并且缺乏解释性。 可解释的人工智能（XAI）能够增强深度学习模型决策的透明度，在如仇恨言论检测这样敏感领域中尤为重要。本文介绍了一个多模态、可解释的网页应用程序，用于检测代码混合的语言——印地语和英语中的性别歧视文本及网络图片（memes）。该系统利用了支持多语言和多模式设置的最先进的变压器模型。对于基于文本的性别歧视识别，系统使用XLM-RoBERTa (XLM-R) 和多语言双向编码器表示模型(mBERT)，分析大约4,193条评论数据集。对于从网络图片（memes）中进行的多模态性别歧视识别，该系统利用了mBERT + EfficientNet和mBERT + ResNET，并且基于一个大约有4,218个网络图片的数据集进行了训练。此外，它还通过可解释性技术提供了特征重要性得分，包括Shapley Additive Values (SHAP) 和 Local Interpretable Model Agnostic Explanations (LIME)。 该应用程序旨在成为研究人员和内容审核员的工具，以推动这一领域的进一步研究、对抗性别数字暴力，并确保一个安全的在线环境。系统通过人类评估者提供的Chatbot Usability Questionnaire（CUQ）和User Experience Questionnaire（UEQ）反馈进行了评估，从而确定整体可用性。
URL
https://arxiv.org/abs/2601.08457
