Abstract
The automatic detection of disinformation presents a significant challenge in the field of natural language processing. This task addresses a multifaceted societal and communication issue, which needs approaches that extend beyond the identification of general linguistic patterns through data-driven algorithms. In this research work, we hypothesise that text classification methods are not able to capture the nuances of disinformation and they often ground their decision in superfluous features. Hence, we apply a post-hoc explainability method (SHAP, SHapley Additive exPlanations) to identify spurious elements with high impact on the classification models. Our findings show that non-informative elements (e.g., URLs and emoticons) should be removed and named entities (e.g., Rwanda) should be pseudo-anonymized before training to avoid models' bias and increase their generalization capabilities. We evaluate this methodology with internal dataset and external dataset before and after applying extended data preprocessing and named entity replacement. The results show that our proposal enhances on average the performance of a disinformation classification method with external test data in 65.78% without a significant decrease of the internal test performance.
Abstract (translated)
自动检测虚假信息在自然语言处理领域中面临重大挑战。这一任务涉及到多方面的社会和沟通问题，需要超越通过数据驱动算法识别一般语言模式的方法。在这项研究工作中，我们假设文本分类方法无法捕捉到虚假信息的细微差别，并且它们通常依赖于无关紧要的特征进行决策。因此，我们应用了一种事后可解释性方法（SHAP, SHapley Additive exPlanations）来识别对分类模型影响较大的不相关元素。我们的研究发现表明，在训练之前应该移除非相关信息（例如网址和表情符号），并匿名化命名实体（例如卢旺达）以避免模型偏差，提高其泛化能力。 我们使用内部数据集和外部数据集在应用扩展的数据预处理和命名实体替换前后评估了这一方法。结果显示，我们的提议方法平均提高了65.78%的虚假信息分类方法在外部测试数据上的性能，并且没有显著降低内部测试的表现。
URL
https://arxiv.org/abs/2502.04863
