Abstract
The ease and the speed of spreading misinformation and propaganda on the Web motivate the need to develop trustworthy technology for detecting fallacies in natural language arguments. However, state-of-the-art language modeling methods exhibit a lack of robustness on tasks like logical fallacy classification that require complex reasoning. In this paper, we propose a Case-Based Reasoning method that classifies new cases of logical fallacy by language-modeling-driven retrieval and adaptation of historical cases. We design four complementary strategies to enrich the input representation for our model, based on external information about goals, explanations, counterarguments, and argument structure. Our experiments in in-domain and out-of-domain settings indicate that Case-Based Reasoning improves the accuracy and generalizability of language models. Our ablation studies confirm that the representations of similar cases have a strong impact on the model performance, that models perform well with fewer retrieved cases, and that the size of the case database has a negligible effect on the performance. Finally, we dive deeper into the relationship between the properties of the retrieved cases and the model performance.
Abstract (translated)
在互联网中传播虚假信息和宣传的便利性和速度激励着开发可靠的技术,以检测自然语言推理中的谬误。然而,最先进的语言建模方法在需要进行复杂推理的任务中表现出缺乏鲁棒性。在本文中,我们提出了一种基于案例推理的方法,该方法通过语言建模驱动的历史案例的检索和适应,将新的案例逻辑谬误进行分类。我们设计四个互补的策略,以丰富我们的模型输入表示,基于目标、解释、反证和论点结构外部信息。我们的在领域和跨领域的实验结果表明,基于案例推理可以提高语言模型的准确性和泛化能力。我们的削弱研究确认,类似案例的表示对模型性能有强烈的影响,模型在检索案例更少的情况下表现良好,而数据库大小对性能的影响几乎忽略不计。最后,我们深入探讨了检索案例的属性和模型性能之间的关系。
URL
https://arxiv.org/abs/2301.11879
