Abstract
We propose WIBA, a novel framework and suite of methods that enable the comprehensive understanding of "What Is Being Argued" across contexts. Our approach develops a comprehensive framework that detects: (a) the existence, (b) the topic, and (c) the stance of an argument, correctly accounting for the logical dependence among the three tasks. Our algorithm leverages the fine-tuning and prompt-engineering of Large Language Models. We evaluate our approach and show that it performs well in all the three capabilities. First, we develop and release an Argument Detection model that can classify a piece of text as an argument with an F1 score between 79% and 86% on three different benchmark datasets. Second, we release a language model that can identify the topic being argued in a sentence, be it implicit or explicit, with an average similarity score of 71%, outperforming current naive methods by nearly 40%. Finally, we develop a method for Argument Stance Classification, and evaluate the capability of our approach, showing it achieves a classification F1 score between 71% and 78% across three diverse benchmark datasets. Our evaluation demonstrates that WIBA allows the comprehensive understanding of What Is Being Argued in large corpora across diverse contexts, which is of core interest to many applications in linguistics, communication, and social and computer science. To facilitate accessibility to the advancements outlined in this work, we release WIBA as a free open access platform (wiba.dev).
Abstract (translated)
我们提出了WIBA，一种新的框架和方法，可让您全面理解跨背景下的“什么是被争论的”。我们的方法开发了一个全面的框架，能检测到：（a）存在的论据，（b）主题和（c）论点，正确地解释了这三个任务之间的逻辑依赖关系。我们的算法利用了大型语言模型的微调和高提示工程。我们评估了我们的方法，并证明了它在所有三个能力上都表现出色。首先，我们开发并发布了一个人工论据检测模型，在三个不同的基准数据集上，该模型的F1得分在79%至86%之间。其次，我们发布了一个语言模型，可以识别句子中正在被争论的主题，无论是隐含还是明确的，平均相似度为71%，几乎比当前的 naive 方法快40%。最后，我们开发了一种论点分类方法，并评估了我们的方法的表现，证明它在大致不同的基准数据集上的分类F1得分在71%至78%之间。我们的评估表明，WIBA允许在大型语料库的多样性背景下全面理解“什么是被争论的”，这正是许多语言学、交流和计算机科学等应用的核心兴趣所在。为了方便您了解本工作的进展，我们已将WIBA作为免费开源平台（wiba.dev）发布。
URL
https://arxiv.org/abs/2405.00828
