Abstract
Counter-speech generation is at the core of many expert activities, such as fact-checking and hate speech, to counter harmful content. Yet, existing work treats counter-speech generation as pure text generation task, mainly based on Large Language Models or NGO experts. These approaches show severe drawbacks due to the limited reliability and coherence in the generated countering text, and in scalability, respectively. To close this gap, we introduce a novel framework to model counter-speech generation as knowledge-wise text generation process. Our framework integrates advanced Retrieval-Augmented Generation (RAG) pipelines to ensure the generation of trustworthy counter-speech for 8 main target groups identified in the hate speech literature, including women, people of colour, persons with disabilities, migrants, Muslims, Jews, LGBT persons, and other. We built a knowledge base over the United Nations Digital Library, EUR-Lex and the EU Agency for Fundamental Rights, comprising a total of 32,792 texts. We use the MultiTarget-CONAN dataset to empirically assess the quality of the generated counter-speech, both through standard metrics (i.e., JudgeLM) and a human evaluation. Results show that our framework outperforms standard LLM baselines and competitive approach, on both assessments. The resulting framework and the knowledge base pave the way for studying trustworthy and sound counter-speech generation, in hate speech and beyond.
Abstract (translated)
反言论生成是许多专家活动的核心，例如事实核查和仇恨言论处理，旨在对抗有害内容。然而，现有的研究将反言论生成视为纯粹的文本生成任务，并主要依赖于大型语言模型或非政府组织（NGO）专家。这些方法因生成的反驳文本可靠性和连贯性有限以及在可扩展性方面的问题而表现出严重缺陷。 为弥补这一不足，我们引入了一种新颖框架，将反言论生成建模为基于知识的文本生成过程。我们的框架整合了先进的检索增强生成（RAG）管道，以确保针对仇恨言论文献中识别出的8个主要目标群体产生可信的反驳言论，这些群体包括女性、有色人种、残疾人、移民、穆斯林、犹太人、LGBT人士以及其他群体。我们在联合国数字图书馆、EUR-Lex和欧盟基本权利局的基础上构建了一个知识库，总计包含32,792条文本。 我们使用MultiTarget-CONAN数据集通过标准指标（即JudgeLM）和人工评估来实证检验生成的反言论质量。结果显示，我们的框架在两项评估中均优于传统的大型语言模型基线和其他竞争方法。 由此产生的框架和知识库为研究可信且合理的反言论生成铺平了道路，不仅限于仇恨言论领域。
URL
https://arxiv.org/abs/2510.12316
