Abstract
To reduce the toxic degeneration in a pretrained Language Model (LM), previous work on Language Model detoxification has focused on reducing the toxicity of the generation itself (self-toxicity) without consideration of the context. As a result, a type of implicit offensive language where the generations support the offensive language in the context is ignored. Different from the LM controlling tasks in previous work, where the desired attributes are fixed for generation, the desired stance of the generation depends on the offensiveness of the context. Therefore, we propose a novel control method to do context-dependent detoxification with the stance taken into consideration. We introduce meta prefixes to learn the contextualized stance control strategy and to generate the stance control prefix according to the input context. The generated stance prefix is then combined with the toxicity control prefix to guide the response generation. Experimental results show that our proposed method can effectively learn the context-dependent stance control strategies while keeping a low self-toxicity of the underlying LM.
Abstract (translated)
为了减少预训练语言模型(LM)的毒性退化,先前在语言模型清洗方面的工作主要关注减少生成自身的毒性(自毒性),而忽视了上下文的影响。因此,忽略了一种隐含的攻击性语言形式,即生成在上下文中支持攻击性语言的表达方式。与之前工作中的LM控制任务不同,该任务中的生成属性是固定的,而生成的立场取决于上下文的攻击性程度。因此,我们提出了一种新的控制方法,考虑了立场的影响,以进行上下文依赖性清洗。我们引入了元前缀,学习上下文背景下的立场控制策略,并根据输入上下文生成立场控制前缀。生成的立场控制前缀随后与毒性控制前缀相结合,以指导响应生成。实验结果显示,我们提出的方法可以在保持底层LM低自毒性的同时,有效地学习上下文背景下的立场控制策略。
URL
https://arxiv.org/abs/2301.10368
