Abstract
Recent breakthroughs in Large Language Models (LLMs) have revealed remarkable generative capabilities and emerging self-regulatory mechanisms, including self-correction and self-rewarding. However, current detoxification techniques rarely exploit these built-in abilities; instead, they rely on external modules, labor-intensive data annotation, or human intervention --factors that hinder scalability and consistency. In this paper, we introduce a fully self-reflective detoxification framework that harnesses the inherent capacities of LLMs to detect, correct toxic content, and refine LLMs without external modules and data annotation. Specifically, we propose a Toxic Signal Detector --an internal self-identification mechanism, coupled with a systematic intervention process to transform toxic text into its non-toxic counterpart. This iterative procedure yields a contrastive detoxification dataset used to fine-tune the model, enhancing its ability for safe and coherent text generation. Experiments on benchmark datasets such as DetoxLLM and ParaDetox show that our method achieves better detoxification performance than state-of-the-art methods while preserving semantic fidelity. By obviating the need for human intervention or external components, this paper reveals the intrinsic self-detoxification ability of LLMs, offering a consistent and effective approach for mitigating harmful content generation. Ultimately, our findings underscore the potential for truly self-regulated language models, paving the way for more responsible and ethically guided text generation systems.
Abstract (translated)
最近在大型语言模型（LLMs）方面取得的突破揭示了其卓越的生成能力和新兴的自我监管机制，包括自校正和自我奖励。然而，目前的去毒化技术很少利用这些内在能力；相反，它们依赖于外部模块、劳动密集型的数据标注或人工干预——这些因素阻碍了可扩展性和一致性。在这篇论文中，我们介绍了一个完全自我反思的去毒框架，该框架利用LLMs固有的能力来检测、纠正有毒内容，并在没有外部模块和数据注释的情况下优化模型。具体而言，我们提出了一种“毒性信号探测器”——一种内部自我识别机制，并结合一个系统干预过程将有毒文本转换为非有毒版本。这一迭代程序生成了用于微调模型的对比去毒数据集，从而增强了其安全和连贯文本生成的能力。在DetoxLLM和ParaDetox等基准数据集上的实验表明，我们的方法比现有的最佳方法实现了更好的去毒效果，并且保持了语义准确性。通过消除对人工干预或外部组件的需求，本文揭示了LLMs内在的自我去毒性能力，提供了一种一致且有效的方法来减轻有害内容生成。最终，我们的发现强调了完全自主调节语言模型的潜力，为更加负责任和道德引导的文本生成系统铺平道路。
URL
https://arxiv.org/abs/2601.11776
