Abstract
Ensuring the safety alignment of Large Language Models (LLMs) is crucial to generating responses consistent with human values. Despite their ability to recognize and avoid harmful queries, LLMs are vulnerable to "jailbreaking" attacks, where carefully crafted prompts elicit them to produce toxic content. One category of jailbreak attacks is reformulating the task as adversarial attacks by eliciting the LLM to generate an affirmative response. However, the typical attack in this category GCG has very limited attack success rate. In this study, to better study the jailbreak attack, we introduce the DSN (Don't Say No) attack, which prompts LLMs to not only generate affirmative responses but also novelly enhance the objective to suppress refusals. In addition, another challenge lies in jailbreak attacks is the evaluation, as it is difficult to directly and accurately assess the harmfulness of the attack. The existing evaluation such as refusal keyword matching has its own limitation as it reveals numerous false positive and false negative instances. To overcome this challenge, we propose an ensemble evaluation pipeline incorporating Natural Language Inference (NLI) contradiction assessment and two external LLM evaluators. Extensive experiments demonstrate the potency of the DSN and the effectiveness of ensemble evaluation compared to baseline methods.
Abstract (translated)
确保大型语言模型（LLMs）的安全对生成符合人类价值观的响应至关重要。尽管它们能够识别并避免有害查询，但LLMs仍然容易受到“破解”攻击，这种攻击是通过对LLM生成具有毒性内容的精心策划的提示来实现的。其中一种破解攻击是将任务重新建模为对抗性攻击，通过让LLM生成积极响应。然而，这种攻击类型的典型攻击成功率非常有限。 在本研究中，为了更好地研究破解攻击，我们引入了DSN（不要说“不”）攻击，该攻击要求LLM不仅生成积极响应，而且还通过增强目标来抑制拒绝。此外，另一个挑战是破解攻击的评估，因为很难直接且准确地评估攻击的危害。现有的评估方法，如拒绝关键词匹配，本身也有其局限性，因为它揭示了大量的误判和误判实例。为了克服这个挑战，我们提出了一个包含自然语言推理（NLI）矛盾评估和两个外部LLM评估器的元学习评估管道。大量的实验证明，DSN和元学习的组合比基线方法更具有威力。
URL
https://arxiv.org/abs/2404.16369
