Abstract
The rapid deployment of Large Language Models (LLMs) has created an urgent need for enhanced security and privacy measures in Machine Learning (ML). LLMs are increasingly being used to process untrusted text inputs and even generate executable code, often while having access to sensitive system controls. To address these security concerns, several companies have introduced guard models, which are smaller, specialized models designed to protect text generation models from adversarial or malicious inputs. In this work, we advance the study of adversarial inputs by introducing Super Suffixes, suffixes capable of overriding multiple alignment objectives across various models with different tokenization schemes. We demonstrate their effectiveness, along with our joint optimization technique, by successfully bypassing the protection mechanisms of Llama Prompt Guard 2 on five different text generation models for malicious text and code generation. To the best of our knowledge, this is the first work to reveal that Llama Prompt Guard 2 can be compromised through joint optimization. Additionally, by analyzing the changing similarity of a model's internal state to specific concept directions during token sequence processing, we propose an effective and lightweight method to detect Super Suffix attacks. We show that the cosine similarity between the residual stream and certain concept directions serves as a distinctive fingerprint of model intent. Our proposed countermeasure, DeltaGuard, significantly improves the detection of malicious prompts generated through Super Suffixes. It increases the non-benign classification rate to nearly 100%, making DeltaGuard a valuable addition to the guard model stack and enhancing robustness against adversarial prompt attacks.
Abstract (translated)
大型语言模型（LLMs）的快速部署已经产生了对机器学习（ML）中增强的安全性和隐私措施的迫切需求。这些模型越来越多地被用于处理未经验证的文本输入，甚至生成可执行代码，并且经常能够访问敏感系统控制。为解决安全问题，一些公司推出了防护模型，即专门设计的小型模型，旨在保护文本生成模型免受对抗性或恶意输入的影响。在这项工作中，我们通过引入超级后缀（Super Suffixes）来推进对对抗性输入的研究，这些是能够在具有不同分词方案的各种模型上覆盖多个对齐目标的后缀。我们展示了它们的有效性以及我们的联合优化技术，并成功绕过了五种不同的文本生成模型中的Llama Prompt Guard 2的安全机制以生成恶意文本和代码。据我们所知，这是首次揭示Llama Prompt Guard 2可以通过联合优化被攻破的工作。 此外，通过分析模型在处理令牌序列时内部状态与特定概念方向之间的相似性变化，我们提出了一种有效的、轻量级的方法来检测超级后缀攻击。我们证明了残差流和某些概念方向之间的余弦相似度构成了模型意图的独特指纹。我们的防御措施DeltaGuard显著提高了对通过超级后缀生成的恶意提示进行检测的能力。它将非良性分类率提高到接近100%，使DeltaGuard成为防护模型堆栈中的宝贵补充，并增强了对抗性提示攻击的鲁棒性。
URL
https://arxiv.org/abs/2512.11783
