Abstract
Frontier language model capabilities are improving rapidly. We thus need stronger mitigations against bad actors misusing increasingly powerful systems. Prior work has shown that activation probes may be a promising misuse mitigation technique, but we identify a key remaining challenge: probes fail to generalize under important production distribution shifts. In particular, we find that the shift from short-context to long-context inputs is difficult for existing probe architectures. We propose several new probe architecture that handle this long-context distribution shift. We evaluate these probes in the cyber-offensive domain, testing their robustness against various production-relevant shifts, including multi-turn conversations, static jailbreaks, and adaptive red teaming. Our results demonstrate that while multimax addresses context length, a combination of architecture choice and training on diverse distributions is required for broad generalization. Additionally, we show that pairing probes with prompted classifiers achieves optimal accuracy at a low cost due to the computational efficiency of probes. These findings have informed the successful deployment of misuse mitigation probes in user-facing instances of Gemini, Google's frontier language model. Finally, we find early positive results using AlphaEvolve to automate improvements in both probe architecture search and adaptive red teaming, showing that automating some AI safety research is already possible.
Abstract (translated)
前沿语言模型的能力正在迅速提升，因此我们需要更强的措施来防止恶意用户滥用日益强大的系统。先前的研究表明，激活探测器可能是一种有前景的误用缓解技术，但我们识别到一个关键挑战：现有的探测器在面对重要的生产分布变化时无法泛化。特别是我们发现，从短上下文输入向长上下文输入的变化对现有探测器架构来说是一个难题。 为此，我们提出了一些新的探测器架构来处理这种长上下文的分布偏移。我们在网络攻击领域测试了这些探测器的鲁棒性，评估它们在各种生产相关变化中的表现，包括多轮对话、静态破解和适应性的红队行动（模拟对手）。我们的结果显示，虽然多重最大值方法可以解决上下文长度的问题，但要实现广泛泛化，则需要结合架构选择与训练多样化分布。此外，我们还发现将探测器与提示分类器配对可以在低计算成本下达到最佳准确率。 这些研究成果已经指导了谷歌前沿语言模型Gemini中误用缓解探测器的成功部署。最后，我们通过AlphaEvolve早期获得了积极的结果，用于自动化改进探测器架构搜索和适应性红队行动，表明部分AI安全性研究的自动化已经成为可能。
URL
https://arxiv.org/abs/2601.11516
