Abstract
Despite the fact that large language models (LLMs) show exceptional skill in instruction following tasks, this strength can turn into a vulnerability when the models are required to disregard certain instructions. Instruction-following tasks typically involve a clear task description and input text containing the target data to be processed. However, when the input itself resembles an instruction, confusion may arise, even if there is explicit prompting to distinguish between the task instruction and the input. We refer to this phenomenon as instructional distraction. In this paper, we introduce a novel benchmark, named DIM-Bench, specifically designed to assess LLMs' performance under instructional distraction. The benchmark categorizes real-world instances of instructional distraction and evaluates LLMs across four instruction tasks: rewriting, proofreading, translation, and style transfer -- alongside five input tasks: reasoning, code generation, mathematical reasoning, bias detection, and question answering. Our experimental results reveal that even the most advanced LLMs are susceptible to instructional distraction, often failing to accurately follow user intent in such cases.
Abstract (translated)
尽管大型语言模型（LLM）在遵循指令的任务中表现出色，但当这些模型需要忽略某些指令时，这种优势可能会变成弱点。通常，指令跟随任务包括一个明确的任务描述和包含要处理的目标数据的输入文本。然而，如果输入本身看起来像一条指令，即使有明确的提示来区分任务指令和输入内容，也可能产生混淆。我们将这一现象称为“指令干扰”。在本文中，我们介绍了一个新基准测试DIM-Bench，该基准专门用于评估LLM在面临指令干扰时的表现。该基准将现实世界中的指令干扰实例分类，并对四种指令任务（重写、校对、翻译和风格转换）以及五种输入任务（推理、代码生成、数学推理、偏见检测和问题回答）进行评价。我们的实验结果表明，即使是最先进的LLM也容易受到指令干扰的影响，在这种情况下往往无法准确遵循用户的意图。
URL
https://arxiv.org/abs/2502.04362
