Abstract
Recent research highlights concerns about the trustworthiness of third-party Pre-Trained Language Models (PTLMs) due to potential backdoor attacks. These backdoored PTLMs, however, are effective only for specific pre-defined downstream tasks. In reality, these PTLMs can be adapted to many other unrelated downstream tasks. Such adaptation may lead to unforeseen consequences in downstream model outputs, consequently raising user suspicion and compromising attack stealthiness. We refer to this phenomenon as backdoor complications. In this paper, we undertake the first comprehensive quantification of backdoor complications. Through extensive experiments using 4 prominent PTLMs and 16 text classification benchmark datasets, we demonstrate the widespread presence of backdoor complications in downstream models fine-tuned from backdoored PTLMs. The output distribution of triggered samples significantly deviates from that of clean samples. Consequently, we propose a backdoor complication reduction method leveraging multi-task learning to mitigate complications without prior knowledge of downstream tasks. The experimental results demonstrate that our proposed method can effectively reduce complications while maintaining the efficacy and consistency of backdoor attacks. Our code is available at this https URL.
Abstract (translated)
最近的研究指出，第三方预训练语言模型（PTLM）的可信度因潜在的后门攻击而受到关注。然而，这些被植入了后门的PTLM仅在特定的预先定义的下游任务中有效。实际上，这些PTLM可以适应许多其他不相关的下游任务。这种适应可能导致下游模型输出出现意想不到的结果，进而引起用户的怀疑并破坏攻击的隐蔽性。我们将这一现象称为“后门复杂化”。在这篇论文中，我们首次全面量化了后门复杂化的程度。通过使用4个著名的PTLM和16个文本分类基准数据集进行广泛的实验，我们展示了在从被植入后门的PTLM微调而来的下游模型中普遍存在后门复杂化现象。触发样本的输出分布与干净样本显著不同。 因此，我们提出了一种利用多任务学习的方法来减少后门复杂化，并且不需要预先了解下游任务的知识即可实现这一目标。实验结果表明，我们的方法能够有效减少复杂性的同时保持后门攻击的有效性和一致性。我们的代码可在[此处](https://example.com)获取（注：实际链接请替换为论文中提供的真实URL）。
URL
https://arxiv.org/abs/2505.11586
