Abstract
Neural contextual bandits are vulnerable to adversarial attacks, where subtle perturbations to rewards, actions, or contexts induce suboptimal decisions. We introduce AdvBandit, a black-box adaptive attack that formulates context poisoning as a continuous-armed bandit problem, enabling the attacker to jointly learn and exploit the victim's evolving policy. The attacker requires no access to the victim's internal parameters, reward function, or gradient information; instead, it constructs a surrogate model using a maximum-entropy inverse reinforcement learning module from observed context-action pairs and optimizes perturbations against this surrogate using projected gradient descent. An upper confidence bound-aware Gaussian process guides arm selection. An attack-budget control mechanism is also introduced to limit detection risk and overhead. We provide theoretical guarantees, including sublinear attacker regret and lower bounds on victim regret linear in the number of attacks. Experiments on three real-world datasets (Yelp, MovieLens, and Disin) against various victim contextual bandits demonstrate that our attack model achieves higher cumulative victim regret than state-of-the-art baselines.
Abstract (translated)
神经上下文型多臂赌博机容易受到对抗性攻击的影响，这种攻击通过对奖励、动作或背景信息进行细微的扰动来诱导次优决策。我们引入了AdvBandit这一黑盒自适应攻击方法，它将上下文污染问题作为连续行动的多臂赌博机问题进行建模，并使攻击者能够同时学习和利用受害者的不断变化的策略。这种攻击无需访问受害者内部参数、奖励函数或梯度信息；相反，通过使用从观察到的背景动作对中构建的最大熵逆强化学习模块，来创建一个替代模型，并通过对这个替代模型使用投影梯度下降法进行优化以生成扰动。臂选择由上置信边界感知高斯过程指导。还引入了攻击预算控制机制来限制检测风险和开销。我们提供了理论保证，包括攻击者的次线性遗憾以及与攻击数量成正比的受害者线性遗憾下限。在三个真实世界数据集（Yelp、MovieLens 和 Disin）上针对各种受害者的上下文型多臂赌博机进行实验后发现，我们的攻击模型实现了更高的累积受害者遗憾值，超过了最先进的基准方法。
URL
https://arxiv.org/abs/2603.00567
