Abstract
Evaluating the robustness of Video classification models is very challenging, specifically when compared to image-based models. With their increased temporal dimension, there is a significant increase in complexity and computational cost. One of the key challenges is to keep the perturbations to a minimum to induce misclassification. In this work, we propose a multi-agent reinforcement learning approach (spatial and temporal) that cooperatively learns to identify the given video's sensitive spatial and temporal regions. The agents consider temporal coherence in generating fine perturbations, leading to a more effective and visually imperceptible attack. Our method outperforms the state-of-the-art solutions on the Lp metric and the average queries. Our method enables custom distortion types, making the robustness evaluation more relevant to the use case. We extensively evaluate 4 popular models for video action recognition on two popular datasets, HMDB-51 and UCF-101.
Abstract (translated)
评估视频分类模型的鲁棒性非常具有挑战性，尤其是与基于图像的模型相比。由于增加了时间维度，其复杂性和计算成本显著增加。其中一个关键挑战是尽量减少扰动以诱导错误分类。在本工作中，我们提出了一种多智能体强化学习方法（空间和时间），该方法协同学习识别给定视频中敏感的空间和时间区域。这些代理在生成精细的扰动时考虑了时间一致性，从而导致更有效且视觉上难以察觉的攻击。我们的方法在Lp度量和平均查询数量方面优于最先进的解决方案。此外，我们的方法支持自定义失真类型，使鲁棒性评估更加符合应用场景需求。我们对四个流行的视频动作识别模型在两个流行数据集（HMDB-51 和 UCF-101）上进行了广泛评估。
URL
https://arxiv.org/abs/2506.05431
