Abstract
In the literature, prior research on Security-oriented Video Understanding (SVU) has predominantly focused on detecting and localize the threats (e.g., shootings, robberies) in videos, while largely lacking the effective capability to generate and evaluate the threat causes. Motivated by these gaps, this paper introduces a new chat paradigm SVU task, i.e., In-depth Security-oriented Video Understanding (DeepSVU), which aims to not only identify and locate the threats but also attribute and evaluate the causes threatening segments. Furthermore, this paper reveals two key challenges in the proposed task: 1) how to effectively model the coarse-to-fine physical-world information (e.g., human behavior, object interactions and background context) to boost the DeepSVU task; and 2) how to adaptively trade off these factors. To tackle these challenges, this paper proposes a new Unified Physical-world Regularized MoE (UPRM) approach. Specifically, UPRM incorporates two key components: the Unified Physical-world Enhanced MoE (UPE) Block and the Physical-world Trade-off Regularizer (PTR), to address the above two challenges, respectively. Extensive experiments conduct on our DeepSVU instructions datasets (i.e., UCF-C instructions and CUVA instructions) demonstrate that UPRM outperforms several advanced Video-LLMs as well as non-VLM approaches. Such this http URL justify the importance of the coarse-to-fine physical-world information in the DeepSVU task and demonstrate the effectiveness of our UPRM in capturing such information.
Abstract (translated)
在相关文献中，面向安全的视频理解（Security-oriented Video Understanding, SVU）的研究主要集中在检测和定位视频中的威胁行为（如枪击、抢劫等），但较少关注生成和评估这些威胁的原因。鉴于这些研究空白，本文提出了一种新的基于对话的SVU任务——深入的安全视频理解（In-depth Security-oriented Video Understanding，DeepSVU）。这一新任务旨在不仅识别并定位威胁，还对威胁原因进行归因和评价。 为了应对这一新任务中的挑战，本文指出两个关键问题： 1. 如何有效地建模从粗到细的物理世界信息（例如人类行为、物体交互以及背景环境），以促进DeepSVU任务。 2. 如何适应性地权衡这些因素之间的关系。 为了解决上述挑战，本文提出了一种新的统一物理世界正则化Mixture-of-Experts (MoE)方法——UPRM。具体来说，UPRM包含了两个核心组件：统一物理世界增强的MoE（Unified Physical-world Enhanced MoE, UPE）模块和物理世界的权衡正则器（Physical-world Trade-off Regularizer, PTR），分别解决上述两大挑战。 在我们构建的DeepSVU指令数据集（即UCF-C指令和CUVA指令）上进行了广泛的实验，结果表明UPRM优于几个先进的视频LLMs以及非VLM方法。这样的实验结果证明了粗到细物理世界信息在DeepSVU任务中的重要性，并展示了我们的UPRM在捕捉这些信息方面的有效性。 请注意，在最后的句子中，“Such this http URL” 应该是一个链接或者引用，但由于原文可能有错误或排版问题，这里并未提供具体网址。如果这是一个有意为之的参考，请根据上下文调整其表述以符合语言习惯和语义连贯性。
URL
https://arxiv.org/abs/2602.18019
