Abstract
Massive frame redundancy and limited context window make efficient frame selection crucial for long-video understanding with large vision-language models (LVLMs). Prevailing approaches, however, adopt a flat sampling paradigm which treats the video as an unstructured collection of frames. In this paper, we introduce Event-Anchored Frame Selection (EFS), a hierarchical, event-aware pipeline. Leveraging self-supervised DINO embeddings, EFS first partitions the video stream into visually homogeneous temporal segments, which serve as proxies for semantic events. Within each event, it then selects the most query-relevant frame as an anchor. These anchors act as structural priors that guide a global refinement stage using an adaptive Maximal Marginal Relevance (MMR) scheme. This pipeline ensures the final keyframe set jointly optimizes for event coverage, query relevance, and visual diversity. As a training-free, plug-and-play module, EFS can be seamlessly integrated into off-the-shelf LVLMs, yielding substantial gains on challenging video understanding benchmarks. Specifically, when applied to LLaVA-Video-7B, EFS improves accuracy by 4.7%, 4.9%, and 8.8% on VideoMME, LongVideoBench, and MLVU, respectively.
Abstract (translated)
大规模的帧冗余和有限的上下文窗口使得长视频理解中大型视觉语言模型（LVLM）的有效帧选择变得至关重要。然而，现有方法普遍采用扁平化的采样模式，将视频视为无结构的帧集合。在本文中，我们引入了基于事件的锚点帧选择（EFS），这是一种分层、事件感知的流程。利用自监督DINO嵌入，EFS首先将视频流划分为视觉上同质的时间段，这些时间段充当语义事件的代理。然后，在每个事件内部，它会选择与查询最相关的帧作为锚点。这些锚点作为结构先验引导全局细化阶段使用自适应最大边缘相关性（MMR）方案进行操作。这一流程确保最终的关键帧集合在事件覆盖率、查询相关性和视觉多样性方面共同优化。作为一个无需训练的即插即用模块，EFS可以无缝集成到现成的LVLM中，在具有挑战性的视频理解基准测试上取得了显著的改进。具体而言，当应用于LLaVA-Video-7B时，EFS在VideoMME、LongVideoBench和MLVU上的准确率分别提高了4.7%、4.9%和8.8%。
URL
https://arxiv.org/abs/2603.00983
