Abstract
Although Video Large Language Models (VLLMs) have shown remarkable capabilities in video understanding, they are required to process high volumes of visual tokens, causing significant computational inefficiency. Existing VLLMs acceleration frameworks usually compress spatial and temporal redundancy independently, which overlooks the spatiotemporal relationships, thereby leading to suboptimal spatiotemporal compression. The highly correlated visual features are likely to change in spatial position, scale, orientation, and other attributes over time due to the dynamic nature of video. Building on this insight, we introduce FlashVID, a training-free inference acceleration framework for VLLMs. Specifically, FlashVID utilizes Attention and Diversity-based Token Selection (ADTS) to select the most representative tokens for basic video representation, then applies Tree-based Spatiotemporal Token Merging (TSTM) for fine-grained spatiotemporal redundancy elimination. Extensive experiments conducted on three representative VLLMs across five video understanding benchmarks demonstrate the effectiveness and generalization of our method. Notably, by retaining only 10% of visual tokens, FlashVID preserves 99.1% of the performance of LLaVA-OneVision. Consequently, FlashVID can serve as a training-free and plug-and-play module for extending long video frames, which enables a 10x increase in video frame input to Qwen2.5-VL, resulting in a relative improvement of 8.6% within the same computational budget. Code is available at this https URL.
Abstract (translated)
尽管视频大型语言模型（VLLMs）在视频理解方面表现出色，但它们处理大量视觉令牌的需求导致了显著的计算效率低下。现有的VLLM加速框架通常独立地压缩空间和时间冗余，这忽视了时空关系，从而导致次优的时空压缩效果。由于视频的动态特性，高度相关的视觉特征可能随时间在空间位置、尺度、方向等属性上发生变化。 基于这一见解，我们引入了FlashVID，这是一个用于VLLM推理加速且无需训练框架。具体而言，FlashVID利用注意力和多样性令牌选择（ADTS）来选取代表性的视频基本表示的令牌，然后应用树基时空令牌合并（TSTM）进行细粒度的时空冗余消除。 在五个视频理解基准测试中的三个代表性VLLM上进行的广泛实验表明了我们方法的有效性和泛化能力。值得注意的是，通过保留仅10%的视觉令牌，FlashVID能够保持LLaVA-OneVision 99.1%的表现水平。因此，FlashVID可以作为一个无需训练且即插即用的模块来扩展长视频帧，从而在不增加计算预算的情况下，使Qwen2.5-VL的视频帧输入量增加了10倍，并相对提高了8.6%的表现。 代码可在[此链接](https://this-url.com)获得。
URL
https://arxiv.org/abs/2602.08024
