Abstract
Video-language models (VLMs) are reshaping video querying services, bringing unified solutions to complex perception and reasoning tasks. However, deploying large VLMs in real-world systems remains challenging due to their high resource demands, and remote-based deployment often results in unacceptable response delays. Although small, locally deployable VLMs offer faster responses, they unavoidably fall short in accuracy. To reconcile this trade-off, we propose QuickGrasp, a responsive, quality of service (QoS)-aware system that bridges this gap through a local-first architecture with on-demand edge augmentation. Built upon the highly modular architecture of VLMs, QuickGrasp shares the vision representation across model variants to avoid redundant computation. To maximize system-wide efficiency, QuickGrasp introduces three key designs: accelerated video tokenization, query-adaptive edge augmentation, and delay-aware, accuracy-preserving vision token density configuration. We implement a prototype of QuickGrasp and evaluate it across multiple video understanding benchmarks. The results show that QuickGrasp matches the accuracy of large VLMs while achieving up to a 12.8x reduction in response delay. QuickGrasp represents a key advancement toward building responsive video querying services for open-world understanding that fully leverage the capabilities of VLMs.
Abstract (translated)
视频语言模型（VLM）正在重塑视频查询服务，为复杂的感知和推理任务提供了统一的解决方案。然而，由于大型VLM对资源需求较高，在实际系统中部署它们仍然面临挑战，并且基于远程部署通常会导致不可接受的响应延迟。虽然小型、本地可部署的VLM可以提供更快的响应速度，但它们不可避免地在准确性方面表现较差。为了调和这一权衡，我们提出了QuickGrasp，这是一种响应迅速的服务质量（QoS）感知系统，通过一种以本地为主导架构并按需边缘增强的方式来弥合这一差距。基于VLM高度模块化的架构，QuickGrasp共享不同模型变体间的视觉表示，避免了重复计算。为了最大化整个系统的效率，QuickGrasp引入了三个关键设计：加速视频标记化、查询自适应边缘增强以及延迟感知和准确性保持的视觉令牌密度配置。 我们实施了一个QuickGrasp原型，并在多个视频理解基准上对其进行了评估。结果显示，QuickGrasp能够在与大型VLM匹敌准确度的同时，将响应延迟最多降低12.8倍。QuickGrasp代表了构建响应迅速的视频查询服务的关键进展，这些服务能够充分运用VLM的能力来应对开放世界理解中的挑战。
URL
https://arxiv.org/abs/2603.00126
