Abstract
Multimodal Large Language Models have achieved significant success in offline video understanding, yet their application to streaming videos is severely limited by the linear explosion of visual tokens, which often leads to Out-of-Memory (OOM) errors or catastrophic forgetting. Existing visual retention and memory management methods typically rely on uniform sampling, low-level physical metrics, or passive cache eviction. However, these strategies often lack intrinsic semantic awareness, potentially disrupting contextual coherence and blurring transient yet critical semantic transitions. To address these limitations, we propose CurveStream, a training-free, curvature-aware hierarchical visual memory management framework. Our approach is motivated by the key observation that high-curvature regions along continuous feature trajectories closely align with critical global semantic transitions. Based on this geometric insight, CurveStream evaluates real-time semantic intensity via a Curvature Score and integrates an online K-Sigma dynamic threshold to adaptively route frames into clear and fuzzy memory states under a strict token budget. Evaluations across diverse temporal scales confirm that this lightweight framework, CurveStream, consistently yields absolute performance gains of over 10% (e.g., 10.69% on StreamingBench and 13.58% on OVOBench) over respective baselines, establishing new state-of-the-art results for streaming video this http URL code will be released at this https URL.
Abstract (translated)
多模态大语言模型在离线视频理解中已取得显著成功，但其在流式视频上的应用却因视觉token的线性爆炸而受到严重限制，这常导致内存溢出(OOM)错误或灾难性遗忘。现有的视觉保留与内存管理方法通常依赖均匀采样、低层物理指标或被动的缓存驱逐策略。然而，这些策略往往缺乏内在的语义感知能力，可能破坏上下文连贯性，并模糊短暂却关键的语义转折。为应对这些局限，我们提出CurveStream——一个无需训练、具备曲率感知的层次化视觉内存管理框架。该方法源于一个关键观察：连续特征轨迹上的高曲率区域与关键的全局语义转折高度吻合。基于这一几何洞察，CurveStream通过曲率评分评估实时语义强度，并整合在线K-Sigma动态阈值，在严格token预算下将帧自适应路由至清晰与模糊内存状态。跨多样时间尺度的评估证实，这一轻量级框架CurveStream相较于各基线模型持续取得超过10%的绝对性能提升（例如在StreamingBench上提升10.69%，在OVOBench上提升13.58%），为流式视频理解建立了新的最先进结果。代码将在此链接发布。
URL
https://arxiv.org/abs/2603.19571
