Abstract
Always-on sensing is essential for next-generation edge/wearable AI systems, yet continuous high-fidelity RGB video capture remains prohibitively expensive for resource-constrained mobile and edge platforms. We present a new paradigm for efficient streaming video understanding: grayscale-always, color-on-demand. Through preliminary studies, we discover that color is not always necessary. Sparse RGB frames suffice for comparable performance when temporal structure is preserved via continuous grayscale streams. Building on this insight, we propose ColorTrigger, an online training-free trigger that selectively activates color capture based on windowed grayscale affinity analysis. Designed for real-time edge deployment, ColorTrigger uses lightweight quadratic programming to detect chromatic redundancy causally, coupled with credit-budgeted control and dynamic token routing to jointly reduce sensing and inference costs. On streaming video understanding benchmarks, ColorTrigger achieves 91.6% of full-color baseline performance while using only 8.1% RGB frames, demonstrating substantial color redundancy in natural videos and enabling practical always-on video sensing on resource-constrained devices.
Abstract (translated)
持续感知对下一代边缘/可穿戴AI系统至关重要，但连续高保真RGB视频采集对资源受限的移动和边缘平台而言成本过高。我们提出了一种高效流式视频理解的新范式：灰度常开，按需彩色。通过初步研究发现，颜色并非始终必要——当通过连续灰度流保持时间结构时，稀疏的RGB帧即可实现相当的性能。基于这一洞察，我们提出了ColorTrigger，一种无需在线训练的触发机制，该机制通过窗口化灰度相似性分析选择性激活彩色采集。专为实时边缘部署设计，ColorTrigger采用轻量级二次规划进行因果色度冗余检测，并结合信用预算控制和动态令牌路由，共同降低感知与推理成本。在流式视频理解基准测试中，ColorTrigger达到全彩色基线性能的91.6%，而仅使用8.1%的RGB帧，证明了自然视频中存在显著的色彩冗余，并为资源受限设备上的实用持续视频感知提供了可能。
URL
https://arxiv.org/abs/2603.22466
