Abstract
Neuro-symbolic approaches to long-form video question answering (LVQA) have demonstrated significant accuracy improvements by grounding temporal reasoning in formal verification. However, existing methods incur prohibitive latency overheads, up to 90x slower than base VLM prompting, rendering them impractical for latency-sensitive edge deployments. We present LE-NeuS, a latency-efficient neuro-symbolic framework that preserves the accuracy benefits of temporal logic-guided video understanding while drastically reducing inference latency. Our key insight is that the dominant computational bottleneck arises from sequential and dense proposition detection across video frames during automaton construction. We address this through two principled optimizations: (1) CLIP guided two-stage adaptive sampling that exploits visual redundancy to skip semantically similar frames while preserving temporal boundaries, and (2) batched proposition detection that parallelizes VLM inference across temporal windows. Theoretically, we derive latency bounds as a function of video length, proposition complexity, and sampling density, establishing conditions under which latency efficiency is achievable. Empirically, on LongVideoBench and Video-MME benchmarks deployed on NVIDIA H100 GPUs, LE-NeuS reduces the latency gap from 90x to approximately 10x while maintaining >10% accuracy gains on temporally complex queries.
Abstract (translated)
神经符号方法在长视频问答（LVQA）中通过将时间推理建立在形式验证基础上，展示了显著的准确性提升。然而，现有的方法由于计算延迟过大而变得不可行，其速度比基础视觉语言模型提示方法慢90倍，这对于对延迟敏感的边缘部署来说是不切实际的。我们提出了LE-NeuS，这是一种低延迟神经符号框架，在保持时间逻辑指导下的视频理解准确性的同时，大大减少了推理延迟。 我们的关键见解在于计算瓶颈主要来自于在构建自动机时对整个视频帧进行顺序和密集的状态检测过程。为此，我们通过两项原理性的优化来解决这一问题：（1）CLIP引导的两阶段自适应采样方法，该方法利用视觉冗余跳过语义相似的帧同时保持时间边界；（2）批量状态检测，将VLM推理并行化处理以跨越多个时间窗口。 理论上，我们推导出了视频长度、状态复杂性和采样密度作为函数影响延迟效率的界限，并建立了实现低延迟计算的有效条件。实验上，在NVIDIA H100 GPU上的LongVideoBench和Video-MME基准测试中，LE-NeuS将延迟差距从90倍减少到大约10倍，同时在处理时间复杂度高的查询时保持超过10%的准确性提升。 这种方法通过优化神经符号模型中的计算瓶颈，使其能够更有效地应用于长视频理解和问答任务，并且对于需要快速响应的应用场景具有重要意义。
URL
https://arxiv.org/abs/2602.23553
