Abstract
Video Large Language Models (VideoLLMs) have recently achieved strong performance in video understanding tasks. However, we identify a previously underexplored generation failure: severe output repetition, where models degenerate into self-reinforcing loops of repeated phrases or sentences. This failure mode is not captured by existing VideoLLM benchmarks, which focus primarily on task accuracy and factual correctness. We introduce VideoSTF, the first framework for systematically measuring and stress-testing output repetition in VideoLLMs. VideoSTF formalizes repetition using three complementary n-gram-based metrics and provides a standardized testbed of 10,000 diverse videos together with a library of controlled temporal transformations. Using VideoSTF, we conduct pervasive testing, temporal stress testing, and adversarial exploitation across 10 advanced VideoLLMs. We find that output repetition is widespread and, critically, highly sensitive to temporal perturbations of video inputs. Moreover, we show that simple temporal transformations can efficiently induce repetitive degeneration in a black-box setting, exposing output repetition as an exploitable security vulnerability. Our results reveal output repetition as a fundamental stability issue in modern VideoLLMs and motivate stability-aware evaluation for video-language systems. Our evaluation code and scripts are available at: this https URL.
Abstract (translated)
最近，视频大型语言模型（Video Large Language Models, VideoLLM）在视频理解任务中表现出色。然而，我们发现了一个此前未被充分探索的生成失败模式：严重的输出重复现象，即模型陷入自我强化的循环，不断重复相同的短语或句子。现有的VideoLLM基准测试主要关注任务准确性和事实正确性，并没有捕捉到这种失效模式。为此，我们提出了VideoSTF（Video Stability Test Framework），这是第一个系统地衡量和压力测试视频生成中输出重复现象的框架。VideoSTF使用三种互补的n-gram基元度量方法来正式定义重复问题，并提供了一个包含10,000个多样化视频的标准测试平台，以及一个控制时间转换的库。 通过利用VideoSTF，我们对十个高级VideoLLM进行了广泛的测试、时序压力测试和对抗性利用。我们的研究发现，输出重复现象普遍存在，并且关键地，它对于输入视频的时间扰动非常敏感。此外，我们还展示了在黑盒设置中，简单的时间转换可以高效地诱导模型出现重复性的退化行为，揭示了这种输出重复模式作为可被利用的安全漏洞。 这些结果表明，输出重复是现代VideoLLM中的一个根本性稳定性问题，并激发了视频-语言系统中以稳定性能为导向的评估需求。我们的评测代码和脚本可在以下链接获取：this https URL.
URL
https://arxiv.org/abs/2602.10639
