Abstract
Despite recent advancements, video captioning models still face significant limitations in accurately describing fine-grained motion details and suffer from severe hallucination issues. These challenges become particularly prominent when generating captions for motion-centric videos, where precise depiction of intricate movements and limb dynamics is crucial yet often neglected. To alleviate this gap, we introduce an automated annotation pipeline that integrates kinematic-based motion computation with linguistic parsing, enabling detailed decomposition and description of complex human motions. Based on this pipeline, we construct and release the Kinematic Parsing Motion Benchmark (KPM-Bench), a novel open-source dataset designed to facilitate fine-grained motion understanding. KPM-Bench consists of (i) fine-grained video-caption pairs that comprehensively illustrate limb-level dynamics in complex actions, (ii) diverse and challenging question-answer pairs focusing specifically on motion understanding, and (iii) a meticulously curated evaluation set specifically designed to assess hallucination phenomena associated with motion descriptions. Furthermore, to address hallucination issues systematically, we propose the linguistically grounded Motion Parsing and Extraction (MoPE) algorithm, capable of accurately extracting motion-specific attributes directly from textual captions. Leveraging MoPE, we introduce a precise hallucination evaluation metric that functions independently of large-scale vision-language or language-only models. By integrating MoPE into the GRPO post-training framework, we effectively mitigate hallucination problems, significantly improving the reliability of motion-centric video captioning models.
Abstract (translated)
尽管近期取得了进展，视频字幕模型在准确描述精细的运动细节方面仍然面临重大局限，并且存在严重的幻觉问题。这些问题在生成以动作为中心的视频的字幕时尤为突出，在这些情况下，精确描绘复杂的肢体动态至关重要但往往被忽视。为了解决这一差距，我们引入了一种自动注释流水线，该流水线结合了基于运动学的运动计算和语言解析功能，能够详细地分解并描述复杂的人体动作。在此基础上，我们构建并发布了Kinematic Parsing Motion Benchmark（KPM-Bench），这是一个旨在促进精细运动理解的新颖开源数据集。 KPM-Bench 包括： (i) 精细粒度的视频-字幕对，全面展示在复杂的动作中肢体级别的动态变化； (ii) 一系列多样且具有挑战性的问答对，专门针对运动理解设计；以及 (iii) 一套精心策划的评估集，专门用于评估与运动描述相关的幻觉现象。 此外，为了系统地解决幻觉问题，我们提出了一种基于语言基础的Motion Parsing and Extraction（MoPE）算法，能够从文本字幕中准确提取特定于运动的属性。利用 MoPE，我们引入了一个精确的幻觉评估指标，该指标可以独立于大规模视觉-语言或仅语言模型运行。 通过将 MoPE 集成到 GRPO 后训练框架中，我们有效地减轻了幻觉问题，并显著提高了以动作为中心的视频字幕模型的可靠性。
URL
https://arxiv.org/abs/2602.17768
