Abstract
This paper introduces our solution for Track 2 in AI City Challenge 2024. The task aims to solve traffic safety description and analysis with the dataset of Woven Traffic Safety (WTS), a real-world Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding. Our solution mainly focuses on the following points: 1) To solve dense video captioning, we leverage the framework of dense video captioning with parallel decoding (PDVC) to model visual-language sequences and generate dense caption by chapters for video. 2) Our work leverages CLIP to extract visual features to more efficiently perform cross-modality training between visual and textual representations. 3) We conduct domain-specific model adaptation to mitigate domain shift problem that poses recognition challenge in video understanding. 4) Moreover, we leverage BDD-5K captioned videos to conduct knowledge transfer for better understanding WTS videos and more accurate captioning. Our solution has yielded on the test set, achieving 6th place in the competition. The open source code will be available at this https URL
Abstract (translated)
本文介绍了我们为2024年AI城市挑战赛第二 track 2 提出的解决方案。该任务旨在利用Woven Traffic Safety（WTS）数据集中的数据解决交通安全的描述和分析。我们的解决方案主要关注以下几点：1）为了解决视频摘要问题，我们利用了密集视频摘要与并行解码（PDVC）框架来建模视觉-语言序列并生成视频的章节摘要。2）我们的工作利用了CLIP来提取视觉特征，以更有效地在视觉和文本表示之间进行跨模态训练。3）我们进行了领域特定模型适应，以减轻在视频理解中出现的领域漂移问题。4）此外，我们利用BDD-5K捕获的视频进行知识传递，以更好地理解WTS视频并获得更准确的摘要。我们的解决方案在测试集上已经实现了成果，获得了竞争中的第六名。开源代码将在此处 https:// URL 下载。
URL
https://arxiv.org/abs/2404.08229
