Abstract
The proliferation of video content production has led to vast amounts of data, posing substantial challenges in terms of analysis efficiency and resource utilization. Addressing this issue calls for the development of robust video analysis tools. This paper proposes a novel approach leveraging Generative Artificial Intelligence (GenAI) to facilitate streamlined video analysis. Our tool aims to deliver tailored textual summaries of user-defined queries, offering a focused insight amidst extensive video datasets. Unlike conventional frameworks that offer generic summaries or limited action recognition, our method harnesses the power of GenAI to distil relevant information, enhancing analysis precision and efficiency. Employing YOLO-V8 for object detection and Gemini for comprehensive video and text analysis, our solution achieves heightened contextual accuracy. By combining YOLO with Gemini, our approach furnishes textual summaries extracted from extensive CCTV footage, enabling users to swiftly navigate and verify pertinent events without the need for exhaustive manual review. The quantitative evaluation revealed a similarity of 72.8%, while the qualitative assessment rated an accuracy of 85%, demonstrating the capability of the proposed method.
Abstract (translated)
视频内容制作的激增导致了大量的数据，这对分析效率和资源利用提出了重大挑战。为解决这一问题，开发强大的视频分析工具至关重要。本文提出了一种新方法，利用生成式人工智能（Generative Artificial Intelligence, GenAI）来促进高效的视频分析流程。我们的工具旨在根据用户的自定义查询提供定制化的文本摘要，在庞大的视频数据集中提供有针对性的洞察。 与传统的框架相比，这些传统框架通常只提供通用总结或有限的动作识别，我们的方法则运用GenAI提取相关的信息，从而提高分析的准确性和效率。通过使用YOLO-V8进行目标检测和Gemini进行全面的视频及文本分析，本解决方案实现了更高的上下文准确性。结合YOLO与Gemini的能力，我们能够从大量的闭路电视（CCTV）录像中抽取文本摘要，使用户能够在无需详细人工审查的情况下快速导航并验证相关事件。 定量评估显示相似度为72.8%，而定性评估则达到了85%的准确率，这展示了所提出方法的有效能力。
URL
https://arxiv.org/abs/2501.04764
