Abstract
Robust scene segmentation and keyframe extraction are essential preprocessing steps in video understanding pipelines, supporting tasks such as indexing, summarization, and semantic retrieval. However, existing methods often lack generalizability across diverse video types and durations. We present a unified, adaptive framework for automatic scene detection and keyframe selection that handles formats ranging from short-form media to long-form films, archival content, and surveillance footage. Our system dynamically selects segmentation policies based on video length: adaptive thresholding for short videos, hybrid strategies for mid-length ones, and interval-based splitting for extended recordings. This ensures consistent granularity and efficient processing across domains. For keyframe selection, we employ a lightweight module that scores sampled frames using a composite metric of sharpness, luminance, and temporal spread, avoiding complex saliency models while ensuring visual relevance. Designed for high-throughput workflows, the system is deployed in a commercial video analysis platform and has processed content from media, education, research, and security domains. It offers a scalable and interpretable solution suitable for downstream applications such as UI previews, embedding pipelines, and content filtering. We discuss practical implementation details and outline future enhancements, including audio-aware segmentation and reinforcement-learned frame scoring.
Abstract (translated)
鲁棒的场景分割和关键帧提取是视频理解流程中的重要预处理步骤，支持诸如索引、摘要生成和语义检索等任务。然而，现有的方法往往在不同类型的视频（从短视频到长片电影、档案内容以及监控录像）上缺乏泛化能力。我们提出了一种统一的自适应框架，用于自动场景检测和关键帧选择，能够处理各种格式的视频。我们的系统根据视频长度动态选择分割策略：对于短视频采用自适应阈值法；中等长度的视频则使用混合策略；而对于长时间记录，则采用基于间隔的分隔方法。这确保了跨领域的粒度一致性和高效处理。 在关键帧选取方面，我们采用了一个轻量级模块，通过一个综合指标（包括锐度、亮度和时间跨度）来评分抽取的帧，并避免复杂的显著性模型的同时保证视觉相关性。为高吞吐量工作流程设计的该系统已在商用视频分析平台上部署并处理了来自媒体、教育、研究和安全领域的大量内容。它提供了一个可扩展且易于解释的解决方案，适用于下游应用如用户界面预览、嵌入式管道以及内容过滤。 文中详细讨论了实际实施细节，并概述了未来增强方向，包括音频感知分割和强化学习帧评分等技术。
URL
https://arxiv.org/abs/2506.00667
