Paper Reading AI Learner

VDCook:DIY video data cook your MLLMs

2026-03-04 10:39:04
Chengwei Wu

Abstract

We introduce VDCook: a self-evolving video data operating system, a configurable video data construction platform for researchers and vertical domain teams. Users initiate data requests via natural language queries and adjustable parameters (scale, retrieval-synthesis ratio, quality threshold). The system automatically performs query optimization, concurrently running real video retrieval and controlled synthesis modules. It ultimately generates in-domain data packages with complete provenance and metadata, along with reproducible Notebooks. Unlike traditional static, one-time-built datasets, VDCook enables continuous updates and domain expansion through its automated data ingestion mechanism based on MCP (Model Context Protocol)\cite{mcp2024anthropic}, transforming datasets into dynamically evolving open ecosystems. The system also provides multi-dimensional metadata annotation (scene segmentation, motion scoring, OCR ratio, automatic captioning, etc.), laying the foundation for flexible subsequent data `cooking' and indexing\cite{vlogger}. This platform aims to significantly lower the barrier to constructing specialized video training datasets through infrastructure-level solutions, while supporting community contributions and a governance-enabled data expansion paradigm. \textbf{Project demo:} this https URL

Abstract (translated)

我们介绍VDCook:一个自我演进的视频数据操作系统,以及为研究人员和垂直领域团队提供的可配置视频数据构建平台。用户可以通过自然语言查询和可调参数(规模、检索-合成比例、质量阈值)发起数据请求。系统自动进行查询优化,并同时运行真实视频检索与受控合成模块。最终生成具有完整来源信息和元数据的领域内数据包,以及可重现的笔记本文件。 不同于传统的静态一次性构建的数据集,VDCook通过基于MCP(模型上下文协议)\cite{mcp2024anthropic}的自动化数据摄取机制,实现了持续更新及领域的扩展,将数据集转化为动态演进的开放生态系统。此外,该系统还提供了多维度元数据标注功能(场景分割、运动评分、OCR比率、自动字幕生成等),为灵活的数据处理和索引奠定了基础\cite{vlogger}。此平台旨在通过基础设施级解决方案大幅降低构建专业视频训练数据集的技术门槛,并支持社区贡献及治理驱动的数据扩展模式。 **项目演示地址:** [this https URL]

URL

https://arxiv.org/abs/2603.05539

PDF

https://arxiv.org/pdf/2603.05539.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot