Abstract
We present Kling-Omni, a generalist generative framework designed to synthesize high-fidelity videos directly from multimodal visual language inputs. Adopting an end-to-end perspective, Kling-Omni bridges the functional separation among diverse video generation, editing, and intelligent reasoning tasks, integrating them into a holistic system. Unlike disjointed pipeline approaches, Kling-Omni supports a diverse range of user inputs, including text instructions, reference images, and video contexts, processing them into a unified multimodal representation to deliver cinematic-quality and highly-intelligent video content creation. To support these capabilities, we constructed a comprehensive data system that serves as the foundation for multimodal video creation. The framework is further empowered by efficient large-scale pre-training strategies and infrastructure optimizations for inference. Comprehensive evaluations reveal that Kling-Omni demonstrates exceptional capabilities in in-context generation, reasoning-based editing, and multimodal instruction following. Moving beyond a content creation tool, we believe Kling-Omni is a pivotal advancement toward multimodal world simulators capable of perceiving, reasoning, generating and interacting with the dynamic and complex worlds.
Abstract (translated)
我们介绍了Kling-Omni，这是一个专为从多模态视觉语言输入直接合成高保真视频而设计的通用生成框架。通过端到端的方法视角，Kling-Omni弥合了各种视频生成、编辑和智能推理任务之间的功能分离，并将它们整合成一个整体系统。与独立的工作流程方法不同，Kling-Omni支持多样化的用户输入形式，包括文本指令、参考图像和视频上下文信息，并将其处理为统一的多模态表示，以交付电影级质量和高度智能化的视频内容创作。为了支持这些能力，我们构建了一个全面的数据系统，作为多模态视频创作的基础。此外，该框架通过高效的大规模预训练策略和推理基础设施优化得到了进一步增强。 综合评估表明，Kling-Omni在上下文生成、基于推理的编辑以及多模态指令遵循方面表现出了卓越的能力。超越了单纯的内容创建工具角色，我们认为Kling-Omni是向具备感知、推理、生成及与动态复杂世界互动能力的多模态世界模拟器迈进的重要进步。
URL
https://arxiv.org/abs/2512.16776
