Abstract
We present daVinci-MagiHuman, an open-source audio-video generative foundation model for human-centric generation. daVinci-MagiHuman jointly generates synchronized video and audio using a single-stream Transformer that processes text, video, and audio within a unified token sequence via self-attention only. This single-stream design avoids the complexity of multi-stream or cross-attention architectures while remaining easy to optimize with standard training and inference infrastructure. The model is particularly strong in human-centric scenarios, producing expressive facial performance, natural speech-expression coordination, realistic body motion, and precise audio-video synchronization. It supports multilingual spoken generation across Chinese (Mandarin and Cantonese), English, Japanese, Korean, German, and French. For efficient inference, we combine the single-stream backbone with model distillation, latent-space super-resolution, and a Turbo VAE decoder, enabling generation of a 5-second 256p video in 2 seconds on a single H100 GPU. In automatic evaluation, daVinci-MagiHuman achieves the highest visual quality and text alignment among leading open models, along with the lowest word error rate (14.60%) for speech intelligibility. In pairwise human evaluation, it achieves win rates of 80.0% against Ovi 1.1 and 60.9% against LTX 2.3 over 2000 comparisons. We open-source the complete model stack, including the base model, the distilled model, the super-resolution model, and the inference codebase.
Abstract (translated)
我们推出daVinci-MagiHuman，这是一个面向人类中心生成的开源音视频生成基础模型。daVinci-MagiHuman采用仅通过自注意力机制在统一标记序列中处理文本、视频和音频的单流Transformer架构，联合生成同步的视频与音频。这种单流设计避免了多流或交叉注意力架构的复杂性，同时易于使用标准训练和推理基础设施进行优化。该模型在人类中心场景中表现尤为突出，能够生成富有表现力的面部表演、自然的语音-表情协调、逼真的身体动作以及精确的音视频同步。它支持中文（普通话和粤语）、英语、日语、韩语、德语和法语的多语言口语生成。为实现高效推理，我们将单流主干网络与模型蒸馏、潜在空间超分辨率以及Turbo VAE解码器相结合，可在单张H100 GPU上2秒内生成5秒256p视频。在自动评估中，daVinci-MagiHuman在视觉质量和文本对齐方面领先于其他主流开源模型，并以14.60%的最低词错误率实现语音可懂度最优。在成对人工评估中，基于2000次对比，该模型对Ovi 1.1的胜率达80.0%，对LTX 2.3的胜率达60.9%。我们开源了完整的模型技术栈，包括基础模型、蒸馏模型、超分辨率模型及推理代码库。
URL
https://arxiv.org/abs/2603.21986
