Abstract
This paper introduces V2A-DPO, a novel Direct Preference Optimization (DPO) framework tailored for flow-based video-to-audio generation (V2A) models, incorporating key adaptations to effectively align generated audio with human preferences. Our approach incorporates three core innovations: (1) AudioScore-a comprehensive human preference-aligned scoring system for assessing semantic consistency, temporal alignment, and perceptual quality of synthesized audio; (2) an automated AudioScore-driven pipeline for generating large-scale preference pair data for DPO optimization; (3) a curriculum learning-empowered DPO optimization strategy specifically tailored for flow-based generative models. Experiments on benchmark VGGSound dataset demonstrate that human-preference aligned Frieren and MMAudio using V2A-DPO outperform their counterparts optimized using Denoising Diffusion Policy Optimization (DDPO) as well as pre-trained baselines. Furthermore, our DPO-optimized MMAudio achieves state-of-the-art performance across multiple metrics, surpassing published V2A models.
Abstract (translated)
本文介绍了V2A-DPO，这是一种新颖的直接偏好优化（DPO）框架，专门针对基于流的视频到音频生成（V2A）模型进行设计，并作出关键调整以有效使生成的音频与人类偏好相一致。我们的方法包含三个核心创新： 1. AudioScore：一种综合的人类偏好对齐评分系统，用于评估合成音频在语义一致性、时间同步性和感知质量方面的表现。 2. 自动化的AudioScore驱动管道，用于为DPO优化生成大规模的偏好配对数据。 3. 一种基于课程学习增强的DPO优化策略，特别针对流式生成模型进行了定制。 在基准VGGSound数据集上的实验表明，使用V2A-DPO的人类偏好对齐Frieren和MMAudio优于使用去噪扩散策略优化（DDPO）以及预训练基线进行优化的同类模型。此外，我们通过DPO优化的MMAudio在多个指标上达到了最先进的性能水平，超越了已发布的视频到音频生成模型。
URL
https://arxiv.org/abs/2603.11089
