Abstract
Existing for audio- and pose-driven human animation methods often struggle with stiff head movements and blurry hands, primarily due to the weak correlation between audio and head movements and the structural complexity of hands. To address these issues, we propose VividAnimator, an end-to-end framework for generating high-quality, half-body human animations driven by audio and sparse hand pose conditions. Our framework introduces three key innovations. First, to overcome the instability and high cost of online codebook training, we pre-train a Hand Clarity Codebook (HCC) that encodes rich, high-fidelity hand texture priors, significantly mitigating hand degradation. Second, we design a Dual-Stream Audio-Aware Module (DSAA) to model lip synchronization and natural head pose dynamics separately while enabling interaction. Third, we introduce a Pose Calibration Trick (PCT) that refines and aligns pose conditions by relaxing rigid constraints, ensuring smooth and natural gesture transitions. Extensive experiments demonstrate that Vivid Animator achieves state-of-the-art performance, producing videos with superior hand detail, gesture realism, and identity consistency, validated by both quantitative metrics and qualitative evaluations.
Abstract (translated)
现有的音频和姿态驱动的人体动画方法通常在头部僵硬运动和手部模糊方面存在问题，主要是由于音频与头部动作之间的相关性较弱以及手部结构的复杂性。为了解决这些问题，我们提出了VividAnimator，这是一个端到端框架，用于生成高质量、半身人体动画，这些动画由音频和稀疏的手部姿态条件驱动。 我们的框架引入了三个关键创新点： 1. 为了克服在线代码库训练中的不稳定性和高成本问题，我们预训练了一个手清晰度代码库（HCC），它编码了丰富的高保真手部纹理先验知识，大大减少了手部的退化现象。 2. 我们设计了一种双流音频感知模块（DSAA），能够分别建模唇同步和自然头部姿态动态变化，并且使它们可以相互作用。 3. 我们引入了一个姿态校准技巧（PCT），通过放松刚性约束来细化并对齐姿态条件，从而确保手势过渡的平滑性和自然度。 广泛的实验表明，Vivid Animator 达到了最先进的性能水平，在手部细节、动作现实感和身份一致性方面表现出优越的表现，并且这些结果不仅由定量指标验证，还通过定性评估得到了确认。
URL
https://arxiv.org/abs/2510.10269
