Abstract
Audio-driven human animation has attracted wide attention thanks to its practical applications. However, critical challenges remain in generating high-resolution, long-duration videos with consistent appearance and natural hand motions. Existing methods extend videos using overlapping motion frames but suffer from error accumulation, leading to identity drift, color shifts, and scene instability. Additionally, hand movements are poorly modeled, resulting in noticeable distortions and misalignment with the audio. In this work, we propose InfinityHuman, a coarse-to-fine framework that first generates audio-synchronized representations, then progressively refines them into high-resolution, long-duration videos using a pose-guided refiner. Since pose sequences are decoupled from appearance and resist temporal degradation, our pose-guided refiner employs stable poses and the initial frame as a visual anchor to reduce drift and improve lip synchronization. Moreover, to enhance semantic accuracy and gesture realism, we introduce a hand-specific reward mechanism trained with high-quality hand motion data. Experiments on the EMTD and HDTF datasets show that InfinityHuman achieves state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync. Ablation studies further confirm the effectiveness of each module. Code will be made public.
Abstract (translated)
基于音频的人体动画由于其实用性吸引了广泛的关注。然而，生成高分辨率、长时间且外观一致、手部动作自然的视频仍然面临重大挑战。现有方法通过使用重叠运动帧来延长视频，但会遭受错误积累的问题，导致身份漂移、色彩偏移和场景不稳定。此外，手部动作建模不足也会造成明显的失真与音频不同步现象。为此，我们提出了InfinityHuman框架，这是一种从粗到细的生成模型，首先生成与音频同步的表示，然后逐步细化为高分辨率长时间视频，通过姿势引导的精炼器完成这一过程。 由于姿势序列与外观解耦且能够抵抗时序退化，我们的姿势引导式精炼器利用稳定的姿势和初始帧作为视觉锚点来减少漂移并提高唇部同步效果。为了增强语义准确性及手势真实性，我们引入了一种针对手部动作的特定奖励机制，并通过高质量的手部运动数据进行训练。 在EMTD和HDTF数据集上的实验表明，InfinityHuman在视频质量、身份保持、手部精度以及唇部同步方面均达到了当前最佳性能。消融研究进一步证实了每个模块的有效性。代码将公开发布。
URL
https://arxiv.org/abs/2508.20210
