Abstract
Recent advancements in diffusion models have significantly improved the realism and generalizability of character-driven animation, enabling the synthesis of high-quality motion from just a single RGB image and a set of driving poses. Nevertheless, generating temporally coherent long-form content remains challenging. Existing approaches are constrained by computational and memory limitations, as they are typically trained on short video segments, thus performing effectively only over limited frame lengths and hindering their potential for extended coherent generation. To address these constraints, we propose TalkingPose, a novel diffusion-based framework specifically designed for producing long-form, temporally consistent human upper-body animations. TalkingPose leverages driving frames to precisely capture expressive facial and hand movements, transferring these seamlessly to a target actor through a stable diffusion backbone. To ensure continuous motion and enhance temporal coherence, we introduce a feedback-driven mechanism built upon image-based diffusion models. Notably, this mechanism does not incur additional computational costs or require secondary training stages, enabling the generation of animations with unlimited duration. Additionally, we introduce a comprehensive, large-scale dataset to serve as a new benchmark for human upper-body animation.
Abstract (translated)
最近在扩散模型方面的进展显著提升了以角色驱动的动画的真实感和泛化能力，使得仅通过一张RGB图像和一组驱动姿势即可合成高质量的动作成为可能。然而，生成具有时间连贯性的长篇内容仍然颇具挑战性。现有方法受到计算资源和内存限制的影响，因为它们通常在短视频片段上进行训练，因此只能在有限的帧长度内有效运作，并且阻碍了其长时间连贯生成的能力。为了解决这些限制，我们提出了TalkingPose，这是一种专门为生产长期、时间一致的人类上半身动画而设计的新扩散框架。 TalkingPose利用驱动帧精确捕捉表情丰富的面部和手部动作，并通过一个稳定的扩散骨干网络将其无缝转移至目标演员身上。为了确保连续运动并增强时间一致性，我们在基于图像的扩散模型之上引入了一种反馈驱动机制。值得注意的是，这一机制不会增加额外的计算成本或需要二次训练阶段，从而能够生成无限时长的动画。 此外，我们还引入了一个全面且大规模的数据集，作为人类上半身动画的新基准标准。
URL
https://arxiv.org/abs/2512.00909
