Abstract
Producing expressive facial animations from static images is a challenging task. Prior methods relying on explicit geometric priors (e.g., facial landmarks or 3DMM) often suffer from artifacts in cross reenactment and struggle to capture subtle emotions. Furthermore, existing approaches lack support for multi-character animation, as driving features from different individuals frequently interfere with one another, complicating the task. To address these challenges, we propose FantasyPortrait, a diffusion transformer based framework capable of generating high-fidelity and emotion-rich animations for both single- and multi-character scenarios. Our method introduces an expression-augmented learning strategy that utilizes implicit representations to capture identity-agnostic facial dynamics, enhancing the model's ability to render fine-grained emotions. For multi-character control, we design a masked cross-attention mechanism that ensures independent yet coordinated expression generation, effectively preventing feature interference. To advance research in this area, we propose the Multi-Expr dataset and ExprBench, which are specifically designed datasets and benchmarks for training and evaluating multi-character portrait animations. Extensive experiments demonstrate that FantasyPortrait significantly outperforms state-of-the-art methods in both quantitative metrics and qualitative evaluations, excelling particularly in challenging cross reenactment and multi-character contexts. Our project page is this https URL.
Abstract (translated)
从静态图像生成富有表现力的面部动画是一项具有挑战性的任务。以往依赖显式几何先验（如面部特征点或3DMM）的方法在跨场景重现时常常产生伪影，并且难以捕捉细微的情感变化。此外，现有的方法缺乏对多角色动画的支持，因为不同个体驱动特征之间的相互干扰会使得任务复杂化。为了应对这些挑战，我们提出了FantasyPortrait，这是一种基于扩散变换器的框架，能够为单角色和多角色场景生成高保真度、情感丰富的动画。我们的方法引入了一种增强表情的学习策略，利用隐式表示来捕捉无关身份的脸部动态变化，从而增强了模型渲染细微情绪的能力。对于多角色控制，我们设计了一种掩码交叉注意力机制，确保各个表达的独立性和协调性，有效防止特征干扰。为了推进该领域的研究，我们提出了Multi-Expr数据集和ExprBench，这是专门为训练和评估多角色肖像动画而特别设计的数据集和基准测试。广泛的实验表明，FantasyPortrait在定量指标和定性评价中均显著优于当前最先进的方法，在具有挑战性的跨场景重现和多角色情景下尤其表现出色。 我们的项目页面在此链接：https://这个URL提供具体信息和资源。
URL
https://arxiv.org/abs/2507.12956
