Abstract
Recent advances in neural portrait animation have demonstrated remarked potential for applications in virtual avatars, telepresence, and digital content creation. However, traditional explicit warping approaches often struggle with accurate motion transfer or recovering missing regions, while recent attention-based warping methods, though effective, frequently suffer from high complexity and weak geometric grounding. To address these issues, we propose SynergyWarpNet, an attention-guided cooperative warping framework designed for high-fidelity talking head synthesis. Given a source portrait, a driving image, and a set of reference images, our model progressively refines the animation in three stages. First, an explicit warping module performs coarse spatial alignment between the source and driving image using 3D dense optical flow. Next, a reference-augmented correction module leverages cross-attention across 3D keypoints and texture features from multiple reference images to semantically complete occluded or distorted regions. Finally, a confidence-guided fusion module integrates the warped outputs with spatially-adaptive fusing, using a learned confidence map to balance structural alignment and visual consistency. Comprehensive evaluations on benchmark datasets demonstrate state-of-the-art performance.
Abstract (translated)
最近在神经肖像动画领域的进展展示了其在虚拟化身、远程呈现和数字内容创作方面的重要潜力。然而，传统的显式扭曲方法通常难以实现精确的动作转移或恢复缺失区域，而基于注意力的扭曲方法虽然有效，但常常面临高复杂度和弱几何基础的问题。为了解决这些问题，我们提出了SynergyWarpNet，这是一种以注意力引导的合作扭曲框架，专门用于高质量的话筒头合成。 给定一个源肖像、一张驱动图像以及一系列参考图像，我们的模型通过三个阶段逐步细化动画效果： 1. **显式扭曲模块**：使用3D密集光流在源图像和驱动图像之间执行粗略的空间对齐。 2. **参考增强校正模块**：利用跨注意力机制，在多个参考图像的3D关键点和纹理特征之间进行交互，从而语义化地完成遮挡或变形区域。 3. **信心引导融合模块**：通过空间自适应融合方法整合扭曲后的输出，并使用学习到的信心图来平衡结构对齐与视觉一致性。 在基准数据集上的全面评估表明了该模型的最先进的性能。
URL
https://arxiv.org/abs/2512.17331
