Abstract
With recent advancements in neural rendering and motion capture algorithms, remarkable progress has been made in photorealistic human avatar modeling, unlocking immense potential for applications in virtual reality, augmented reality, remote communication, and industries such as gaming, film, and medicine. However, existing methods fail to provide complete, faithful, and expressive control over human avatars due to their entangled representation of facial expressions and body movements. In this work, we introduce Expressive Virtual Avatars (EVA), an actor-specific, fully controllable, and expressive human avatar framework that achieves high-fidelity, lifelike renderings in real time while enabling independent control of facial expressions, body movements, and hand gestures. Specifically, our approach designs the human avatar as a two-layer model: an expressive template geometry layer and a 3D Gaussian appearance layer. First, we present an expressive template tracking algorithm that leverages coarse-to-fine optimization to accurately recover body motions, facial expressions, and non-rigid deformation parameters from multi-view videos. Next, we propose a novel decoupled 3D Gaussian appearance model designed to effectively disentangle body and facial appearance. Unlike unified Gaussian estimation approaches, our method employs two specialized and independent modules to model the body and face separately. Experimental results demonstrate that EVA surpasses state-of-the-art methods in terms of rendering quality and expressiveness, validating its effectiveness in creating full-body avatars. This work represents a significant advancement towards fully drivable digital human models, enabling the creation of lifelike digital avatars that faithfully replicate human geometry and appearance.
Abstract (translated)
随着神经渲染和动作捕捉算法的最新进展，逼真的数字人类模型取得了显著进步，为虚拟现实、增强现实、远程通信以及游戏、电影和医学等行业带来了巨大的潜力。然而，现有的方法由于面部表情和身体运动表示纠缠不清的问题，无法提供对数字人偶完整、忠实且富有表现力的控制。在本工作中，我们引入了具有表现力的虚拟人偶（EVA），这是一种特定于演员的、完全可控的表现力强的人类模型框架，它能够实现实时高保真度、栩栩如生的渲染，并允许独立控制面部表情、身体运动和手部动作。 具体来说，我们的方法将人类模型设计为两层：一个具有表现力的模板几何层和一个3D高斯外观层。首先，我们提出了一种利用粗到细优化技术从多视角视频中准确恢复身体运动、面部表情和非刚性变形参数的表现力模板跟踪算法。接着，我们提出了一个新的解耦3D高斯外观模型，旨在有效区分人体和面部的外观特征。与统一的高斯估计方法不同，我们的方法采用了两个专门且独立的模块来分别建模身体和脸部。 实验结果表明，EVA在渲染质量和表现力方面超越了现有的最先进技术，验证了其创建全身数字人偶的有效性。这项工作代表了向完全可控的数字化人类模型发展的重要一步，能够创造出忠实再现人体几何结构和外观特征的栩栩如生的数字人偶。
URL
https://arxiv.org/abs/2505.15385
