Abstract
This paper introduces Motion Personalization, a new task that generates personalized motions aligned with text descriptions using several basic motions containing Persona. To support this novel task, we introduce a new large-scale motion dataset called PerMo (PersonaMotion), which captures the unique personas of multiple actors. We also propose a multi-modal finetuning method of a pretrained motion diffusion model called PersonaBooth. PersonaBooth addresses two main challenges: i) A significant distribution gap between the persona-focused PerMo dataset and the pretraining datasets, which lack persona-specific data, and ii) the difficulty of capturing a consistent persona from the motions vary in content (action type). To tackle the dataset distribution gap, we introduce a persona token to accept new persona features and perform multi-modal adaptation for both text and visuals during finetuning. To capture a consistent persona, we incorporate a contrastive learning technique to enhance intra-cohesion among samples with the same persona. Furthermore, we introduce a context-aware fusion mechanism to maximize the integration of persona cues from multiple input motions. PersonaBooth outperforms state-of-the-art motion style transfer methods, establishing a new benchmark for motion personalization.
Abstract (translated)
这篇论文介绍了Motion Personalization，这是一个新任务，旨在生成与文本描述相匹配的个性化动作序列，利用包含Persona的基本动作。为支持这一新颖的任务，我们引入了一个新的大规模动作数据集，称为PerMo（PersonaMotion），该数据集捕捉了多个演员的独特人格特征。此外，我们提出了一种基于预训练的动作扩散模型进行多模态微调的方法，名为PersonaBooth。 PersonaBooth解决了两个主要挑战：i) 专注于人物的PerMo数据集与缺乏特定人物信息的数据集之间的显著分布差距；ii) 从内容（动作类型）各异的动作中捕捉一致的人物特征的困难。为解决数据集分布差异问题，我们引入了一个接受新人物特性的角色令牌，并在微调期间对文本和视觉进行多模态适应。为了捕捉一致的人格特征，我们结合了对比学习技术，以增强具有相同人格特征样本之间的内部一致性。此外，我们还介绍了一种上下文感知融合机制，以最大化从多个输入动作中整合人物线索。 PersonaBooth在动作风格转移方法上超越了现有最佳水平，并为动作个性化建立了一个新的基准标准。
URL
https://arxiv.org/abs/2503.07390
