Abstract
This paper investigates the performance of transformer-based architectures for person identification in natural, face-to-face conversation scenario. We implement and evaluate a two-stream framework that separately models spatial configurations and temporal motion patterns of 133 COCO WholeBody keypoints, extracted from a subset of the CANDOR conversational corpus. Our experiments compare pre-trained and from-scratch training, investigate the use of velocity features, and introduce a multi-scale temporal transformer for hierarchical motion modeling. Results demonstrate that domain-specific training significantly outperforms transfer learning, and that spatial configurations carry more discriminative information than temporal dynamics. The spatial transformer achieves 95.74% accuracy, while the multi-scale temporal transformer achieves 93.90%. Feature-level fusion pushes performance to 98.03%, confirming that postural and dynamic information are complementary. These findings highlight the potential of transformer architectures for person identification in natural interactions and provide insights for future multimodal and cross-cultural studies.
Abstract (translated)
本文研究了基于Transformer架构的人识别性能，特别是在自然的面对面对话场景中。我们实现并评估了一个两流框架，该框架分别对从CANDOR对话语料库的一个子集中提取的133个COCO WholeBody关键点的空间配置和时间运动模式进行建模。我们的实验比较了预训练和从头开始训练的效果，探讨了使用速度特征的可能性，并引入了一种多尺度时间Transformer来进行层次化运动建模。实验结果表明，特定领域的训练显著优于迁移学习，并且空间配置比时间动态包含更多的判别性信息。在空间Transformer中实现了95.74%的准确率，而多尺度时间Transformer则达到了93.90%。特征级融合将性能提升至98.03%，证实了姿势和动态信息是互补的。这些发现突显了基于Transformer架构的人识别在自然互动中的潜力，并为未来跨模态和跨文化研究提供了见解。
URL
https://arxiv.org/abs/2510.04753
