Abstract
This study explores two frameworks for co-speech gesture generation, AQ-GT and its semantically-augmented variant AQ-GT-a, to evaluate their ability to convey meaning through gestures and how humans perceive the resulting movements. Using sentences from the SAGA spatial communication corpus, contextually similar sentences, and novel movement-focused sentences, we conducted a user-centered evaluation of concept recognition and human-likeness. Results revealed a nuanced relationship between semantic annotations and performance. The original AQ-GT framework, lacking explicit semantic input, was surprisingly more effective at conveying concepts within its training domain. Conversely, the AQ-GT-a framework demonstrated better generalization, particularly for representing shape and size in novel contexts. While participants rated gestures from AQ-GT-a as more expressive and helpful, they did not perceive them as more human-like. These findings suggest that explicit semantic enrichment does not guarantee improved gesture generation and that its effectiveness is highly dependent on the context, indicating a potential trade-off between specialization and generalization.
Abstract (translated)
这项研究探讨了两种共话语势生成框架，AQ-GT及其语义增强变体AQ-GT-a，以评估它们通过手势传达意义的能力以及人类如何感知这些动作。利用SAGA空间交流语料库中的句子、上下文相似的句子和新型运动焦点句子，我们进行了一项以人为中心的概念识别和人性化程度评估。结果揭示了语义标注与性能之间存在复杂的关系。 原始AQ-GT框架由于缺乏显式语义输入，在其训练领域内传达概念的效果反而更佳。相反，AQ-GT-a框架在表示形状和大小的新上下文中表现出更好的泛化能力。尽管参与者认为来自AQ-GT-a的手势更具表现力且更有帮助，但他们并没有觉得这些手势更加人性化。 这些发现表明，显式的语义丰富并不保证能提高手势生成的效果，并且其效果高度依赖于具体的情境，暗示了专业化与泛化之间可能存在权衡。
URL
https://arxiv.org/abs/2510.17599
