Abstract
In practical applications, computer vision tasks often need to be addressed simultaneously. Multitask learning typically achieves this by jointly training a single deep neural network to learn shared representations, providing efficiency and improving generalization. Although action and gesture recognition are closely related tasks, since they focus on body and hand movements, current state-of-the-art methods handle them separately. In this paper, we show that employing a multi-task learning paradigm for action and gesture recognition results in more efficient, robust and generalizable visual representations, by leveraging the synergies between these tasks. Extensive experiments on multiple action and gesture datasets demonstrate that handling actions and gestures in a single architecture can achieve better performance for both tasks in comparison to their single-task learning variants.
Abstract (translated)
在实际应用中，计算机视觉任务通常需要同时处理。多任务学习通过联合训练单一的深度神经网络来学习共享表示，从而提高了效率并改进了泛化能力。尽管动作识别和手势识别是紧密相关的任务（它们都关注身体和手部的动作），当前最先进的方法仍倾向于分别处理这些任务。在本文中，我们展示了使用一个多任务学习框架来进行动作和手势识别可以产生更有效、更具鲁棒性和通用性的视觉表示，通过利用这些任务之间的协同效应。在多个动作和手势数据集上进行的广泛实验表明，在单一架构下同时处理动作和手势相比各自的单任务学习变体能够实现更好的性能。
URL
https://arxiv.org/abs/2505.17867
