Abstract
Sign language is the primary communication language for people with disabling hearing loss. Sign language recognition (SLR) systems aim to recognize sign gestures and translate them into spoken language. One of the main challenges in SLR is the scarcity of annotated datasets. To address this issue, we propose a semi-supervised learning (SSL) approach for SLR (SSLR), employing a pseudo-label method to annotate unlabeled samples. The sign gestures are represented using pose information that encodes the signer's skeletal joint points. This information is used as input for the Transformer backbone model utilized in the proposed approach. To demonstrate the learning capabilities of SSL across various labeled data sizes, several experiments were conducted using different percentages of labeled data with varying numbers of classes. The performance of the SSL approach was compared with a fully supervised learning-based model on the WLASL-100 dataset. The obtained results of the SSL model outperformed the supervised learning-based model with less labeled data in many cases.
Abstract (translated)
手语是听觉障碍人士的主要交流语言。手语识别（SLR）系统旨在识别手势并将其翻译成口语。在SLR中，主要挑战之一是标注数据的稀缺性。为了解决这个问题，我们提出了一种针对SLR的半监督学习（SSL）方法（SSLR），采用伪标签技术来注释未标记的数据样本。手语通过包含表演者骨骼关节点信息的姿态表示进行编码，并将其作为Transformer骨干模型输入。为了展示不同标注数据量下SSL的学习能力，使用WLASL-100数据集进行了多项实验，在这些实验中采用了具有不同类别数量的各种比例的标注数据。将所提出的半监督学习方法与基于完全有监督学习的方法在相同数据集上进行对比。结果显示，在许多情况下，当可用标记数据较少时，SSLR模型的表现优于基于监督学习的模型。
URL
https://arxiv.org/abs/2504.16640
