Abstract
Effective communication is paramount for the inclusion of deaf individuals in society. However, persistent communication barriers due to limited Sign Language (SL) knowledge hinder their full participation. In this context, Sign Language Recognition (SLR) systems have been developed to improve communication between signing and non-signing individuals. In particular, there is the problem of recognizing isolated signs (Isolated Sign Language Recognition, ISLR) of great relevance in the development of vision-based SL search engines, learning tools, and translation systems. This work proposes an ISLR approach where body, hands, and facial landmarks are extracted throughout time and encoded as 2-D images. These images are processed by a convolutional neural network, which maps the visual-temporal information into a sign label. Experimental results demonstrate that our method surpassed the state-of-the-art in terms of performance metrics on two widely recognized datasets in Brazilian Sign Language (LIBRAS), the primary focus of this study. In addition to being more accurate, our method is more time-efficient and easier to train due to its reliance on a simpler network architecture and solely RGB data as input.
Abstract (translated)
有效的沟通对于确保聋人融入社会至关重要。然而，由于缺乏手语（SL）知识而导致的持续沟通障碍，阻碍了聋人充分参与。在这种情况下，签名语言识别（SLR）系统已经开发出来，以改善手语和 non-signer 之间的沟通。特别是，在视觉导向 SL 搜索引擎、学习工具和翻译系统的发展中，存在识别孤立手语（Isolated Sign Language Recognition，ISLR）的重大问题。本文提出了一种 ISLR方法，通过提取人体的身体、手部、面部特征并编码为二维图像，在整个过程中进行编码。这些图像通过卷积神经网络进行处理，将视觉-时间信息映射到手语标签。实验结果表明，在我们的方法在巴西手语（LIBRAS）两个广泛认可的数据集上的性能指标方面超过了最先进的水平，这是本研究的主要目标。除了更加准确外，由于其依赖较简单的网络架构和对 RGB 数据输入的唯一性，我们的方法还具有更高效和易训练的特点。
URL
https://arxiv.org/abs/2404.19148
