Abstract
Human-robot collaboration (HRC) in the construction industry depends on precise and prompt recognition of human motion intentions and actions by robots to maximize safety and workflow efficiency. There is a research gap in comparing data modalities, specifically signals and videos, for motion intention recognition. To address this, the study leverages deep learning to assess two different modalities in recognizing workers' motion intention at the early stage of movement in drywall installation tasks. The Convolutional Neural Network - Long Short-Term Memory (CNN-LSTM) model utilizing surface electromyography (sEMG) data achieved an accuracy of around 87% with an average time of 0.04 seconds to perform prediction on a sample input. Meanwhile, the pre-trained Video Swin Transformer combined with transfer learning harnessed video sequences as input to recognize motion intention and attained an accuracy of 94% but with a longer average time of 0.15 seconds for a similar prediction. This study emphasizes the unique strengths and trade-offs of both data formats, directing their systematic deployments to enhance HRC in real-world construction projects.
Abstract (translated)
人类与机器人协作（HRC）在建筑行业中依赖于机器人对人类运动意图和动作的精确且及时识别，以最大化安全性和工作流程效率。目前的研究中存在一个缺口，即比较不同数据模态（特别是信号和视频）对于运动意图识别的效果。为了解决这一问题，研究利用深度学习技术评估了两种不同的模态在干墙安装任务早期阶段工人运动意图识别中的表现。 使用表面肌电信号(sEMG)数据的卷积神经网络-长短时记忆(CNN-LSTM)模型实现了约87%的准确率，并且平均只需要0.04秒的时间来预测单一样本输入。另一方面，预训练的视频Swin Transformer结合了迁移学习方法，利用视频序列作为输入识别运动意图，达到了94%的准确率，但其平均处理时间较长，为0.15秒。 这项研究强调了两种数据格式的独特优势和权衡，并指出了它们在实际建筑项目中系统部署的方向，以进一步提升人类与机器人协作的效果。
URL
https://arxiv.org/abs/2509.07990
