Abstract
In smart manufacturing environments, accurate and real-time recognition of worker actions is essential for productivity, safety, and human-machine collaboration. While skeleton-based human activity recognition (HAR) offers robustness to lighting, viewpoint, and background variations, most existing approaches rely on centralized datasets, which are impractical in privacy-sensitive industrial scenarios. This paper presents a federated learning (FL) framework for pose-based HAR using a custom skeletal dataset of eight industrially relevant upper-body gestures, captured from five participants and processed using a modified FastPose model. Two temporal backbones, an LSTM and a Transformer encoder, are trained and evaluated under four paradigms: centralized, local (per-client), FL with weighted federated averaging (FedAvg), and federated ensemble learning (FedEnsemble). On the global test set, the FL Transformer improves over centralized training by +12.4 percentage points, with FedEnsemble delivering a +16.3 percentage points gain. On an unseen external client, FL and FedEnsemble exceed centralized accuracy by +52.6 and +58.3 percentage points, respectively. These results demonstrate that FL not only preserves privacy but also substantially enhances cross-user generalization, establishing it as a practical solution for scalable, privacy-aware HAR in heterogeneous industrial settings.
Abstract (translated)
在智能制造环境中，准确且实时地识别工人的动作对于提高生产效率、保障安全以及促进人机协作至关重要。尽管基于骨架的人体活动识别（HAR）方法能够应对光照变化、视角变化和背景干扰等挑战，但大多数现有的方法都依赖于集中化的数据集，在注重隐私保护的工业场景中难以实施。本文提出了一种用于姿态识别的联邦学习（FL）框架，并使用了一个定制的骨骼数据集，该数据集包含了八种与工业相关的上半身手势动作，这些动作由五位参与者进行捕捉并利用修改后的FastPose模型进行处理。文中采用了两种时间序列骨干网络结构——LSTM和Transformer编码器，在四种不同的训练范式下进行了训练和评估：集中化训练、本地（每个客户端）训练、带有加权联邦平均算法的联邦学习（FedAvg），以及联邦集成学习（FedEnsemble）。在全局测试集上，FL Transformer相比集中化训练提高了12.4个百分点，而FedEnsemble则带来了额外提升，提高了16.3个百分点。对于未见过的外部客户端，在精度方面，FL和FedEnsemble分别比集中化方法高出52.6和58.3个百分点。这些结果表明联邦学习不仅能够保护隐私信息，还显著提升了跨用户泛化能力，从而确立了其作为异构工业环境中可扩展、隐私保护型HAR解决方案的实用性。
URL
https://arxiv.org/abs/2508.14113
