Abstract
Federated learning (FL) has emerged as a promising paradigm for privacy-preserving multi-camera video understanding. However, applying FL to cross-view scenarios faces three major challenges: (i) heterogeneous viewpoints and backgrounds lead to highly non-IID client distributions and overfitting to view-specific patterns, (ii) local distribution biases cause misaligned representations that hinder consistent cross-view semantics, and (iii) large video architectures incur prohibitive communication overhead. To address these issues, we propose FedCVU, a federated framework with three components: VS-Norm, which preserves normalization parameters to handle view-specific statistics; CV-Align, a lightweight contrastive regularization module to improve cross-view representation alignment; and SLA, a selective layer aggregation strategy that reduces communication without sacrificing accuracy. Extensive experiments on action understanding and person re-identification tasks under a cross-view protocol demonstrate that FedCVU consistently boosts unseen-view accuracy while maintaining strong seen-view performance, outperforming state-of-the-art FL baselines and showing robustness to domain heterogeneity and communication constraints.
Abstract (translated)
联邦学习（FL）作为一种有前景的隐私保护多摄像头视频理解范式已受到广泛关注。然而，将FL应用于跨视角场景面临三大挑战：（i）异构视角与背景导致客户端数据分布高度非独立同分布，且模型易过拟合至视角特定模式；（ii）本地分布偏差导致表征错位，阻碍跨视角语义一致性；（iii）大型视频架构带来高昂的通信开销。针对上述问题，我们提出FedCVU联邦框架，包含三个核心组件：VS-Norm通过保留归一化参数处理视角特定统计信息；CV-Align作为轻量级对比正则化模块提升跨视角表征对齐；SLA采用选择性层聚合策略在保持精度的同时降低通信成本。在跨视角协议下的动作理解与行人重识别任务中，大量实验表明FedCVU在维持已见视角性能的同时持续提升未见视角准确率，其表现优于现有联邦学习基线方法，并对领域异构性与通信约束展现出强鲁棒性。
URL
https://arxiv.org/abs/2603.21647
