Abstract
Human silhouette extraction is a fundamental task in computer vision with applications in various downstream tasks. However, occlusions pose a significant challenge, leading to incomplete and distorted silhouettes. To address this challenge, we introduce POISE: Pose Guided Human Silhouette Extraction under Occlusions, a novel self-supervised fusion framework that enhances accuracy and robustness in human silhouette prediction. By combining initial silhouette estimates from a segmentation model with human joint predictions from a 2D pose estimation model, POISE leverages the complementary strengths of both approaches, effectively integrating precise body shape information and spatial information to tackle occlusions. Furthermore, the self-supervised nature of \POISE eliminates the need for costly annotations, making it scalable and practical. Extensive experimental results demonstrate its superiority in improving silhouette extraction under occlusions, with promising results in downstream tasks such as gait recognition. The code for our method is available this https URL.
Abstract (translated)
人体轮廓提取是计算机视觉中一个基本任务，在各种下游任务中有应用。然而，遮挡带来的挑战相当大，导致轮廓不完整和扭曲。为解决这个问题，我们引入了POISE：在遮挡下的人体轮廓提取，一种新颖的自监督融合框架，可以提高人体轮廓预测的准确性和鲁棒性。通过将来自分割模型的初始轮廓估计与来自2D姿势估计模型的人体关节预测相结合，POISE有效地利用了两种方法的互补优势，将精确的身体形状信息和空间信息结合起来解决遮挡问题。此外，自监督的 nature of POISE 消除了需要昂贵注释的需求，使得它具有可扩展性和实用性。大量的实验结果表明，在遮挡条件下，POISE在改善轮廓提取方面具有优越性，同时在下游任务（如步态识别）中取得了有益的结果。我们的方法的代码可以从https://URL中获取。
URL
https://arxiv.org/abs/2311.05077
