Abstract
Human parsing aims to segment each pixel of the human image with fine-grained semantic categories. However, current human parsers trained with clean data are easily confused by numerous image corruptions such as blur and noise. To improve the robustness of human parsers, in this paper, we construct three corruption robustness benchmarks, termed LIP-C, ATR-C, and Pascal-Person-Part-C, to assist us in evaluating the risk tolerance of human parsing models. Inspired by the data augmentation strategy, we propose a novel heterogeneous augmentation-enhanced mechanism to bolster robustness under commonly corrupted conditions. Specifically, two types of data augmentations from different views, i.e., image-aware augmentation and model-aware image-to-image transformation, are integrated in a sequential manner for adapting to unforeseen image corruptions. The image-aware augmentation can enrich the high diversity of training images with the help of common image operations. The model-aware augmentation strategy that improves the diversity of input data by considering the model's randomness. The proposed method is model-agnostic, and it can plug and play into arbitrary state-of-the-art human parsing frameworks. The experimental results show that the proposed method demonstrates good universality which can improve the robustness of the human parsing models and even the semantic segmentation models when facing various image common corruptions. Meanwhile, it can still obtain approximate performance on clean data.
Abstract (translated)
人类解析旨在以精细的语义分类对人类图像的每一像素进行分割。然而,当前使用干净数据训练的人类解析器容易受到诸如模糊和噪声等常见的图像损坏。为了改善人类解析器的鲁棒性,在本文中,我们建立了三个 corruption 鲁棒性基准,称为 LIP-C、ATR-C 和 Pascal-Person-Part-C,以协助我们评估人类解析模型的风险容忍度。受到数据增强策略启发,我们提出了一种异质增强机制,以在常见的损坏条件下增强鲁棒性。具体来说,从不同视角提供的数据增强有两种类型,即图像意识增强和模型意识的图像到图像变换,通过Sequentially integrated approach 适应意想不到的图像损坏。图像意识增强可以通过常见的图像操作丰富训练图像的高度多样性。模型意识增强策略通过考虑模型的随机性来提高输入数据的多样性。我们提出的方法是非模型特定的,它可以与任意先进的人类解析框架插件和玩耍。实验结果显示,我们提出的方法表现出良好的通用性,可以提高在面临各种常见图像损坏时人类解析模型和语义分割模型的鲁棒性。同时,在干净数据上仍然可以实现近似性能。
URL
https://arxiv.org/abs/2309.00938
