Abstract
Modern machine learning datasets can have biases for certain representations that are leveraged by algorithms to achieve high performance without learning to solve the underlying task. This problem is referred to as "representation bias". The question of how to reduce the representation biases of a dataset is investigated and a new dataset REPresentAtion bIas Removal (REPAIR) procedure is proposed. This formulates bias minimization as an optimization problem, seeking a weight distribution that penalizes examples easy for a classifier built on a given feature representation. Bias reduction is then equated to maximizing the ratio between the classification loss on the reweighted dataset and the uncertainty of the ground-truth class labels. This is a minimax problem that REPAIR solves by alternatingly updating classifier parameters and dataset resampling weights, using stochastic gradient descent. An experimental set-up is also introduced to measure the bias of any dataset for a given representation, and the impact of this bias on the performance of recognition models. Experiments with synthetic and action recognition data show that dataset REPAIR can significantly reduce representation bias, and lead to improved generalization of models trained on REPAIRed datasets. The tools used for characterizing representation bias, and the proposed dataset REPAIR algorithm, are available at https://github.com/JerryYLi/Dataset-REPAIR/.
Abstract (translated)
现代机器学习数据集可能对某些表示有偏见，这些表示被算法利用以在不学习解决底层任务的情况下实现高性能。这个问题被称为“表示偏差”。研究了如何减少数据集表示偏差的问题，提出了一种新的数据集表示偏差消除（修复）方法。这将偏差最小化定义为一个优化问题，寻求一个权重分布来惩罚基于给定特征表示的分类器容易出现的示例。然后将偏差减少等同于最大化重新加权数据集上的分类损失与地面真值类标签的不确定性之间的比率。这是一个极大极小问题，通过交替更新分类器参数和数据集重采样权重，利用随机梯度下降来修复。还引入了一个实验装置来测量给定表示的任何数据集的偏差，以及这种偏差对识别模型性能的影响。对合成数据和动作识别数据的实验表明，数据集修复可以显著地减少表示偏差，并改进了在修复数据集上训练的模型的泛化。用于描述表示偏差的工具和建议的数据集修复算法可在https://github.com/jerryli/dataset repair/上找到。
URL
https://arxiv.org/abs/1904.07911
