Abstract
The aim of Active Learning is to select the most informative samples from an unlabelled set of data. This is useful in cases where the amount of data is large and labelling is expensive, such as in machine vision or medical imaging. Two particularities of machine vision are first, that most of the images produced are free of defects, and second, that the amount of images produced is so big that we cannot store all acquired images. This results, on the one hand, in a strong class imbalance in defect distribution and, on the other hand, in a potential label shift caused by limited storage. To understand how these two forms of imbalance affect active learning algorithms, we propose a simulation study based on two open-source datasets. We artificially create datasets for which we control the levels of class imbalance and label shift. Three standard active learning selection strategies are compared: random sampling, entropy-based selection, and core-set selection. We demonstrate that active learning strategies, and in particular the entropy-based and core-set selections, remain interesting and efficient even for highly imbalanced datasets. We also illustrate and measure the loss of efficiency that occurs in the situation a strong label shift.
Abstract (translated)
主动学习的目标是从未标记的数据集中挑选出最具信息量的样本。这在数据量大且标注成本高的情况下非常有用，比如机器视觉或医学影像领域。机器视觉有两个特点：一是生成的大部分图像没有缺陷；二是产生的图像数量庞大到无法全部存储。这两点导致了两个问题：一方面，缺陷分布呈现出强烈的类别不平衡；另一方面，由于有限的存储空间，可能会发生标签偏移。 为了理解这两种不平衡如何影响主动学习算法，我们基于两个开源数据集进行了一项模拟研究，并人为创建了一些具有不同类不平衡和标签偏移级别的数据集。我们比较了三种标准的主动学习选择策略：随机抽样、熵基选择以及核心集选择。研究表明，即使在极端类别不平衡的数据集中，主动学习策略（特别是基于熵的选择和核心集选择）仍然有效且高效。同时，我们也展示了并量化了强标签偏移情况下效率降低的情况。
URL
https://arxiv.org/abs/2601.06209
