Abstract
Ensemble methods can deliver surprising performance gains but also bring significantly higher computational costs, e.g., can be up to 2048X in large-scale ensemble tasks. However, we found that the majority of computations in ensemble methods are redundant. For instance, over 77% of samples in CIFAR-100 dataset can be correctly classified with only a single ResNet-18 model, which indicates that only around 23% of the samples need an ensemble of extra models. To this end, we propose an inference efficient ensemble learning method, to simultaneously optimize for effectiveness and efficiency in ensemble learning. More specifically, we regard ensemble of models as a sequential inference process and learn the optimal halting event for inference on a specific sample. At each timestep of the inference process, a common selector judges if the current ensemble has reached ensemble effectiveness and halt further inference, otherwise filters this challenging sample for the subsequent models to conduct more powerful ensemble. Both the base models and common selector are jointly optimized to dynamically adjust ensemble inference for different samples with various hardness, through the novel optimization goals including sequential ensemble boosting and computation saving. The experiments with different backbones on real-world datasets illustrate our method can bring up to 56\% inference cost reduction while maintaining comparable performance to full ensemble, achieving significantly better ensemble utility than other baselines. Code and supplemental materials are available at this https URL.
Abstract (translated)
群集方法可以带来令人惊奇的性能提升,但同时也带来了显著更高的计算成本,例如在大规模群集任务中可以达到2048X。然而,我们发现大多数群集计算都是冗余的。例如,CIFAR-100数据集超过77%的样本可以用单个ResNet-18模型正确分类,这表明只有约23%的样本需要额外的群集模型。为此,我们提出了一种高效的推理群集学习方法,同时优化群集学习的效率和效果。更具体地说,我们将群集模型视为一个顺序推理过程,并学习针对特定样本的推理最佳停止事件。在每个推理步骤中,一个共同选择器判断当前群集是否已经达到了群集效果,并停止进一步推理,否则过滤这个具有挑战性的样本为后续的模型提供更强大的群集。基模型和共同选择器都同时优化,以动态调整不同样本的群集推理,通过包括顺序群集Boost和计算节省等新颖的优化目标。在真实数据集上的不同基模型实验表明,我们的方法可以实现高达56\%的推理成本降低,同时与完整群集相比保持类似的性能,实现了比其他基准方法更好的群集 utility。代码和补充材料可在该httpsURL上获取。
URL
https://arxiv.org/abs/2301.12378