Paper Reading AI Learner

Towards Inference Efficient Deep Ensemble Learning

2023-01-29 06:48:53
Ziyue Li, Kan Ren, Yifan Yang, Xinyang Jiang, Yuqing Yang, Dongsheng Li

Abstract

Ensemble methods can deliver surprising performance gains but also bring significantly higher computational costs, e.g., can be up to 2048X in large-scale ensemble tasks. However, we found that the majority of computations in ensemble methods are redundant. For instance, over 77% of samples in CIFAR-100 dataset can be correctly classified with only a single ResNet-18 model, which indicates that only around 23% of the samples need an ensemble of extra models. To this end, we propose an inference efficient ensemble learning method, to simultaneously optimize for effectiveness and efficiency in ensemble learning. More specifically, we regard ensemble of models as a sequential inference process and learn the optimal halting event for inference on a specific sample. At each timestep of the inference process, a common selector judges if the current ensemble has reached ensemble effectiveness and halt further inference, otherwise filters this challenging sample for the subsequent models to conduct more powerful ensemble. Both the base models and common selector are jointly optimized to dynamically adjust ensemble inference for different samples with various hardness, through the novel optimization goals including sequential ensemble boosting and computation saving. The experiments with different backbones on real-world datasets illustrate our method can bring up to 56\% inference cost reduction while maintaining comparable performance to full ensemble, achieving significantly better ensemble utility than other baselines. Code and supplemental materials are available at this https URL.

Abstract (translated)

群集方法可以带来令人惊奇的性能提升,但同时也带来了显著更高的计算成本,例如在大规模群集任务中可以达到2048X。然而,我们发现大多数群集计算都是冗余的。例如,CIFAR-100数据集超过77%的样本可以用单个ResNet-18模型正确分类,这表明只有约23%的样本需要额外的群集模型。为此,我们提出了一种高效的推理群集学习方法,同时优化群集学习的效率和效果。更具体地说,我们将群集模型视为一个顺序推理过程,并学习针对特定样本的推理最佳停止事件。在每个推理步骤中,一个共同选择器判断当前群集是否已经达到了群集效果,并停止进一步推理,否则过滤这个具有挑战性的样本为后续的模型提供更强大的群集。基模型和共同选择器都同时优化,以动态调整不同样本的群集推理,通过包括顺序群集Boost和计算节省等新颖的优化目标。在真实数据集上的不同基模型实验表明,我们的方法可以实现高达56\%的推理成本降低,同时与完整群集相比保持类似的性能,实现了比其他基准方法更好的群集 utility。代码和补充材料可在该httpsURL上获取。

URL

https://arxiv.org/abs/2301.12378

PDF

https://arxiv.org/pdf/2301.12378.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot