Abstract
The increasing deployment of large language models (LLMs) in natural language processing (NLP) tasks raises concerns about energy efficiency and sustainability. While prior research has largely focused on energy consumption during model training, the inference phase has received comparatively less attention. This study systematically evaluates the trade-offs between model accuracy and energy consumption in text classification inference across various model architectures and hardware configurations. Our empirical analysis shows that the best-performing model in terms of accuracy can also be energy-efficient, while larger LLMs tend to consume significantly more energy with lower classification accuracy. We observe substantial variability in inference energy consumption ($<$mWh to $>$kWh), influenced by model type, model size, and hardware specifications. Additionally, we find a strong correlation between inference energy consumption and model runtime, indicating that execution time can serve as a practical proxy for energy usage in settings where direct measurement is not feasible. These findings have implications for sustainable AI development, providing actionable insights for researchers, industry practitioners, and policymakers seeking to balance performance and resource efficiency in NLP applications.
Abstract (translated)
在自然语言处理（NLP）任务中部署大规模语言模型（LLMs）引发了关于能源效率和可持续性的担忧。尽管先前的研究主要集中在模型训练过程中的能耗问题，但在推理阶段的关注相对较少。本研究系统地评估了不同模型架构和硬件配置下文本分类推理过程中模型准确性和能量消耗之间的权衡关系。我们的实证分析表明，在准确性方面表现最佳的模型也可以是能源效率高的，而更大的LLMs通常会消耗更多能量且分类精度更低。 我们观察到推理阶段的能量消耗存在显著差异（从不到毫瓦时到超过千瓦时），这受到模型类型、模型大小以及硬件规格的影响。此外，我们发现推理能耗与模型运行时间之间有很强的相关性，表明在无法直接测量的情况下，执行时间可以作为能源使用的一个实用替代指标。 这些研究结果对可持续AI的发展具有重要影响，为寻求平衡性能和资源效率的研究人员、行业从业者及政策制定者提供了可操作的见解。
URL
https://arxiv.org/abs/2508.14170
