Abstract
Text classification plays a pivotal role in edge computing applications like industrial monitoring, health diagnostics, and smart assistants, where low latency and high accuracy are both key requirements. Generative classifiers, in particular, have been shown to exhibit robustness to out-of-distribution and noisy data, which is an extremely critical consideration for deployment in such real-time edge environments. However, deploying such models on edge devices faces computational and memory constraints. Post Training Quantization (PTQ) reduces model size and compute costs without retraining, making it ideal for edge deployment. In this work, we present a comprehensive comparative study of generative and discriminative Long Short Term Memory (LSTM)-based text classification models with PTQ using the Brevitas quantization library. We evaluate both types of classifier models across multiple bitwidths and assess their robustness under regular and noisy input conditions. We find that while discriminative classifiers remain robust, generative ones are more sensitive to bitwidth, calibration data used during PTQ, and input noise during quantized inference. We study the influence of class imbalance in calibration data for both types of classifiers, comparing scenarios with evenly and unevenly distributed class samples including their effect on weight adjustments and activation profiles during PTQ. Using test statistics derived from nonparametric hypothesis testing, we identify that using class imbalanced data during calibration introduces insufficient weight adaptation at lower bitwidths for generative LSTM classifiers, thereby leading to degraded performance. This study underscores the role of calibration data in PTQ and when generative classifiers succeed or fail under noise, aiding deployment in edge environments.
Abstract (translated)
文本分类在边缘计算应用中扮演着关键角色，尤其是在工业监控、健康诊断和智能助手等领域，这些领域对低延迟和高准确性都有严格要求。生成式分类器特别展现出对分布外（out-of-distribution）数据及噪声数据的鲁棒性，这是部署于实时边缘环境中的极重要考量因素之一。然而，在边缘设备上部署此类模型会面临计算资源和内存限制的问题。通过使用Post-Training Quantization (PTQ)技术可以在不重新训练的情况下减小模型大小并降低计算成本，这使得该技术非常适合边缘部署。 在本研究中，我们提供了一种基于Brevitas量化库的生成式与判别式长短期记忆（LSTM）文本分类模型的全面比较分析。通过不同位宽(bitwidth)和常规及噪声输入条件对两种类型的分类器进行了评估，并考察了它们的鲁棒性。我们发现尽管判别式分类器依然保持其稳健性，但生成式的分类器则更敏感于位宽、PTQ过程中使用的校准数据以及量化推理期间的输入噪音的影响。 此外，还研究了两类分类器在校准数据中的类别不平衡（class imbalance）影响。通过比较不同类型的样本分布（包括均匀与非均匀分布情况），分析其对权重调整和激活配置文件在PTQ过程中的影响。使用从非参数假设检验得出的统计测试方法发现，在较低位宽时，生成式LSTM分类器在校准过程中使用类别不平衡数据会导致不足的权重重定，从而导致性能下降。 这项研究强调了校准数据在PTQ中的作用以及生成式分类器在噪音环境下的成功或失败情况，这对边缘环境部署具有指导意义。
URL
https://arxiv.org/abs/2507.09687
