Abstract
The comparison between discriminative and generative classifiers has intrigued researchers since Efron's seminal analysis of logistic regression versus discriminant analysis. While early theoretical work established that generative classifiers exhibit lower sample complexity but higher asymptotic error in simple linear settings, these trade-offs remain unexplored in the transformer era. We present the first comprehensive evaluation of modern generative and discriminative architectures - Auto-regressive modeling, Masked Language Modeling, Discrete Diffusion, and Encoders for text classification. Our study reveals that the classical 'two regimes' phenomenon manifests distinctly across different architectures and training paradigms. Beyond accuracy, we analyze sample efficiency, calibration, noise robustness, and ordinality across diverse scenarios. Our findings offer practical guidance for selecting the most suitable modeling approach based on real-world constraints such as latency and data limitations.
Abstract (translated)
判别分类器与生成式分类器之间的比较自Efron对逻辑回归和判别分析的开创性分析以来，一直吸引着研究人员的兴趣。早期的理论工作表明，在简单的线性设置中，生成式分类器表现出更低的数据复杂度但更高的渐近误差。然而，在Transformer时代，这些权衡关系仍未得到充分探索。 我们首次全面评估了现代生成式和判别式架构——包括自回归建模、掩码语言模型、离散扩散以及用于文本分类的编码器。我们的研究揭示了经典的“两种模式”现象在不同架构和训练范式中表现出不同的特征。除了准确性之外，我们还分析了样本效率、校准性、对噪声的鲁棒性和顺序性在各种场景中的表现。 这些发现为基于现实世界约束（如延迟和数据限制）选择最合适的建模方法提供了实用指导。
URL
https://arxiv.org/abs/2506.12181
