Abstract
Transformer-based models like BERT excel at short text classification but struggle with long document classification (LDC) due to input length limitations and computational inefficiencies. In this work, we propose an efficient, zero-shot approach to LDC that leverages sentence ranking to reduce input context without altering the model architecture. Our method enables the adaptation of models trained on short texts, such as headlines, to long-form documents by selecting the most informative sentences using a TF-IDF-based ranking strategy. Using the MahaNews dataset of long Marathi news articles, we evaluate three context reduction strategies that prioritize essential content while preserving classification accuracy. Our results show that retaining only the top 50\% ranked sentences maintains performance comparable to full-document inference while reducing inference time by up to 35\%. This demonstrates that sentence ranking is a simple yet effective technique for scalable and efficient zero-shot LDC.
Abstract (translated)
基于Transformer的模型（如BERT）在处理短文本分类时表现出色，但在长文档分类（LDC）方面却面临输入长度限制和计算效率低下的挑战。为此，我们提出了一种高效且无需微调的方法来应对长文档分类问题，该方法利用句子排名减少输入上下文而不改变模型架构。我们的方法允许将训练于短文本（如新闻标题）上的模型应用于长篇文档，并通过使用基于TF-IDF的排名策略选择最具信息量的句子实现这一目标。 我们使用MahaNews数据集中的长马拉地语新闻文章，评估了三种减少上下文的方法，这些方法优先考虑关键内容的同时保持分类准确性。实验结果表明，保留前50%排名最高的句子可以将推理时间最多缩短35%，而同时维持与完整文档推理时相当的性能表现。 这表明基于句子排名的技术是一种简单且有效的方法，可用于零样本长文档分类（LDC），以实现可扩展性和效率。
URL
https://arxiv.org/abs/2508.17490
