Abstract
While dense retrieval models have become the standard for state-of-the-art information retrieval, their deployment is often constrained by high memory requirements and reliance on GPU accelerators for vector similarity search. Learned sparse retrieval offers a compelling alternative by enabling efficient search via inverted indices, yet it has historically received less attention than dense approaches. In this report, we introduce LACONIC, a family of learned sparse retrievers based on the Llama-3 architecture (1B, 3B, and 8B). We propose a streamlined two-phase training curriculum consisting of (1) weakly supervised pre-finetuning to adapt causal LLMs for bidirectional contextualization and (2) high-signal finetuning using curated hard negatives. Our results demonstrate that LACONIC effectively bridges the performance gap with dense models: the 8B variant achieves a state-of-the-art 60.2 nDCG on the MTEB Retrieval benchmark, ranking 15th on the leaderboard as of January 1, 2026, while utilizing 71\% less index memory than an equivalent dense model. By delivering high retrieval effectiveness on commodity CPU hardware with a fraction of the compute budget required by competing models, LACONIC provides a scalable and efficient solution for real-world search applications.
Abstract (translated)
虽然密集检索模型已成为最先进的信息检索标准，但它们的部署通常受到高内存需求和对GPU加速器进行向量相似度搜索依赖性的限制。相比之下，通过逆向索引实现高效搜索的学习稀疏检索提供了一种有吸引力的替代方案，尽管在过去它比密集方法得到了较少的关注。 在本报告中，我们介绍了LACONIC，这是一个基于Llama-3架构（1B、3B和8B）的系列学习稀疏检索器。我们提出了一项简化的两阶段训练课程：首先通过弱监督预微调将因果LLM适应为双向上下文化；其次使用精心挑选的硬负样本进行高信号微调。我们的结果显示，LACONIC有效地缩小了与密集模型之间的性能差距：其8B变体在MTEB检索基准上实现了60.2 nDCG的最佳状态，截至2026年1月1日，在排行榜上排名第15位，并且相比等效的密集模型减少了71%的索引内存使用量。通过仅使用竞争模型所需计算预算的一小部分，LACONIC就能在普通CPU硬件上实现高检索效率，从而为现实世界的搜索应用提供了可扩展和高效的解决方案。
URL
https://arxiv.org/abs/2601.01684
