Abstract
We present LOCUS (LOw-cost Customization for Universal Specialization), a pipeline that consumes few-shot data to streamline the construction and training of NLP models through targeted retrieval, synthetic data generation, and parameter-efficient tuning. With only a small number of labeled examples, LOCUS discovers pertinent data in a broad repository, synthesizes additional training samples via in-context data generation, and fine-tunes models using either full or low-rank (LoRA) parameter adaptation. Our approach targets named entity recognition (NER) and text classification (TC) benchmarks, consistently outperforming strong baselines (including GPT-4o) while substantially lowering costs and model sizes. Our resultant memory-optimized models retain 99% of fully fine-tuned accuracy while using barely 5% of the memory footprint, also beating GPT-4o on several benchmarks with less than 1% of its parameters.
Abstract (translated)
我们介绍了LOCUS（LOw-cost Customization for Universal Specialization），这是一种通过目标检索、合成数据生成和参数高效调整来消耗少量样本数据，从而简化自然语言处理模型构建和训练的流水线。仅使用少量标记示例，LOCUS可以在广泛的数据存储库中发现相关数据，利用上下文数据生成额外的训练样本，并采用全参数或低秩（LoRA）参数适应方法进行微调。我们的方法针对命名实体识别（NER）和文本分类（TC）基准测试，在性能上持续优于强大的基线模型（包括GPT-4o），同时大幅降低成本和模型规模。我们优化后的内存模型在使用不到全量微调所需内存的5%的情况下，保留了99%的精度，并且在某些基准测试中，参数仅为GPT-4o的不到1%时依然超越了后者的表现。
URL
https://arxiv.org/abs/2512.06239
