Abstract
As large language models (LLMs) continue to scale up, their performance on various downstream tasks has significantly improved. However, evaluating their capabilities has become increasingly expensive, as performing inference on a large number of benchmark samples incurs high computational costs. In this paper, we revisit the model-item performance matrix and show that it exhibits sparsity, that representative items can be selected as anchors, and that the task of efficient benchmarking can be formulated as a sparse optimization problem. Based on these insights, we propose SparseEval, a method that, for the first time, adopts gradient descent to optimize anchor weights and employs an iterative refinement strategy for anchor selection. We utilize the representation capacity of MLP to handle sparse optimization and propose the Anchor Importance Score and Candidate Importance Score to evaluate the value of each item for task-aware refinement. Extensive experiments demonstrate the low estimation error and high Kendall's~$\tau$ of our method across a variety of benchmarks, showcasing its superior robustness and practicality in real-world scenarios. Code is available at {this https URL}.
Abstract (translated)
随着大型语言模型（LLMs）的规模不断扩大，它们在各种下游任务中的性能有了显著提升。然而，评估这些模型的能力变得越来越昂贵，因为对大量基准样本进行推理会带来高昂的计算成本。在这篇论文中，我们重新审视了模型-项目表现矩阵，并展示了它表现出稀疏性、代表性的项目可以被选为锚点，以及高效基准测试的任务可以被表述为一个稀疏优化问题。基于这些见解，我们提出了SparseEval方法，首次采用梯度下降来优化锚权重，并采用迭代细化策略进行锚选择。我们利用MLP的表示能力处理稀疏优化，并提出锚重要性得分和候选重要性得分以评估每个项目在任务感知细化中的价值。广泛的实验展示了我们的方法在各种基准测试中具有低估计误差和高肯德尔$\tau$，体现了其在现实世界场景中的优越稳健性和实用性。代码可在{此链接}获取。
URL
https://arxiv.org/abs/2602.07909
