Abstract
Advances in natural language processing and large language models are driving a major transformation in Human Capital Management, with a growing interest in building smart systems based on language technologies for talent acquisition, upskilling strategies, and workforce planning. However, the adoption and progress of these technologies critically depend on the development of reliable and fair models, properly evaluated on public data and open benchmarks, which have so far been unavailable in this domain. To address this gap, we present TalentCLEF 2025, the first evaluation campaign focused on skill and job title intelligence. The lab consists of two tasks: Task A - Multilingual Job Title Matching, covering English, Spanish, German, and Chinese; and Task B - Job Title-Based Skill Prediction, in English. Both corpora were built from real job applications, carefully anonymized, and manually annotated to reflect the complexity and diversity of real-world labor market data, including linguistic variability and gender-marked expressions. The evaluations included monolingual and cross-lingual scenarios and covered the evaluation of gender bias. TalentCLEF attracted 76 registered teams with more than 280 submissions. Most systems relied on information retrieval techniques built with multilingual encoder-based models fine-tuned with contrastive learning, and several of them incorporated large language models for data augmentation or re-ranking. The results show that the training strategies have a larger effect than the size of the model alone. TalentCLEF provides the first public benchmark in this field and encourages the development of robust, fair, and transferable language technologies for the labor market.
Abstract (translated)
自然语言处理和大型语言模型的进步正在推动人力资源管理的重大转型，人们对基于语言技术构建智能系统以支持人才获取、技能提升策略及劳动力规划的兴趣日益增长。然而，这些技术的采用和发展在很大程度上依赖于开发可靠且公平的模型，并经过公共数据集和开放基准测试的恰当评估，在此领域迄今为止尚无可用资源。为填补这一空白，我们推出了TalentCLEF 2025，这是首个专注于技能与职业岗位智能的评估活动。 该实验室包括两个任务：任务A - 多语言职位匹配，涵盖英语、西班牙语、德语和中文；以及任务B - 基于职务名称的技能预测，在英语中进行。两份数据集均基于实际工作申请构建，并进行了仔细匿名处理及人工标注以反映现实劳动力市场的复杂性和多样性，包括语言变异性及性别标记表达。 评估涵盖了单语种和跨语种场景，并涉及性别偏见的评价。TalentCLEF吸引了76支注册队伍提交了超过280份作品。大多数系统依赖于多语言编码模型与信息检索技术结合使用，这些模型通过对比学习进行微调，有些则融入大型语言模型用于数据增强或重新排序。 结果显示，训练策略比单纯模型大小的影响更大。TalentCLEF为该领域提供了首个公共基准，并鼓励开发稳健、公平且可转移的语言技术以应用于劳动力市场。
URL
https://arxiv.org/abs/2507.13275
