Paper Reading AI Learner

Contrastive Bi-Encoder Models for Multi-Label Skill Extraction: Enhancing ESCO Ontology Matching with BERT and Attention Mechanisms

2026-01-14 03:43:45
Yongming Sun

Abstract

Fine-grained labor market analysis increasingly relies on mapping unstructured job advertisements to standardized skill taxonomies such as ESCO. This mapping is naturally formulated as an Extreme Multi-Label Classification (XMLC) problem, but supervised solutions are constrained by the scarcity and cost of large-scale, taxonomy-aligned annotations--especially in non-English settings where job-ad language diverges substantially from formal skill definitions. We propose a zero-shot skill extraction framework that eliminates the need for manually labeled job-ad training data. The framework uses a Large Language Model (LLM) to synthesize training instances from ESCO definitions, and introduces hierarchically constrained multi-skill generation based on ESCO Level-2 categories to improve semantic coherence in multi-label contexts. On top of the synthetic corpus, we train a contrastive bi-encoder that aligns job-ad sentences with ESCO skill descriptions in a shared embedding space; the encoder augments a BERT backbone with BiLSTM and attention pooling to better model long, information-dense requirement statements. An upstream RoBERTa-based binary filter removes non-skill sentences to improve end-to-end precision. Experiments show that (i) hierarchy-conditioned generation improves both fluency and discriminability relative to unconstrained pairing, and (ii) the resulting multi-label model transfers effectively to real-world Chinese job advertisements, achieving strong zero-shot retrieval performance (F1@5 = 0.72) and outperforming TF--IDF and standard BERT baselines. Overall, the proposed pipeline provides a scalable, data-efficient pathway for automated skill coding in labor economics and workforce analytics.

Abstract (translated)

精细粒度的劳动力市场分析越来越依赖于将非结构化的职位广告映射到诸如ESCO(欧洲技能/能力、资格框架)等标准化的技能分类体系。这种映射自然可以被形式化为一个极多标签分类(XMLC)问题,但监督解决方案由于大规模、与分类体系对齐标注数据的稀缺和高昂成本而受到限制——特别是在非英语环境下,职位广告语言与正式技能定义之间存在显著差异的情况下更是如此。我们提出了一种零样本技能提取框架,该框架消除了手动标记职位广告训练数据的需求。该框架使用大型语言模型(LLM)从ESCO定义中合成训练实例,并引入基于ESCO第二级类别的分层约束多技能生成机制,以在多标签上下文中提高语义一致性。在此人工合成的语料库之上,我们训练了一个对比双编码器来将职位广告句子与ESCO技能描述对齐到共享嵌入空间;该编码器通过添加双向长短时记忆(BiLSTM)和注意力池化层增强BERT骨干模型,以更好地处理长且信息密集的要求陈述。一个上游的RoBERTa基二元过滤器用于去除非技能语句,从而提高整体精度。 实验表明: (i) 分层条件生成提高了相对于无约束配对而言的语言流畅性和可区分性; (ii) 所得到的多标签模型能够有效转移到真实世界中的中文职位广告上,并取得了强大的零样本检索性能(F1@5 = 0.72),并且优于TF-IDF和标准BERT基线。 总体来说,所提出的处理流程为劳动力经济学和工作力量分析领域的自动化技能编码提供了一条可扩展且数据高效的途径。

URL

https://arxiv.org/abs/2601.09119

PDF

https://arxiv.org/pdf/2601.09119.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot