Paper Reading AI Learner

A Text-Based Knowledge-Embedded Soft Sensing Modeling Approach for General Industrial Process Tasks Based on Large Language Model

2025-01-09 08:59:14
Shuo Tong, Han Liu, Runyuan Guo, Xueqiong Tian, Wenqing Wang, Ding Liu, Youmin Zhang

Abstract

Data-driven soft sensors (DDSS) have become mainstream methods for predicting key performance indicators in process industries. However, DDSS development requires complex and costly customized designs tailored to various tasks during the modeling process. Moreover, DDSS are constrained to a single structured data modality, limiting their ability to incorporate additional contextual knowledge. Furthermore, DDSSs' limited representation learning leads to weak predictive performance with scarce data. To address these challenges, we propose a general framework named LLM-TKESS (large language model for text-based knowledge-embedded soft sensing), harnessing the powerful general problem-solving capabilities, cross-modal knowledge transfer abilities, and few-shot capabilities of LLM for enhanced soft sensing modeling. Specifically, an auxiliary variable series encoder (AVS Encoder) is proposed to unleash LLM's potential for capturing temporal relationships within series and spatial semantic relationships among auxiliary variables. Then, we propose a two-stage fine-tuning alignment strategy: in the first stage, employing parameter-efficient fine-tuning through autoregressive training adjusts LLM to rapidly accommodate process variable data, resulting in a soft sensing foundation model (SSFM). Subsequently, by training adapters, we adapt the SSFM to various downstream tasks without modifying its architecture. Then, we propose two text-based knowledge-embedded soft sensors, integrating new natural language modalities to overcome the limitations of pure structured data models. Furthermore, benefiting from LLM's pre-existing world knowledge, our model demonstrates outstanding predictive capabilities in small sample conditions. Using the thermal deformation of air preheater rotor as a case study, we validate through extensive experiments that LLM-TKESS exhibits outstanding performance.

Abstract (translated)

数据驱动的软传感器(DDSS)已成为过程工业中预测关键绩效指标的主要方法。然而,DDSS的发展需要在建模过程中进行复杂的、成本高昂且针对特定任务定制的设计。此外,DDSS仅限于单一结构化数据模式,限制了其融合额外上下文知识的能力。更进一步的是,由于DDSS的表示学习能力有限,在数据稀缺的情况下预测性能较弱。为了解决这些挑战,我们提出了一种名为LLM-TKESS(基于大型语言模型的知识嵌入文本软传感)的一般框架,利用LLM强大的通用问题解决能力、跨模态知识转移能力和少量样本处理能力来增强软传感器建模。 具体而言,我们提出了一个辅助变量序列编码器(AVS编码器),以释放LLM在捕捉时间序列内的时间关系和辅助变量间的空间语义关系方面的潜力。然后,我们提出了一种两阶段的微调对齐策略:第一阶段通过自回归训练进行参数高效的微调,使LLM能够快速适应过程变量数据,形成一个软传感基础模型(SSFM)。随后,在不修改其架构的情况下,通过对适配器进行训练来将SSFM调整为各种下游任务。此外,我们提出了两种基于文本的知识嵌入式软传感器,通过引入新的自然语言模态克服了纯结构化数据模型的局限性。 得益于LLM预先存在的世界知识,我们的模型在小样本条件下展示了卓越的预测能力。以空气预热器转子的热变形为例,在广泛的实验验证中,我们证明了LLM-TKESS表现出色。

URL

https://arxiv.org/abs/2501.05075

PDF

https://arxiv.org/pdf/2501.05075.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot