Abstract
Data-driven soft sensors (DDSS) have become mainstream methods for predicting key performance indicators in process industries. However, DDSS development requires complex and costly customized designs tailored to various tasks during the modeling process. Moreover, DDSS are constrained to a single structured data modality, limiting their ability to incorporate additional contextual knowledge. Furthermore, DDSSs' limited representation learning leads to weak predictive performance with scarce data. To address these challenges, we propose a general framework named LLM-TKESS (large language model for text-based knowledge-embedded soft sensing), harnessing the powerful general problem-solving capabilities, cross-modal knowledge transfer abilities, and few-shot capabilities of LLM for enhanced soft sensing modeling. Specifically, an auxiliary variable series encoder (AVS Encoder) is proposed to unleash LLM's potential for capturing temporal relationships within series and spatial semantic relationships among auxiliary variables. Then, we propose a two-stage fine-tuning alignment strategy: in the first stage, employing parameter-efficient fine-tuning through autoregressive training adjusts LLM to rapidly accommodate process variable data, resulting in a soft sensing foundation model (SSFM). Subsequently, by training adapters, we adapt the SSFM to various downstream tasks without modifying its architecture. Then, we propose two text-based knowledge-embedded soft sensors, integrating new natural language modalities to overcome the limitations of pure structured data models. Furthermore, benefiting from LLM's pre-existing world knowledge, our model demonstrates outstanding predictive capabilities in small sample conditions. Using the thermal deformation of air preheater rotor as a case study, we validate through extensive experiments that LLM-TKESS exhibits outstanding performance.
Abstract (translated)
数据驱动的软传感器(DDSS)已成为过程工业中预测关键绩效指标的主要方法。然而,DDSS的发展需要在建模过程中进行复杂的、成本高昂且针对特定任务定制的设计。此外,DDSS仅限于单一结构化数据模式,限制了其融合额外上下文知识的能力。更进一步的是,由于DDSS的表示学习能力有限,在数据稀缺的情况下预测性能较弱。为了解决这些挑战,我们提出了一种名为LLM-TKESS(基于大型语言模型的知识嵌入文本软传感)的一般框架,利用LLM强大的通用问题解决能力、跨模态知识转移能力和少量样本处理能力来增强软传感器建模。 具体而言,我们提出了一个辅助变量序列编码器(AVS编码器),以释放LLM在捕捉时间序列内的时间关系和辅助变量间的空间语义关系方面的潜力。然后,我们提出了一种两阶段的微调对齐策略:第一阶段通过自回归训练进行参数高效的微调,使LLM能够快速适应过程变量数据,形成一个软传感基础模型(SSFM)。随后,在不修改其架构的情况下,通过对适配器进行训练来将SSFM调整为各种下游任务。此外,我们提出了两种基于文本的知识嵌入式软传感器,通过引入新的自然语言模态克服了纯结构化数据模型的局限性。 得益于LLM预先存在的世界知识,我们的模型在小样本条件下展示了卓越的预测能力。以空气预热器转子的热变形为例,在广泛的实验验证中,我们证明了LLM-TKESS表现出色。
URL
https://arxiv.org/abs/2501.05075