Abstract
Effective analysis of time series data presents significant challenges due to the complex temporal dependencies and cross-channel interactions in multivariate data. Inspired by the way human analysts visually inspect time series to uncover hidden patterns, we ask: can incorporating visual representations enhance automated time-series analysis? Recent advances in multimodal large language models have demonstrated impressive generalization and visual understanding capability, yet their application to time series remains constrained by the modality gap between continuous numerical data and discrete natural language. To bridge this gap, we introduce MLLM4TS, a novel framework that leverages multimodal large language models for general time-series analysis by integrating a dedicated vision branch. Each time-series channel is rendered as a horizontally stacked color-coded line plot in one composite image to capture spatial dependencies across channels, and a temporal-aware visual patch alignment strategy then aligns visual patches with their corresponding time segments. MLLM4TS fuses fine-grained temporal details from the numerical data with global contextual information derived from the visual representation, providing a unified foundation for multimodal time-series analysis. Extensive experiments on standard benchmarks demonstrate the effectiveness of MLLM4TS across both predictive tasks (e.g., classification) and generative tasks (e.g., anomaly detection and forecasting). These results underscore the potential of integrating visual modalities with pretrained language models to achieve robust and generalizable time-series analysis.
Abstract (translated)
时间序列数据分析面临着显著的挑战，因为多变量数据中的复杂时间依赖性和跨通道交互。受人类分析师通过视觉检查时间序列以发现隐藏模式的方式启发，我们提出一个问题：是否可以通过整合视觉表示来增强自动时间序列分析？最近，在多模态大型语言模型方面的进展已经展示了出色的一般化和视觉理解能力，然而这些技术在时间序列上的应用受限于连续数值数据与离散自然语言之间的模态差距。为弥合这一差距，我们引入了MLLM4TS（Multimodal Large Language Model for Time Series），这是一个通过集成专用的视觉分支来利用多模态大型语言模型进行通用时间序列分析的新框架。 在MLLM4TS中，每个时间序列通道都被渲染成一个水平堆叠的颜色编码线图，在一个复合图像内捕获跨通道的空间依赖关系。随后，一种具有时间意识的视觉补丁对齐策略将视觉补丁与其对应的时间段对齐。通过这种方式，MLLM4TS融合了来自数值数据的细粒度时间细节与从视觉表示中提取的整体上下文信息，为多模态时间序列分析提供了一个统一的基础。 在标准基准测试上的广泛实验表明，MLLM4TS在预测任务（如分类）和生成任务（如异常检测和预测）上都表现出有效性。这些结果强调了将视觉模式与预训练语言模型相结合以实现稳健且通用的时间序列分析的潜力。
URL
https://arxiv.org/abs/2510.07513
