Abstract
Satellite image time-series analysis demands fine-grained spatial-temporal reasoning, which remains a challenge for existing multimodal large language models (MLLMs). In this work, we study the capabilities of MLLMs on a novel task that jointly targets temporal change understanding and future scene generation, aiming to assess their potential for modeling complex multimodal dynamics over time. We propose TAMMs, a Temporal-Aware Multimodal Model for satellite image change understanding and forecasting, which enhances frozen MLLMs with lightweight temporal modules for structured sequence encoding and contextual prompting. To guide future image generation, TAMMs introduces a Semantic-Fused Control Injection (SFCI) mechanism that adaptively combines high-level semantic reasoning and structural priors within an enhanced ControlNet. This dual-path conditioning enables temporally consistent and semantically grounded image synthesis. Experiments demonstrate that TAMMs outperforms strong MLLM baselines in both temporal change understanding and future image forecasting tasks, highlighting how carefully designed temporal reasoning and semantic fusion can unlock the full potential of MLLMs for spatio-temporal understanding.
Abstract (translated)
卫星图像时间序列分析需要精细的时空推理，这对现有的多模态大型语言模型（MLLM）来说仍然是一个挑战。在本项工作中，我们研究了这些模型在一个新颖的任务上的能力，该任务旨在同时理解时间变化并生成未来场景，以评估它们建模复杂多模态动态变化随时间演变的能力。 为此，我们提出了TAMMs，即时空感知的多模态模型（Temporal-Aware Multimodal Model），用于卫星图像变化理解和预测。通过为冻结的MLLM添加轻量级的时间模块，该模型能够进行结构化序列编码和上下文提示增强。为了指导未来图像生成，TAMMs引入了语义融合控制注入机制（Semantic-Fused Control Injection, SFCI）。这种机制在改进后的ControlNet中适应性地结合高级别的语义推理和结构性先验知识。双路径条件处理使得图像合成既具有时间一致性又基于语义。 实验结果表明，TAMMs在时间和变化理解以及未来图像预测任务上都超过了强大的MLLM基准模型，这突显了精心设计的时间推理和语义融合如何解锁MLLM在时空理解方面的全部潜力。
URL
https://arxiv.org/abs/2506.18862
