Abstract
Accurate and physically consistent modeling of Earth system dynamics requires machine-learning architectures that operate directly on continuous geophysical fields and preserve their underlying geometric structure. Here we introduce Field-Space attention, a mechanism for Earth system Transformers that computes attention in the physical domain rather than in a learned latent space. By maintaining all intermediate representations as continuous fields on the sphere, the architecture enables interpretable internal states and facilitates the enforcement of scientific constraints. The model employs a fixed, non-learned multiscale decomposition and learns structure-preserving deformations of the input field, allowing coherent integration of coarse and fine-scale information while avoiding the optimization instabilities characteristic of standard single-scale Vision Transformers. Applied to global temperature super-resolution on a HEALPix grid, Field-Space Transformers converge more rapidly and stably than conventional Vision Transformers and U-Net baselines, while requiring substantially fewer parameters. The explicit preservation of field structure throughout the network allows physical and statistical priors to be embedded directly into the architecture, yielding improved fidelity and reliability in data-driven Earth system modeling. These results position Field-Space Attention as a compact, interpretable, and physically grounded building block for next-generation Earth system prediction and generative modeling frameworks.
Abstract (translated)
地球系统动态的准确和物理一致性的建模需要能够在连续地物理场直接操作并保持其内在几何结构的机器学习架构。在这里，我们介绍了Field-Space注意力机制，这是为地球系统的Transformer设计的一种方法，在该机制中计算是在物理域而非在学习到的潜在空间进行的。通过在整个过程中将所有中间表示维护为球面上的连续字段，这种架构能够支持可解释的内部状态，并促进了科学约束条件的应用。该模型采用了一个固定的、非学习的多尺度分解，并学会了输入场保持结构的变形，从而可以连贯地集成粗粒度和细粒度信息，同时避免了标准单尺度Vision Transformer中的优化不稳定性问题。 当我们应用这种Field-Space Transformer方法对HEALPix网格上的全球温度超分辨率进行处理时，与传统的Vision Transformers以及U-Net基准模型相比，它能够更快、更稳定地收敛，并且所需的参数要少得多。在整个网络中显式保持字段结构允许物理和统计先验被直接嵌入到架构中，从而提高了数据驱动的地球系统建模的准确性和可靠性。 这些结果将Field-Space Attention定位为下一代地球系统预测和生成模型框架中的一个紧凑、可解释且具有物理学基础的重要构建模块。
URL
https://arxiv.org/abs/2512.20350
