Abstract
We present OmniSpectra, the first native-resolution foundation model for astronomy spectra. Unlike traditional models, which are limited to fixed-length input sizes or configurations, OmniSpectra handles spectra of any length at their original size, without resampling or interpolation. Despite the large-scale spectroscopic data from diverse surveys fueling the rapid growth of astronomy, existing foundation models are limited to a fixed wavelength range and specific instruments. OmniSpectra is the first foundation model to learn simultaneously from multiple real-world spectra surveys with different configurations at a large scale. We achieve this by designing a novel architecture with adaptive patching across variable lengths, sinusoidal global wavelength encoding, local positional embeddings through depthwise convolution, and validity-aware self-attention masks. Allowing us to learn multi-scale spatial patterns while skipping attention for invalid patches. Even with a limited training example, OmniSpectra demonstrates excellent zero-shot generalization compared to methods tailored for specific tasks. This transfer learning capability makes this model the state-of-the-art across various astronomy tasks, including source classification, redshift estimation, and properties prediction for stars and galaxies. OmniSpectra reduces the need for training individual models for different tasks from scratch, establishing itself as the next-generation astronomy foundation model.
Abstract (translated)
我们介绍了OmniSpectra，这是首个用于天文学光谱的原生分辨率基础模型。与传统的模型不同，后者仅限于固定长度输入大小或配置，OmniSpectra能够处理任何长度的光谱，并且保持其原始尺寸，无需重采样或插值操作。尽管大规模的分光数据集来自各种不同的调查研究推动了天文学的快速发展，现有的基础模型仍然局限于固定的波长范围和特定仪器。OmniSpectra是首个同时从多个具有不同配置的真实世界光谱调查中进行大规模学习的基础模型。 我们通过设计一种新颖的架构来实现这一目标，该架构包括适应于可变长度的自适应补丁技术、全局波长编码（采用正弦函数）、通过深度卷积获取局部位置嵌入以及感知有效性的自我关注掩码。这使OmniSpectra能够学习多尺度的空间模式，并且可以跳过无效补丁的关注过程。 即便在训练示例有限的情况下，与专门针对特定任务的方法相比，OmniSpectra也表现出色的零样本泛化能力。这种迁移学习的能力使得该模型成为天文学各种任务中的最新技术，包括源分类、红移估计以及恒星和星系属性预测。OmniSpectra减少了从头开始为不同任务训练单独模型的需求，并确立了其作为下一代天文基础模型的地位。
URL
https://arxiv.org/abs/2601.15351
