Abstract
Deploying geospatial foundation models on resource-constrained edge devices demands compact architectures that maintain high downstream performance. However, their large parameter counts and the accuracy loss often induced by compression limit practical adoption. In this work, we leverage manifold-constrained optimization framework DLRT to compress large vision transformer-based geospatial foundation models during transfer learning. By enforcing structured low-dimensional parameterizations aligned with downstream objectives, this approach achieves strong compression while preserving task-specific accuracy. We show that the method outperforms of-the-shelf low-rank methods as LoRA. Experiments on diverse geospatial benchmarks confirm substantial parameter reduction with minimal accuracy loss, enabling high-performing, on-device geospatial models.
Abstract (translated)
在资源受限的边缘设备上部署地理空间基础模型需要采用紧凑架构以保持高性能，但这些模型庞大的参数量以及压缩过程中常常导致的精度损失限制了其实用性。在这项工作中，我们利用流形约束优化框架DLRT，在迁移学习阶段对基于视觉变压器的大规模地理空间基础模型进行压缩。通过强制执行与下游目标相一致的结构化低维参数化，该方法在保持任务特定准确性的前提下实现了强大的压缩效果。实验结果表明，这种方法优于现成的低秩方法（如LoRA）。在多种地理空间基准上的实验证明了其能够在大幅减少参数量的同时，几乎不损失精度，从而能够实现高性能、设备内置的地理空间模型。
URL
https://arxiv.org/abs/2601.08882
