Abstract
Semantic ultra high resolution image (UHR) segmentation is essential in remote sensing applications such as aerial mapping and environmental monitoring. Transformer-based models struggle in this setting because memory grows quadratically with token count, constraining either the contextual scope or the spatial resolution. We introduce CASWiT (Context-Aware Stage-Wise Transformer), a dual-branch, Swin-based architecture that injects global cues into fine-grained UHR features. A context encoder processes a downsampled neighborhood to capture long-range dependencies, while a high resolution encoder extracts detailed features from UHR patches. A cross-scale fusion module, combining cross-attention and gated feature injection, enriches high-resolution tokens with context. Beyond architecture, we propose a SimMIM-style pretraining. We mask 75% of the high-resolution image tokens and the low-resolution center region that spatially corresponds to the UHR patch, then train the shared dual-encoder with small decoder to reconstruct the UHR initial image. Extensive experiments on the large-scale IGN FLAIR-HUB aerial dataset demonstrate the effectiveness of CASWiT. Our method achieves 65.83% mIoU, outperforming RGB baselines by 1.78 points. On URUR, CASWiT achieves 49.1% mIoU, surpassing the current SoTA by +0.9% under the official evaluation protocol. All codes are provided on: this https URL.
Abstract (translated)
语义超高清（UHR）图像分割在遥感应用如航空测绘和环境监测中至关重要。然而，基于Transformer的模型在这种环境下表现不佳，因为内存需求随令牌数量呈二次增长，这限制了上下文范围或空间分辨率的大小。为此，我们提出了CASWiT（Context-Aware Stage-Wise Transformer），这是一种双分支、基于Swin的架构，它将全局线索注入到细粒度的UHR特征中。上下文编码器处理降采样的邻域以捕捉长距离依赖关系，而高分辨率编码器则从UHR补丁中提取详细特征。跨尺度融合模块结合了跨注意力和门控特征注入，使高分辨率标记获得上下文信息。 除了架构创新之外，我们还提出了一种类似于SimMIM的预训练方法：我们在超高清图像令牌的75%上打上掩码，并在与UHR补丁空间对应的位置上的低分辨率中心区域进行掩码处理。然后，我们使用共享双编码器和小型解码器来重建初始的超高清图像。 在大规模IGN FLAIR-HUB航空数据集上进行了广泛的实验，证明了CASWiT的有效性。我们的方法实现了65.83%的mIoU（平均交并比），优于RGB基准模型1.78个百分点。在URUR数据集中，CASWiT达到了49.1%的mIoU，在官方评估协议下超越当前最优方案+0.9%。 所有代码均可在此URL上获取：[提供链接]
URL
https://arxiv.org/abs/2601.11310
