Semantic_Segmentation
-
AI-Based Culvert-Sewer Inspection
2026-01-21 16:33:33Christina ThrainerAbstract
Culverts and sewer pipes are critical components of drainage systems, and their failure can lead to serious risks to public safety and the environment. In this thesis, we explore methods to improve automated defect segmentation in culverts and sewer pipes. Collecting and annotating data in this field is cumbersome and requires domain knowledge. Having a large dataset for structural defect detection is therefore not feasible. Our proposed methods are tested under conditions with limited annotated data to demonstrate applicability to real-world scenarios. Overall, this thesis proposes three methods to significantly enhance defect segmentation and handle data scarcity. This can be addressed either by enhancing the training data or by adjusting a models architecture. First, we evaluate preprocessing strategies, including traditional data augmentation and dynamic label injection. These techniques significantly improve segmentation performance, increasing both Intersection over Union (IoU) and F1 score. Second, we introduce FORTRESS, a novel architecture that combines depthwise separable convolutions, adaptive Kolmogorov-Arnold Networks (KAN), and multi-scale attention mechanisms. FORTRESS achieves state-of-the-art performance on the culvert sewer pipe defect dataset, while significantly reducing the number of trainable parameters, as well as its computational cost. Finally, we investigate few-shot semantic segmentation and its applicability to defect detection. Few-shot learning aims to train models with only limited data available. By employing a bidirectional prototypical network with attention mechanisms, the model achieves richer feature representations and achieves satisfactory results across evaluation metrics.
Abstract (translated)
涵洞和污水管是排水系统中的关键组成部分，它们的故障可能会导致严重的公共安全和环境风险。在这篇论文中，我们探讨了改进涵洞及污水管道缺陷自动分割方法的方法。在这一领域收集并标注数据既费时又需要专业知识，因此建立一个用于结构缺陷检测的大规模数据集是不现实的。我们的研究方法是在有限的标注数据条件下进行测试，以展示其应用于实际场景的可能性。 总的来说，本文提出了三种显著提升缺陷分割性能并在面对数据稀缺问题时有效应对的方法，这些方法可以通过增强训练数据或者调整模型架构来实现。首先，我们评估了预处理策略的效果，包括传统的数据扩充以及动态标签注入等技术，这些技术能够显著提高分割效果，增加了交并比（IoU）和F1分数的值。 其次，本文介绍了FORTRESS这一全新架构，它结合了深度可分离卷积、自适应科莫格罗夫-阿诺尔德网络(KAN)以及多尺度注意力机制。FORTRESS在涵洞污水管道缺陷数据集上取得了最先进的性能，同时显著减少了训练参数的数量及其计算成本。 最后，我们探讨了少量样本语义分割技术的应用性，并将其应用于缺陷检测领域中。少量样本学习的目标是使用仅有的有限数量的数据来训练模型。通过采用双向原型网络和注意力机制，该模型能够获取更为丰富的特征表示，并在评价指标上取得了令人满意的结果。
URL
https://arxiv.org/abs/2601.15366
PDF
-
Erosion Attack for Adversarial Training to Enhance Semantic Segmentation Robustness
2026-01-21 12:52:09Yufei Song, Ziqi Zhou, Menghao Deng, Yifan Hu, Shengshan Hu, Minghui Li, Leo Yu ZhangAbstract
Existing segmentation models exhibit significant vulnerability to adversarial this http URL improve robustness, adversarial training incorporates adversarial examples into model training. However, existing attack methods consider only global semantic information and ignore contextual semantic relationships within the samples, limiting the effectiveness of adversarial training. To address this issue, we propose EroSeg-AT, a vulnerability-aware adversarial training framework that leverages EroSeg to generate adversarial examples. EroSeg first selects sensitive pixels based on pixel-level confidence and then progressively propagates perturbations to higher-confidence pixels, effectively disrupting the semantic consistency of the samples. Experimental results show that, compared to existing methods, our approach significantly improves attack effectiveness and enhances model robustness under adversarial training.
Abstract (translated)
现有的分割模型在对抗攻击下表现出显著的脆弱性。为了提高模型的鲁棒性，对抗训练（adversarial training）将对抗样本纳入到模型训练中。然而，现有的一些攻击方法仅考虑全局语义信息，而忽略了样本中的上下文语义关系，从而限制了对抗训练的有效性。 为了解决这一问题，我们提出了一种基于EroSeg的漏洞感知对抗训练框架（EroSeg-AT），该框架利用EroSeg生成对抗样本。EroSeg首先根据像素级别的置信度选择敏感像素，并逐步向高置信度像素传播扰动，从而有效地破坏样本的语义一致性。 实验结果表明，与现有方法相比，我们的方法在攻击效果和对抗训练下的模型鲁棒性方面均有显著提升。
URL
https://arxiv.org/abs/2601.14950
PDF
-
Context Patch Fusion With Class Token Enhancement for Weakly Supervised Semantic Segmentation
2026-01-21 07:12:23Yiyang Fu, Hui Li, Wangyu WuAbstract
Weakly Supervised Semantic Segmentation (WSSS), which relies only on image-level labels, has attracted significant attention for its cost-effectiveness and scalability. Existing methods mainly enhance inter-class distinctions and employ data augmentation to mitigate semantic ambiguity and reduce spurious activations. However, they often neglect the complex contextual dependencies among image patches, resulting in incomplete local representations and limited segmentation accuracy. To address these issues, we propose the Context Patch Fusion with Class Token Enhancement (CPF-CTE) framework, which exploits contextual relations among patches to enrich feature representations and improve segmentation. At its core, the Contextual-Fusion Bidirectional Long Short-Term Memory (CF-BiLSTM) module captures spatial dependencies between patches and enables bidirectional information flow, yielding a more comprehensive understanding of spatial correlations. This strengthens feature learning and segmentation robustness. Moreover, we introduce learnable class tokens that dynamically encode and refine class-specific semantics, enhancing discriminative capability. By effectively integrating spatial and semantic cues, CPF-CTE produces richer and more accurate representations of image content. Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 validate that CPF-CTE consistently surpasses prior WSSS methods.
Abstract (translated)
弱监督语义分割（WSSS）依赖于图像级别的标签，因其成本效益和可扩展性而引起了广泛关注。现有的方法主要通过增强类间区分度并采用数据增强技术来减轻语义模糊性和减少虚假激活。然而，它们往往忽视了图块之间的复杂上下文依赖关系，导致局部表示不完整且分割精度受限。为解决这些问题，我们提出了带有类别标记增强（CPF-CTE）的上下文图块融合框架，该框架利用图块之间的上下文关系来丰富特征表示并提高分割性能。在核心部分，上下文融合双向长短时记忆模块（Contextual-Fusion BiLSTM, CF-BiLSTM）捕获图块间的空间依赖性，并实现双向信息流动，从而对空间相关性提供更全面的理解。这增强了特征学习和分割的鲁棒性。此外，我们引入了可学习的类别标记，这些标记能够动态编码并细化特定类别的语义信息，提高了判别能力。通过有效整合空间和语义线索，CPF-CTE生成了更加丰富且准确的图像内容表示。 在PASCAL VOC 2012及MS COCO 2014数据集上的大量实验验证表明，CPF-CTE方法始终优于先前的WSSS方法。
URL
https://arxiv.org/abs/2601.14718
PDF
-
XD-MAP: Cross-Modal Domain Adaptation using Semantic Parametric Mapping
2026-01-20 21:00:26Frank Bieder, Hendrik K\"onigshof, Haohao Hu, Fabian Immel, Yinzhe Shen, Jan-Hendrik Pauls, Christoph StillerAbstract
Until open-world foundation models match the performance of specialized approaches, the effectiveness of deep learning models remains heavily dependent on dataset availability. Training data must align not only with the target object categories but also with the sensor characteristics and modalities. To bridge the gap between available datasets and deployment domains, domain adaptation strategies are widely used. In this work, we propose a novel approach to transferring sensor-specific knowledge from an image dataset to LiDAR, an entirely different sensing domain. Our method XD-MAP leverages detections from a neural network on camera images to create a semantic parametric map. The map elements are modeled to produce pseudo labels in the target domain without any manual annotation effort. Unlike previous domain transfer approaches, our method does not require direct overlap between sensors and enables extending the angular perception range from a front-view camera to a full 360 view. On our large-scale road feature dataset, XD-MAP outperforms single shot baseline approaches by +19.5 mIoU for 2D semantic segmentation, +19.5 PQth for 2D panoptic segmentation, and +32.3 mIoU in 3D semantic segmentation. The results demonstrate the effectiveness of our approach achieving strong performance on LiDAR data without any manual labeling.
Abstract (translated)
直到开放世界的基础模型能够达到专业化方法的性能，深度学习模型的有效性仍然很大程度上依赖于数据集的可用性。训练数据不仅要与目标对象类别对齐，还要与传感器特性和模式一致。为了弥合现有数据集和部署领域之间的差距，域适应策略被广泛应用。在这项工作中，我们提出了一种将特定传感器的知识从图像数据集中转移到LiDAR（一种完全不同的传感领域）的新型方法。我们的方法XD-MAP利用神经网络在摄像头图像上的检测结果创建语义参数化地图。该方法的地图元素可以生成目标域中的伪标签，而无需任何人工标注工作。与以往的域转换方法不同，我们的方法不需要传感器之间的直接重叠，并且能够将前视摄像头的角度感知范围扩展到360度全方位视角。在我们大规模的道路特征数据集上，XD-MAP在二维语义分割方面比单次基准方法高出+19.5 mIoU，在二维全景分割方面高出+19.5 PQth，在三维语义分割方面高出+32.3 mIoU。实验结果证明了我们的方法在不进行任何人工标注的情况下，对于LiDAR数据能够实现强大的性能。
URL
https://arxiv.org/abs/2601.14477
PDF
-
Atomic Depth Estimation From Noisy Electron Microscopy Data Via Deep Learning
2026-01-20 19:35:53Matan Leibovich, Mai Tan, Adria Marcos-Morales, Sreyas Mohan, Peter A. Crozier, Carlos Fernandez-GrandaAbstract
We present a novel approach for extracting 3D atomic-level information from transmission electron microscopy (TEM) images affected by significant noise. The approach is based on formulating depth estimation as a semantic segmentation problem. We address the resulting segmentation problem by training a deep convolutional neural network to generate pixel-wise depth segmentation maps using simulated data corrupted by synthetic noise. The proposed method was applied to estimate the depth of atomic columns in CeO2 nanoparticles from simulated images and real-world TEM data. Our experiments show that the resulting depth estimates are accurate, calibrated and robust to noise.
Abstract (translated)
URL
https://arxiv.org/abs/2601.17046
PDF
-
SUNSET -- A Sensor-fUsioN based semantic SegmEnTation exemplar for ROS-based self-adaptation
2026-01-20 08:40:57Andreas Wiedholz, Rafael Paintner, Julian Glei{\ss}ner, Alwin Hoffmann, Tobias HuberAbstract
The fact that robots are getting deployed more often in dynamic environments, together with the increasing complexity of their software systems, raises the need for self-adaptive approaches. In these environments robotic software systems increasingly operate amid (1) uncertainties, where symptoms are easy to observe but root causes are ambiguous, or (2) multiple uncertainties appear concurrently. We present SUNSET, a ROS2-based exemplar that enables rigorous, repeatable evaluation of architecture-based self-adaptation in such conditions. It implements a sensor fusion semantic-segmentation pipeline driven by a trained Machine Learning (ML) model whose input preprocessing can be perturbed to induce realistic performance degradations. The exemplar exposes five observable symptoms, where each can be caused by different root causes and supports concurrent uncertainties spanning self-healing and self-optimisation. SUNSET includes the segmentation pipeline, a trained ML model, uncertainty-injection scripts, a baseline controller, and step-by-step integration and evaluation documentation to facilitate reproducible studies and fair comparison.
Abstract (translated)
机器人在动态环境中部署得越来越频繁，加之其软件系统的复杂性不断增加，这提高了自适应方法的需求。在这种环境下，机器人的软件系统越来越多地在以下两种不确定性中运行：（1）不确定性存在于症状易于观察但根本原因模糊的情况；或者（2）多种不确定性同时出现。我们提出了SUNSET，这是一个基于ROS2的示例，旨在支持在这些条件下对架构驱动自适应进行严格且可重复的评估。该示例实现了一个由训练好的机器学习(ML)模型驱动的传感器融合语义分割流水线，并可通过扰动输入预处理来诱导现实中的性能下降。SUNSET展示了五种可观测的症状，每种症状可能有多种根本原因，并支持同时存在的不确定性和自我恢复及自我优化问题。该示例包含一个分割管道、一个训练好的ML模型、不确定性注入脚本、基线控制器以及详细的集成和评估文档，以促进可重复研究和公平比较。
URL
https://arxiv.org/abs/2601.13732
PDF
-
MANGO: A Global Single-Date Paired Dataset for Mangrove Segmentation
2026-01-20 05:11:32Junhyuk Heo, Beomkyu Choi, Hyunjin Shin, Darongsae KwonAbstract
Mangroves are critical for climate-change mitigation, requiring reliable monitoring for effective conservation. While deep learning has emerged as a powerful tool for mangrove detection, its progress is hindered by the limitations of existing datasets. In particular, many resources provide only annual map products without curated single-date image-mask pairs, limited to specific regions rather than global coverage, or remain inaccessible to the public. To address these challenges, we introduce MANGO, a large-scale global dataset comprising 42,703 labeled image-mask pairs across 124 countries. To construct this dataset, we retrieve all available Sentinel-2 imagery within the year 2020 for mangrove regions and select the best single-date observations that align with the mangrove annual mask. This selection is performed using a target detection-driven approach that leverages pixel-wise coordinate references to ensure adaptive and representative image-mask pairings. We also provide a benchmark across diverse semantic segmentation architectures under a country-disjoint split, establishing a foundation for scalable and reliable global mangrove monitoring.
Abstract (translated)
URL
https://arxiv.org/abs/2601.17039
PDF
-
Deep Learning for Semantic Segmentation of 3D Ultrasound Data
2026-01-19 17:54:59Chenyu Liu, Marco Cecotti, Harikrishnan Vijayakumar, Patrick Robinson, James Barson, Mihai CaleapAbstract
Developing cost-efficient and reliable perception systems remains a central challenge for automated vehicles. LiDAR and camera-based systems dominate, yet they present trade-offs in cost, robustness and performance under adverse conditions. This work introduces a novel framework for learning-based 3D semantic segmentation using Calyo Pulse, a modular, solid-state 3D ultrasound sensor system for use in harsh and cluttered environments. A 3D U-Net architecture is introduced and trained on the spatial ultrasound data for volumetric segmentation. Results demonstrate robust segmentation performance from Calyo Pulse sensors, with potential for further improvement through larger datasets, refined ground truth, and weighted loss functions. Importantly, this study highlights 3D ultrasound sensing as a promising complementary modality for reliable autonomy.
Abstract (translated)
开发高效且可靠的感知系统仍然是自动驾驶汽车面临的核心挑战。目前，基于LiDAR和摄像头的系统占据主导地位，但它们在成本、鲁棒性和恶劣条件下的性能方面存在权衡。本研究提出了一种新的基于学习的3D语义分割框架，使用Calyo Pulse——一种模块化、固态的3D超声波传感器系统，适用于严苛和复杂的环境。该研究引入并训练了一个3D U-Net架构，利用空间超声数据进行体积分割。实验结果表明，从Calyo Pulse传感器中可以实现稳健的语义分割性能，并且通过更大规模的数据集、更精确的真实标签以及加权损失函数的应用，有望进一步提升性能。尤为重要的是，本研究强调了3D超声波传感作为可靠自主驾驶的一种有前景的互补模式的重要性。
URL
https://arxiv.org/abs/2601.13263
PDF
-
GridNet-HD: A High-Resolution Multi-Modal Dataset for LiDAR-Image Fusion on Power Line Infrastructure
2026-01-19 13:41:52Antoine Carreaud, Shanci Li, Malo De Lacour, Digre Frinde, Jan Skaloud, Adrien GressinAbstract
This paper presents GridNet-HD, a multi-modal dataset for 3D semantic segmentation of overhead electrical infrastructures, pairing high-density LiDAR with high-resolution oblique imagery. The dataset comprises 7,694 images and 2.5 billion points annotated into 11 classes, with predefined splits and mIoU metrics. Unimodal (LiDAR-only, image-only) and multi-modal fusion baselines are provided. On GridNet-HD, fusion models outperform the best unimodal baseline by +5.55 mIoU, highlighting the complementarity of geometry and appearance. As reviewed in Sec. 2, no public dataset jointly provides high-density LiDAR and high-resolution oblique imagery with 3D semantic labels for power-line assets. Dataset, baselines, and codes are available: this https URL.
Abstract (translated)
本文介绍了GridNet-HD，这是一个用于三维语义分割的多模态数据集，专注于空中电力基础设施，并配对了高密度激光雷达与高分辨率倾斜影像。该数据集包含7,694张图像和25亿个点，被标记为11类，并且具有预定义的数据划分和mIoU度量标准。此外，还提供了单模态（仅激光雷达、仅图像）和多模态融合的基准模型。在GridNet-HD数据集上，融合模型相较于最佳单模态基线提高了+5.55 mIoU，这突显了几何信息与外观特征之间的互补性。根据第2节的回顾，目前没有公开的数据集同时提供高密度激光雷达和高分辨率倾斜影像以及针对电力线路资产的三维语义标签。数据集、基准模型和代码均可在以下网址获取：[此URL]。
URL
https://arxiv.org/abs/2601.13052
PDF
-
Cross-Scale Pretraining: Enhancing Self-Supervised Learning for Low-Resolution Satellite Imagery for Semantic Segmentation
2026-01-19 11:21:19John Waithaka, Gustave Bwirayesu, Moise BusogiAbstract
Self-supervised pretraining in remote sensing is mostly done using mid-spatial resolution (MR) image datasets due to their high availability. Given the release of high-resolution (HR) datasets, we ask how HR datasets can be included in self-supervised pretraining to enhance MR image representation learning and downstream segmentation performance on MR tasks. We design a spatial affinity component that can be added to existing self-supervised learning frameworks and that uses HR imagery to learn better representations of MR imagery. We test the spatial affinity component on two self-supervised learning frameworks and show that it outperforms models pretrained on HR or MR images alone.
Abstract (translated)
在遥感领域的自监督预训练中，通常使用的是中等空间分辨率（MR）图像数据集，因为它们的高可用性。鉴于高分辨率（HR）数据集的发布，我们探讨了如何将这些HR数据集纳入自监督预训练过程，以增强对MR图像表示的学习，并提升下游分割任务中的表现性能。为此，我们设计了一个可以添加到现有自监督学习框架中的空间亲和力组件，该组件利用HR影像来学习更好的MR影像表示。我们在两个自监督学习框架中测试了这个空间亲和力组件，并展示了它在仅使用HR或MR图像预训练的模型上表现更佳的结果。
URL
https://arxiv.org/abs/2601.12964
PDF
-
DepthCropSeg++: Scaling a Crop Segmentation Foundation Model With Depth-Labeled Data
2026-01-18 11:51:09Jiafei Zhang, Songliang Cao, Binghui Xu, Yanan Li, Weiwei Jia, Tingting Wu, Hao Lu, Weijuan Hu, Zhiguo HanAbstract
DepthCropSeg++: a foundation model for crop segmentation, capable of segmenting different crop species under open in-field environment. Crop segmentation is a fundamental task for modern agriculture, which closely relates to many downstream tasks such as plant phenotyping, density estimation, and weed control. In the era of foundation models, a number of generic large language and vision models have been developed. These models have demonstrated remarkable real world generalization due to significant model capacity and largescale datasets. However, current crop segmentation models mostly learn from limited data due to expensive pixel-level labelling cost, often performing well only under specific crop types or controlled environment. In this work, we follow the vein of our previous work DepthCropSeg, an almost unsupervised approach to crop segmentation, to scale up a cross-species and crossscene crop segmentation dataset, with 28,406 images across 30+ species and 15 environmental conditions. We also build upon a state-of-the-art semantic segmentation architecture ViT-Adapter architecture, enhance it with dynamic upsampling for improved detail awareness, and train the model with a two-stage selftraining pipeline. To systematically validate model performance, we conduct comprehensive experiments to justify the effectiveness and generalization capabilities across multiple crop datasets. Results demonstrate that DepthCropSeg++ achieves 93.11% mIoU on a comprehensive testing set, outperforming both supervised baselines and general-purpose vision foundation models like Segmentation Anything Model (SAM) by significant margins (+0.36% and +48.57% respectively). The model particularly excels in challenging scenarios including night-time environment (86.90% mIoU), high-density canopies (90.09% mIoU), and unseen crop varieties (90.09% mIoU), indicating a new state of the art for crop segmentation.
Abstract (translated)
DepthCropSeg++：一种作物分割的基础模型，在开放的田间环境中能够对不同种类的作物进行精确分割。作物分割是现代农业中的基本任务，与植物表型分析、密度估算和杂草控制等下游任务密切相关。在基础模型的时代，已经开发出了多种通用的大规模语言和视觉模型，这些模型由于拥有强大的模型容量和大规模的数据集，在现实世界中展示了出色的泛化能力。然而，目前的作物分割模型大多因为昂贵的像素级标注成本而依赖于有限的数据训练，通常只能针对特定类型的作物或受控环境表现出色。 在此项工作中，我们遵循了先前工作的DepthCropSeg路线——一种近乎无监督的方法进行作物分割，并通过构建一个涵盖30多种作物种类和15种不同环境条件的跨物种、跨场景的大规模数据集（共计28,406张图像）来扩展这种技术。此外，我们基于最先进的语义分割架构ViT-Adapter架构进行了改进，增强了动态上采样以提高细节识别能力，并通过两阶段自训练流水线进行模型训练。为了系统地验证模型性能，我们在多个作物数据集上开展了全面的实验，证明了其有效性和泛化能力。 结果显示，DepthCropSeg++在综合测试集上的mIoU（平均交并比）达到了93.11%，显著优于监督基准模型和通用视觉基础模型如Segmentation Anything Model (SAM)。特别是在夜间环境、高密度树冠以及未见过的作物品种等具有挑战性的场景中，DepthCropSeg++的表现尤为出色，分别达到86.90% mIoU、90.09% mIoU 和 90.09% mIoU，这标志着在作物分割领域达到了新的技术水平。
URL
https://arxiv.org/abs/2601.12366
PDF
-
Context-Aware Semantic Segmentation via Stage-Wise Attention
2026-01-16 14:06:46Antoine Carreaud, Elias Naha, Arthur Chansel, Nina Lahellec, Jan Skaloud, Adrien GressinAbstract
Semantic ultra high resolution image (UHR) segmentation is essential in remote sensing applications such as aerial mapping and environmental monitoring. Transformer-based models struggle in this setting because memory grows quadratically with token count, constraining either the contextual scope or the spatial resolution. We introduce CASWiT (Context-Aware Stage-Wise Transformer), a dual-branch, Swin-based architecture that injects global cues into fine-grained UHR features. A context encoder processes a downsampled neighborhood to capture long-range dependencies, while a high resolution encoder extracts detailed features from UHR patches. A cross-scale fusion module, combining cross-attention and gated feature injection, enriches high-resolution tokens with context. Beyond architecture, we propose a SimMIM-style pretraining. We mask 75% of the high-resolution image tokens and the low-resolution center region that spatially corresponds to the UHR patch, then train the shared dual-encoder with small decoder to reconstruct the UHR initial image. Extensive experiments on the large-scale IGN FLAIR-HUB aerial dataset demonstrate the effectiveness of CASWiT. Our method achieves 65.83% mIoU, outperforming RGB baselines by 1.78 points. On URUR, CASWiT achieves 49.1% mIoU, surpassing the current SoTA by +0.9% under the official evaluation protocol. All codes are provided on: this https URL.
Abstract (translated)
语义超高清（UHR）图像分割在遥感应用如航空测绘和环境监测中至关重要。然而，基于Transformer的模型在这种环境下表现不佳，因为内存需求随令牌数量呈二次增长，这限制了上下文范围或空间分辨率的大小。为此，我们提出了CASWiT（Context-Aware Stage-Wise Transformer），这是一种双分支、基于Swin的架构，它将全局线索注入到细粒度的UHR特征中。上下文编码器处理降采样的邻域以捕捉长距离依赖关系，而高分辨率编码器则从UHR补丁中提取详细特征。跨尺度融合模块结合了跨注意力和门控特征注入，使高分辨率标记获得上下文信息。 除了架构创新之外，我们还提出了一种类似于SimMIM的预训练方法：我们在超高清图像令牌的75%上打上掩码，并在与UHR补丁空间对应的位置上的低分辨率中心区域进行掩码处理。然后，我们使用共享双编码器和小型解码器来重建初始的超高清图像。 在大规模IGN FLAIR-HUB航空数据集上进行了广泛的实验，证明了CASWiT的有效性。我们的方法实现了65.83%的mIoU（平均交并比），优于RGB基准模型1.78个百分点。在URUR数据集中，CASWiT达到了49.1%的mIoU，在官方评估协议下超越当前最优方案+0.9%。 所有代码均可在此URL上获取：[提供链接]
URL
https://arxiv.org/abs/2601.11310
PDF
-
Graph Smoothing for Enhanced Local Geometry Learning in Point Cloud Analysis
2026-01-16 09:01:44Shangbo Yuan, Jie Xu, Ping Hu, Xiaofeng Zhu, Na ZhaoAbstract
Graph-based methods have proven to be effective in capturing relationships among points for 3D point cloud analysis. However, these methods often suffer from suboptimal graph structures, particularly due to sparse connections at boundary points and noisy connections in junction areas. To address these challenges, we propose a novel method that integrates a graph smoothing module with an enhanced local geometry learning module. Specifically, we identify the limitations of conventional graph structures, particularly in handling boundary points and junction areas. In response, we introduce a graph smoothing module designed to optimize the graph structure and minimize the negative impact of unreliable sparse and noisy connections. Based on the optimized graph structure, we improve the feature extract function with local geometry information. These include shape features derived from adaptive geometric descriptors based on eigenvectors and distribution features obtained through cylindrical coordinate transformation. Experimental results on real-world datasets validate the effectiveness of our method in various point cloud learning tasks, i.e., classification, part segmentation, and semantic segmentation.
Abstract (translated)
基于图的方法已被证明在捕捉三维点云分析中各点之间的关系方面非常有效。然而，这些方法往往因为边界点连接稀疏和结点区域连接嘈杂而产生次优的图结构问题。为了解决这些问题，我们提出了一种新方法，该方法集成了图平滑模块与增强型局部几何学习模块。 具体而言，我们指出了传统图结构在处理边界点和结点区域方面的局限性，并提出了一个专门优化图结构、最小化不可靠稀疏和嘈杂连接负面影响的图平滑模块。基于这个优化后的图结构，我们将特征提取功能改进为包含本地几何信息的形式。这些包括通过自适应的基于特征向量的几何描述符获得的形状特征以及通过圆柱坐标变换得到的分布特征。 实验结果在现实世界数据集上验证了我们方法在各种点云学习任务（如分类、部件分割和语义分割）中的有效性。
URL
https://arxiv.org/abs/2601.11102
PDF
-
A Confidence-Variance Theory for Pseudo-Label Selection in Semi-Supervised Learning
2026-01-16 02:51:59Jinshi Liu, Pan LiuAbstract
Most pseudo-label selection strategies in semi-supervised learning rely on fixed confidence thresholds, implicitly assuming that prediction confidence reliably indicates correctness. In practice, deep networks are often overconfident: high-confidence predictions can still be wrong, while informative low-confidence samples near decision boundaries are discarded. This paper introduces a Confidence-Variance (CoVar) theory framework that provides a principled joint reliability criterion for pseudo-label selection. Starting from the entropy minimization principle, we derive a reliability measure that combines maximum confidence (MC) with residual-class variance (RCV), which characterizes how probability mass is distributed over non-maximum classes. The derivation shows that reliable pseudo-labels should have both high MC and low RCV, and that the influence of RCV increases as confidence grows, thereby correcting overconfident but unstable predictions. From this perspective, we cast pseudo-label selection as a spectral relaxation problem that maximizes separability in a confidence-variance feature space, and design a threshold-free selection mechanism to distinguish high- from low-reliability predictions. We integrate CoVar as a plug-in module into representative semi-supervised semantic segmentation and image classification methods. Across PASCAL VOC 2012, Cityscapes, CIFAR-10, and Mini-ImageNet with varying label ratios and backbones, it consistently improves over strong baselines, indicating that combining confidence with residual-class variance provides a more reliable basis for pseudo-label selection than fixed confidence thresholds. (Code: this https URL)
Abstract (translated)
半监督学习中大多数伪标签选择策略依赖于固定的置信度阈值，暗含地假设预测的置信度可靠地指示了正确性。在实践中，深度网络往往过于自信：即使高置信度的预测也可能出错，而位于决策边界附近的低置信度但信息量大的样本则被丢弃。本文介绍了一种基于置信度-方差（Confidence-Variance, CoVar）理论框架，该框架提供了一个原则性的联合可靠性标准用于伪标签选择。从熵最小化原理出发，我们推导出一种结合最大置信度（MC）和残余类方差（RCV）的可靠性测量方法，其中RCV描述了概率质量在非最大类中的分布情况。 我们的推导表明，可靠的伪标签应具有高MC和低RCV，并且随着置信度的增长，RCV的影响增大，从而纠正过于自信但不稳定的预测。从这个角度来看，我们将伪标签选择视为一种谱松弛问题，在置信度-方差特征空间中最大化可分离性，并设计了一种无需阈值的机制来区分高可靠性与低可靠性预测。 我们将CoVar作为插件模块整合进代表性的半监督语义分割和图像分类方法中。在PASCAL VOC 2012、Cityscapes、CIFAR-10和Mini-ImageNet数据集上，通过不同的标签比例和骨干网络进行实验，结果表明：CoVar的一致性改进了强大的基准模型，这表明结合置信度与残余类方差为伪标签选择提供了比固定置信度阈值更可靠的依据。 (代码地址: this https URL)
URL
https://arxiv.org/abs/2601.11670
PDF
-
Urban Socio-Semantic Segmentation with Vision-Language Reasoning
2026-01-15 15:00:36Yu Wang, Yi Wang, Rui Dai, Yujie Wang, Kaikui Liu, Xiangxiang Chu, Yansheng LiAbstract
As hubs of human activity, urban surfaces consist of a wealth of semantic entities. Segmenting these various entities from satellite imagery is crucial for a range of downstream applications. Current advanced segmentation models can reliably segment entities defined by physical attributes (e.g., buildings, water bodies) but still struggle with socially defined categories (e.g., schools, parks). In this work, we achieve socio-semantic segmentation by vision-language model reasoning. To facilitate this, we introduce the Urban Socio-Semantic Segmentation dataset named SocioSeg, a new resource comprising satellite imagery, digital maps, and pixel-level labels of social semantic entities organized in a hierarchical structure. Additionally, we propose a novel vision-language reasoning framework called SocioReasoner that simulates the human process of identifying and annotating social semantic entities via cross-modal recognition and multi-stage reasoning. We employ reinforcement learning to optimize this non-differentiable process and elicit the reasoning capabilities of the vision-language model. Experiments demonstrate our approach's gains over state-of-the-art models and strong zero-shot generalization. Our dataset and code are available in this https URL.
Abstract (translated)
作为人类活动的中心，城市表面包含了大量的语义实体。从卫星图像中分割这些不同的实体对于各种下游应用至关重要。目前先进的分割模型可以可靠地分割由物理属性定义的实体（如建筑物、水域），但对于社会定义的类别（如学校、公园）仍然存在困难。在这项工作中，我们通过视觉-语言模型推理实现了社会语义分割。为此，我们引入了名为SocioSeg的城市社会语义分割数据集，这是一个包含卫星图像、数字地图和按层次结构组织的社会语义实体像素级标签的新资源。 此外，我们提出了一种新颖的视觉-语言推理框架，称为SocioReasoner，该框架通过跨模态识别和多阶段推理来模拟人类识别并标注社会语义实体的过程。我们利用强化学习优化这个非微分过程，并激发了视觉-语言模型的推理能力。实验表明，我们的方法在最先进的模型上表现出色，并具有强大的零样本泛化性能。 我们的数据集和代码可在以下网址获取：[此URL]。
URL
https://arxiv.org/abs/2601.10477
PDF
-
SAM-Aug: Leveraging SAM Priors for Few-Shot Parcel Segmentation in Satellite Time Series
2026-01-14 03:18:04Kai Hu, Yaozu Feng, Vladimir Lysenko, Ya Guo Member, Huayi WuAbstract
Few-shot semantic segmentation of time-series remote sensing images remains a critical challenge, particularly in regions where labeled data is scarce or costly to obtain. While state-of-the-art models perform well under full supervision, their performance degrades significantly under limited labeling, limiting their real-world applicability. In this work, we propose SAM-Aug, a new annotation-efficient framework that leverages the geometry-aware segmentation capability of the Segment Anything Model (SAM) to improve few-shot land cover mapping. Our approach constructs cloud-free composite images from temporal sequences and applies SAM in a fully unsupervised manner to generate geometry-aware mask priors. These priors are then integrated into training through a proposed loss function called RegionSmoothLoss, which enforces prediction consistency within each SAM-derived region across temporal frames, effectively regularizing the model to respect semantically coherent structures. Extensive experiments on the PASTIS-R benchmark under a 5 percent labeled setting demonstrate the effectiveness and robustness of SAM-Aug. Averaged over three random seeds (42, 2025, 4090), our method achieves a mean test mIoU of 36.21 percent, outperforming the state-of-the-art baseline by +2.33 percentage points, a relative improvement of 6.89 percent. Notably, on the most favorable split (seed=42), SAM-Aug reaches a test mIoU of 40.28 percent, representing an 11.2 percent relative gain with no additional labeled data. The consistent improvement across all seeds confirms the generalization power of leveraging foundation model priors under annotation scarcity. Our results highlight that vision models like SAM can serve as useful regularizers in few-shot remote sensing learning, offering a scalable and plug-and-play solution for land cover monitoring without requiring manual annotations or model fine-tuning.
Abstract (translated)
时间序列遥感图像的少样本语义分割仍然是一个关键挑战，尤其是在标记数据稀缺或获取成本高昂的地区。尽管最先进的模型在完全监督下表现良好，但它们在标注受限条件下的性能显著下降，限制了其实用性。在这项工作中，我们提出了SAM-Aug，这是一种新的注释高效框架，它利用Segment Anything Model（SAM）的几何感知分割能力来改进少样本土地覆盖制图。我们的方法通过从时间序列中构建无云合成图像，并以完全无监督的方式应用SAM生成几何感知掩模先验来实现这一点。然后通过引入名为RegionSmoothLoss的损失函数将这些先验集成到训练中，该函数强制每个SAM导出区域在时间帧之间保持预测一致性，从而有效规范模型尊重语义一致结构。 在PASTIS-R基准测试中的5%标注设置下进行了广泛的实验，证明了SAM-Aug的有效性和鲁棒性。经过三个随机种子（42, 2025, 4090）的平均测试mIoU为36.21%，比最先进的基线高出+2.33个百分点，相对改进率为6.89%。值得注意的是，在最有利分割（seed=42）中，SAM-Aug达到了40.28%的测试mIoU，代表了无额外标注数据的情况下11.2%的相对增益。所有种子上的持续改进证实了在标记稀缺条件下利用基础模型先验的一般化能力。 我们的结果表明，像SAM这样的视觉模型可以作为少样本遥感学习中的有用正则化器，在无需人工注释或模型微调的情况下提供可扩展且即插即用的土地覆盖监测解决方案。
URL
https://arxiv.org/abs/2601.09110
PDF
-
Vision Foundation Models for Domain Generalisable Cross-View Localisation in Planetary Ground-Aerial Robotic Teams
2026-01-14 03:11:05Lachlan Holden, Feras Dayoub, Alberto Candela, David Harvey, Tat-Jun ChinAbstract
Accurate localisation in planetary robotics enables the advanced autonomy required to support the increased scale and scope of future missions. The successes of the Ingenuity helicopter and multiple planetary orbiters lay the groundwork for future missions that use ground-aerial robotic teams. In this paper, we consider rovers using machine learning to localise themselves in a local aerial map using limited field-of-view monocular ground-view RGB images as input. A key consideration for machine learning methods is that real space data with ground-truth position labels suitable for training is scarce. In this work, we propose a novel method of localising rovers in an aerial map using cross-view-localising dual-encoder deep neural networks. We leverage semantic segmentation with vision foundation models and high volume synthetic data to bridge the domain gap to real images. We also contribute a new cross-view dataset of real-world rover trajectories with corresponding ground-truth localisation data captured in a planetary analogue facility, plus a high volume dataset of analogous synthetic image pairs. Using particle filters for state estimation with the cross-view networks allows accurate position estimation over simple and complex trajectories based on sequences of ground-view images.
Abstract (translated)
行星机器人中的精确定位能够实现未来任务扩大规模和范围所需的高度自主性。Ingenuity直升机和多个环绕器的成功为未来使用地面-空中机器人团队的任务奠定了基础。在本文中，我们考虑了一种方法，即利用有限视野的单目地面RGB图像作为输入，通过机器学习使探测车能够在局部的空中地图上定位自己。 对于机器学习方法而言，关键挑战在于真实空间数据及其带有位置标签的数据量较少，这些标签适用于训练过程。为此，在本工作中，我们提出了一种新的方法，即利用跨视图定位双编码深度神经网络来对在空中地图中进行探测车的定位。我们采用语义分割和视觉基础模型，并使用大量合成数据来弥合与真实图像之间的领域差距。此外，我们还贡献了一个新建立的真实世界数据集，包括在行星类地设施中捕获的实际轨迹及其相应的精确位置标签，以及大量的类似合成图像对的数据集。 利用粒子滤波器进行状态估计，并结合跨视图网络，可以基于一系列地面视角的图像实现简单和复杂轨迹上的准确位置估算。
URL
https://arxiv.org/abs/2601.09107
PDF
-
DentalX: Context-Aware Dental Disease Detection with Radiographs
2026-01-13 18:32:28Zhi Qin Tan, Xiatian Zhu, Owen Addison, Yunpeng LiAbstract
Diagnosing dental diseases from radiographs is time-consuming and challenging due to the subtle nature of diagnostic evidence. Existing methods, which rely on object detection models designed for natural images with more distinct target patterns, struggle to detect dental diseases that present with far less visual support. To address this challenge, we propose {\bf DentalX}, a novel context-aware dental disease detection approach that leverages oral structure information to mitigate the visual ambiguity inherent in radiographs. Specifically, we introduce a structural context extraction module that learns an auxiliary task: semantic segmentation of dental anatomy. The module extracts meaningful structural context and integrates it into the primary disease detection task to enhance the detection of subtle dental diseases. Extensive experiments on a dedicated benchmark demonstrate that DentalX significantly outperforms prior methods in both tasks. This mutual benefit arises naturally during model optimization, as the correlation between the two tasks is effectively captured. Our code is available at this https URL.
Abstract (translated)
从放射影像诊断牙科疾病是一项耗时且具有挑战性的任务，因为诊断证据往往非常细微。现有的方法依赖于为自然图像设计的对象检测模型，这些模型在目标模式更为明显的场景下表现良好，但在面对牙科疾病这样视觉支持较少的情况下则难以有效检测出病症。为了应对这一挑战，我们提出了{\bf DentalX}，这是一种新型的基于上下文感知的牙科疾病检测方法，利用口腔结构信息来减轻放射影像中固有的视觉模糊性。 具体而言，DentalX引入了一个结构上下文提取模块，该模块学习一个辅助任务：牙科解剖学的语义分割。这个模块可以提取有意义的结构上下文，并将其整合到主要的疾病检测任务中，从而增强对细微牙科疾病的检测能力。在专门为此设计的数据集上进行的大量实验表明，DentalX在这两项任务上的表现都显著优于以前的方法。这种互相增益的效果自然地出现在模型优化过程中，因为该方法能够有效捕捉两个任务之间的相关性。 我们的代码可在此网址获取：[此链接]（请提供实际的有效链接）。
URL
https://arxiv.org/abs/2601.08797
PDF
-
WaveFormer: Frequency-Time Decoupled Vision Modeling with Wave Equation
2026-01-13 14:47:22Zishan Shu, Juntong Wu, Wei Yan, Xudong Liu, Hongyu Zhang, Chang Liu, Youdong Mao, Jie ChenAbstract
Vision modeling has advanced rapidly with Transformers, whose attention mechanisms capture visual dependencies but lack a principled account of how semantic information propagates spatially. We revisit this problem from a wave-based perspective: feature maps are treated as spatial signals whose evolution over an internal propagation time (aligned with network depth) is governed by an underdamped wave equation. In this formulation, spatial frequency-from low-frequency global layout to high-frequency edges and textures-is modeled explicitly, and its interaction with propagation time is controlled rather than implicitly fixed. We derive a closed-form, frequency-time decoupled solution and implement it as the Wave Propagation Operator (WPO), a lightweight module that models global interactions in O(N log N) time-far lower than attention. Building on WPO, we propose a family of WaveFormer models as drop-in replacements for standard ViTs and CNNs, achieving competitive accuracy across image classification, object detection, and semantic segmentation, while delivering up to 1.6x higher throughput and 30% fewer FLOPs than attention-based alternatives. Furthermore, our results demonstrate that wave propagation introduces a complementary modeling bias to heat-based methods, effectively capturing both global coherence and high-frequency details essential for rich visual semantics. Codes are available at: this https URL.
Abstract (translated)
视觉模型在Transformer架构的推动下取得了快速进展，但其注意机制虽能捕捉视觉依赖关系，却缺乏对语义信息如何在空间上传播的原理性解释。我们从波的观点重新审视这个问题：将特征图视为空间信号，并将其随内部传播时间（与网络深度相一致）演化的规律用一个欠阻尼波动方程来描述。在这种框架下，空间频率——从低频全局布局到高频边缘和纹理——被明确建模，且其与传播时间的相互作用不再是隐式固定的，而是可以控制的。 我们推导出了一种闭合形式、频率-时间解耦的解决方案，并将其实现为波传播算子（WPO），这是一种轻量级模块，在O(N log N)时间内模型全局交互，远低于注意力机制。基于WPO，我们提出了一系列WaveFormer模型作为标准ViTs和CNNs的直接替代品，在图像分类、目标检测和语义分割等任务上达到了竞争性的准确率，并且相比基于注意的方法提供了最高达1.6倍的吞吐量和30%更少的FLOPs。此外，我们的结果表明波传播为热基方法引入了一种互补建模偏差，有效捕捉了全局一致性及高频细节，这对丰富视觉语义至关重要。 代码可在以下网址获取：[请参考原文链接以查看准确URL]
URL
https://arxiv.org/abs/2601.08602
PDF
-
Source-Free Domain Adaptation for Geospatial Point Cloud Semantic Segmentation
2026-01-13 09:37:57Yuan Gao, Di Cao, Xiaohuan Xi, Sheng Nie, Shaobo Xia, Cheng WangAbstract
Semantic segmentation of 3D geospatial point clouds is pivotal for remote sensing applications. However, variations in geographic patterns across regions and data acquisition strategies induce significant domain shifts, severely degrading the performance of deployed models. Existing domain adaptation methods typically rely on access to source-domain data. However, this requirement is rarely met due to data privacy concerns, regulatory policies, and data transmission limitations. This motivates the largely underexplored setting of source-free unsupervised domain adaptation (SFUDA), where only a pretrained model and unlabeled target-domain data are available. In this paper, we propose LoGo (Local-Global Dual-Consensus), a novel SFUDA framework specifically designed for geospatial point clouds. At the local level, we introduce a class-balanced prototype estimation module that abandons conventional global threshold filtering in favor of an intra-class independent anchor mining strategy. This ensures that robust feature prototypes can be generated even for sample-scarce tail classes, effectively mitigating the feature collapse caused by long-tailed distributions. At the global level, we introduce an optimal transport-based global distribution alignment module that formulates pseudo-label assignment as a global optimization problem. By enforcing global distribution constraints, this module effectively corrects the over-dominance of head classes inherent in local greedy assignments, preventing model predictions from being severely biased towards majority classes. Finally, we propose a dual-consistency pseudo-label filtering mechanism. This strategy retains only high-confidence pseudo-labels where local multi-augmented ensemble predictions align with global optimal transport assignments for self-training.
Abstract (translated)
三维地理空间点云的语义分割对于遥感应用至关重要。然而，不同地区的地理模式变化和数据采集策略导致了显著的领域偏移（domain shift），这严重降低了已部署模型的表现能力。现有的领域适应方法通常依赖于源域数据的访问权限。但由于数据隐私问题、监管政策以及数据传输限制，这一要求往往难以满足。因此，研究者们开始探索无需源域监督信息的无监督领域自适应（Source-Free Unsupervised Domain Adaptation, SFUDA）场景，在这种场景下，仅有预训练模型和未标注的目标区域数据可用。 本文提出了一种新的SFUDA框架——LoGo（Local-Global Dual-Consensus），专门针对地理空间点云。在局部层面，我们引入了一个平衡的原型估计模块，放弃了传统的全局阈值过滤方法，转而采用一种基于类内独立锚点挖掘策略的方法。这确保了即使对于样本稀缺的小类别，也能生成稳健的特征原型，从而有效地缓解由长尾分布带来的特征坍缩问题。 在全局层面，我们引入了一个基于最优传输（optimal transport）的全局分布对齐模块，该模块将伪标签分配形式化为一个全局优化问题。通过施加全局分布约束，这一模块能够有效纠正局部贪婪分配中头部类别占主导的问题，防止模型预测严重偏向多数类。 最后，我们提出了一种双一致性伪标签过滤机制。这种方法保留了那些在多增广集成预测与全球最优传输分配一致的高置信度伪标签用于自我训练。
URL
https://arxiv.org/abs/2601.08375
PDF
