Semantic_Segmentation
-
Progressive Prompt-Guided Cross-Modal Reasoning for Referring Image Segmentation
2026-03-30 03:33:10Jiachen Li, Hongyun Wang, Jinyu Xu, Wenbo Jiang, Yanchun Ma, Yongjian Liu, Qing Xie, Bolong ZhengAbstract
Referring image segmentation aims to localize and segment a target object in an image based on a free-form referring expression. The core challenge lies in effectively bridging linguistic descriptions with object-level visual representations, especially when referring expressions involve detailed attributes and complex inter-object relationships. Existing methods either rely on cross-modal alignment or employ Semantic Segmentation Prompts, but they often lack explicit reasoning mechanisms for grounding language descriptions to target regions in the image. To address these limitations, we propose PPCR, a Progressive Prompt-guided Cross-modal Reasoning framework for referring image segmentation. PPCR explicitly structures the reasoning process as a Semantic Understanding-Spatial Grounding-Instance Segmentation pipeline. Specifically, PPCR first employs multimodal large language models (MLLMs) to generate Semantic Segmentation Prompt that capture key semantic cues of the target object. Based on this semantic context, Spatial Segmentation Prompt are further generated to reason about object location and spatial extent, enabling a progressive transition from semantic understanding to spatial grounding. The Semantic and Spatial Segmentation prompts are then jointly integrated into the segmentation module to guide accurate target localization and segmentation. Extensive experiments on standard referring image segmentation benchmarks demonstrate that PPCR consistently outperforms existing methods. The code will be publicly released to facilitate reproducibility.
Abstract (translated)
URL
https://arxiv.org/abs/2603.27993
PDF
-
A Cross-Scale Decoder with Token Refinement for Off-Road Semantic Segmentation
2026-03-30 00:59:22Seongkyu Choi Jhonghyun AnAbstract
Off-road semantic segmentation is fundamentally challenged by irregular terrain, vegetation clutter, and inherent annotation ambiguity. Unlike urban scenes with crisp object boundaries, off-road environments exhibit strong class-level similarity among terrain categories, resulting in thick and uncertain transition regions that degrade boundary coherence and destabilize training. Rare or thin structures, such as narrow traversable gaps or isolated obstacles, further receive sparse and unreliable supervision and are easily overwhelmed by dominant background textures. Existing decoder designs either rely on low-scale bottlenecks that oversmooth fine structural details, or repeatedly fuse high-detail features, which tends to amplify annotation noise and incur substantial computational cost. We present a cross-scale decoder that explicitly addresses these challenges through three complementary mechanisms. First, a global--local token refinement module consolidates semantic context on a compact bottleneck lattice, guided by boundary-aware regularization to remain robust under ambiguous supervision. Second, a gated detail bridge selectively injects fine-scale structural cues only once through cross-scale attention, preserving boundary and texture information while avoiding noise accumulation. Third, an uncertainty-guided class-aware point refinement selectively updates the least reliable pixels, improving rare and ambiguous structures with minimal computational overhead. The resulting framework achieves noise-robust and boundary-preserving segmentation tailored to off-road environments, recovering fine structural details while maintaining deployment-friendly efficiency. Experimental results on standard off-road benchmarks demonstrate consistent improvements over prior approaches without resorting to heavy dense feature fusion.
Abstract (translated)
越野场景语义分割面临不规则地形、植被杂乱及固有标注模糊等根本性挑战。与具有清晰物体边界的城市场景不同，越野环境中地形类别间存在强烈的类间相似性，导致模糊且不确定的过渡区域，这削弱了边界连贯性并 destabilizes 训练过程。稀有或细长结构（如狭窄可通行间隙或孤立障碍物）进一步受到稀疏且不可靠的监督，并容易被主导性背景纹理淹没。现有解码器设计要么依赖会过度平滑精细结构细节的低尺度瓶颈，要么反复融合高细节特征，这往往会放大标注噪声并产生巨大的计算成本。我们提出了一种跨尺度解码器，通过三种互补机制明确应对这些挑战。首先，全局-局部标记细化模块在紧凑的瓶颈栅格上整合语义上下文，由边界感知正则化引导，以在模糊监督下保持鲁棒性。其次，门控细节桥接通过跨尺度注意力仅注入一次细尺度结构线索，在保留边界和纹理信息的同时避免噪声累积。第三，不确定性引导的类感知点细化选择性地更新最不可靠的像素，以最小的计算开销改善稀有和模糊结构。所得框架实现了针对越野环境的噪声鲁棒且边界保持的分割，在保持部署友好效率的同时恢复了精细结构细节。在标准越野基准上的实验结果表明，无需依赖沉重的密集特征融合，该方法即持续优于先前方法。
URL
https://arxiv.org/abs/2603.27931
PDF
-
ForestSim: A Synthetic Benchmark for Intelligent Vehicle Perception in Unstructured Forest Environments
2026-03-30 00:43:09Pragat Wagle, Zheng Chen, Lantao LiuAbstract
Robust scene understanding is essential for intelligent vehicles operating in natural, unstructured environments. While semantic segmentation datasets for structured urban driving are abundant, the datasets for extremely unstructured wild environments remain scarce due to the difficulty and cost of generating pixel-accurate annotations. These limitations hinder the development of perception systems needed for intelligent ground vehicles tasked with forestry automation, agricultural robotics, disaster response, and all-terrain mobility. To address this gap, we present ForestSim, a high-fidelity synthetic dataset designed for training and evaluating semantic segmentation models for intelligent vehicles in forested off-road and no-road environments. ForestSim contains 2094 photorealistic images across 25 diverse environments, covering multiple seasons, terrain types, and foliage densities. Using Unreal Engine environments integrated with Microsoft AirSim, we generate consistent, pixel-accurate labels across 20 classes relevant to autonomous navigation. We benchmark ForestSim using state-of-the-art architectures and report strong performance despite the inherent challenges of unstructured scenes. ForestSim provides a scalable and accessible foundation for perception research supporting the next generation of intelligent off-road vehicles. The dataset and code are publicly available: Dataset: this https URL Code: this https URL
Abstract (translated)
鲁棒的环境理解能力对于在自然、非结构化环境中运行的智能车辆至关重要。尽管针对结构化城市驾驶的语义分割数据集已相当丰富，但针对极端非结构化野外环境的数据集仍然稀缺，这源于生成像素级精确标注的困难与高昂成本。这些限制阻碍了智能地面车辆在林业自动化、农业机器人、灾害响应及全地形机动等任务中所需的感知系统发展。为填补这一空白，我们推出了ForestSim——一个专为森林越野及无道路环境中智能车辆训练与评估语义分割模型而设计的高保真合成数据集。ForestSim包含涵盖25种多样环境的2094张照片级真实图像，覆盖多种季节、地形类型和植被密度。通过集成Microsoft AirSim的Unreal Engine环境，我们生成了与20个自主导航相关类别一致的像素级精确标注。我们使用最先进的架构在ForestSim上进行基准测试，报告了在非结构化场景固有挑战下仍表现优异的性能。ForestSim为支持下一代智能越野车辆的感知研究提供了可扩展且易用的基础。数据集及代码已公开：数据集：此链接 代码：此链接
URL
https://arxiv.org/abs/2603.27923
PDF
-
Towards Context-Aware Image Anonymization with Multi-Agent Reasoning
2026-03-29 19:06:43Robert Aufschl\"ager, Jakob Folz, Gautam Savaliya, Manjitha D Vidanalage, Michael Heigl, Martin SchrammAbstract
Street-level imagery contains personally identifiable information (PII), some of which is context-dependent. Existing anonymization methods either over-process images or miss subtle identifiers, while API-based solutions compromise data sovereignty. We present an agentic framework CAIAMAR (\underline{C}ontext-\underline{A}ware \underline{I}mage \underline{A}nonymization with \underline{M}ulti-\underline{A}gent \underline{R}easoning) for context-aware PII segmentation with diffusion-based anonymization, combining pre-defined processing for high-confidence cases with multi-agent reasoning for indirect identifiers. Three specialized agents coordinate via round-robin speaker selection in a Plan-Do-Check-Act (PDCA) cycle, enabling large vision-language models to classify PII based on spatial context (private vs. public property) rather than rigid category rules. The agents implement spatially-filtered coarse-to-fine detection where a scout-and-zoom strategy identifies candidates, open-vocabulary segmentation processes localized crops, and $IoU$-based deduplication ($30\%$ threshold) prevents redundant processing. Modal-specific diffusion guidance with appearance decorrelation substantially reduces re-identification (Re-ID) risks. On CUHK03-NP, our method reduces person Re-ID risk by $73\%$ ($R1$: $16.9\%$ vs. $62.4\%$ baseline). For image quality preservation on CityScapes, we achieve KID: $0.001$, and FID: $9.1$, significantly outperforming existing anonymization. The agentic workflow detects non-direct PII instances across object categories, and downstream semantic segmentation is preserved. Operating entirely on-premise with open-source models, the framework generates human-interpretable audit trails supporting EU's GDPR transparency requirements while flagging failed cases for human review.
Abstract (translated)
街景图像包含个人身份信息（PII），其中部分信息的识别依赖于具体情境。现有匿名化方法要么对图像过度处理，要么遗漏细微标识，而基于API的方案则损害数据主权。本研究提出一种智能体框架CAIAMAR（上下文感知图像匿名化多智能体推理），通过基于扩散的匿名化实现上下文感知的PII分割，结合高置信度案例的预定义处理与针对间接标识的多智能体推理。三个专业化智能体通过PDCA循环中的轮询发言选择机制协同工作，使大视觉语言模型能够基于空间上下文（私有与公共财产）而非僵化的类别规则对PII进行分类。智能体实施空间过滤的粗到精检测策略：通过“侦察-放大”识别候选区域，开放词汇分割处理局部裁剪，并采用基于交并比（IoU）的去重（30%阈值）避免冗余处理。模态特定的扩散引导与外观去关联显著降低了重识别（Re-ID）风险。在CUHK03-NP数据集上，该方法将行人重识别风险降低73%（R1指标：16.9% vs. 基线62.4%）。在CityScapes数据集上，图像质量保持达到KID：0.001、FID：9.1，显著优于现有匿名化方法。该智能体工作流可跨对象类别检测非直接PII实例，且下游语义分割任务得以保留。框架完全基于本地部署并使用开源模型，生成人类可读的审计追踪以满足欧盟GDPR透明度要求，同时标记失败案例供人工复核。
URL
https://arxiv.org/abs/2603.27817
PDF
-
E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences
2026-03-29 16:34:12Biswadeep Sen, Benoit R. Cottereau, Nicolas Cuperlier, Terence SimAbstract
Event-based cameras capture visual information as asynchronous streams of per-pixel brightness changes, generating sparse, temporally precise data. Compared to conventional frame-based sensors, they offer significant advantages in capturing high-speed dynamics while consuming substantially less power. Predicting future event representations from past observations is an important problem, enabling downstream tasks such as future semantic segmentation or object tracking without requiring access to future sensor measurements. While recent state-of-the-art approaches achieve strong performance, they often rely on computationally heavy backbones and, in some cases, large-scale pretraining, limiting their applicability in resource-constrained scenarios. In this work, we introduce E-TIDE, a lightweight, end-to-end trainable architecture for event-tensor prediction that is designed to operate efficiently without large-scale pretraining. Our approach employs the TIDE module (Temporal Interaction for Dynamic Events), motivated by efficient spatiotemporal interaction design for sparse event tensors, to capture temporal dependencies via large-kernel mixing and activity-aware gating while maintaining low computational complexity. Experiments on standard event-based datasets demonstrate that our method achieves competitive performance with significantly reduced model size and training requirements, making it well-suited for real-time deployment under tight latency and memory budgets.
Abstract (translated)
事件相机通过异步流的形式捕获每个像素亮度变化，生成稀疏且时间精确的数据。与传统的帧式传感器相比，它们在捕捉高速动态场景时具有显著优势，同时功耗大幅降低。基于历史观测预测未来事件表示是一个重要问题，它使得未来语义分割或目标跟踪等下游任务无需访问未来传感器测量值即可进行。尽管近期最先进的方法取得了优异性能，但它们通常依赖计算量庞大的骨干网络，部分还需大规模预训练，这限制了其在资源受限场景中的应用。本研究提出E-TIDE，一种轻量级、端到端可训练的事件张量预测架构，专为无需大规模预训练的高效运行而设计。该方法采用TIDE模块（时间交互动态事件模块），其动机在于针对稀疏事件张量设计高效时空交互，通过大核混合和感知活动门控来捕获时间依赖关系，同时保持低计算复杂度。在标准事件数据集上的实验表明，我们的方法在模型规模和训练需求显著减少的情况下仍能达到具有竞争力的性能，非常适合在严格延迟和内存预算下进行实时部署。
URL
https://arxiv.org/abs/2603.27757
PDF
-
Can Unsupervised Segmentation Reduce Annotation Costs for Video Semantic Segmentation?
2026-03-29 13:51:56Samik Some, Vinay P. NamboodiriAbstract
Present-day deep neural networks for video semantic segmentation require a large number of fine-grained pixel-level annotations to achieve the best possible results. Obtaining such annotations, however, is very expensive. On the other hand, raw, unannotated video frames are practically free to obtain. Similarly, coarse annotations, which do not require precise boundaries, are also much cheaper. This paper investigates approaches to reduce the annotation cost required for video segmentation datasets by utilising such resources. We show that using state-of-the-art segmentation foundation models, Segment Anything Model (SAM) and Segment Anything Model 2 (SAM 2), we can utilise both unannotated frames as well as coarse annotations to alleviate the effort required for manual annotation of video segmentation datasets by automating mask generation. Our investigation suggests that if used appropriately, we can reduce the need for annotation by a third with similar performance for video semantic segmentation. More significantly, our analysis suggests that the variety of frames in the dataset is more important than the number of frames for obtaining the best performance.
Abstract (translated)
URL
https://arxiv.org/abs/2603.27697
PDF
-
Transferring Physical Priors into Remote Sensing Segmentation via Large Language Models
2026-03-29 03:55:11Yuxi Lu, Kunqi Li, Zhidong Li, Xiaohan Su, Biao Wu, Chenya Huang, Bin LiangAbstract
Semantic segmentation of remote sensing imagery is fundamental to Earth observation. Achieving accurate results requires integrating not only optical images but also physical variables such as the Digital Elevation Model (DEM), Synthetic Aperture Radar (SAR) and Normalized Difference Vegetation Index (NDVI). Recent foundation models (FMs) leverage pre-training to exploit these variables but still depend on spatially aligned data and costly retraining when involving new sensors. To overcome these limitations, we introduce a novel paradigm for integrating domain-specific physical priors into segmentation models. We first construct a Physical-Centric Knowledge Graph (PCKG) by prompting large language models to extract physical priors from 1,763 vocabularies, and use it to build a heterogeneous, spatial-aligned dataset, Phy-Sky-SA. Building on this foundation, we develop PriorSeg, a physics-aware residual refinement model trained with a joint visual-physical strategy that incorporates a novel physics-consistency loss. Experiments on heterogeneous settings demonstrate that PriorSeg improves segmentation accuracy and physical plausibility without retraining the FMs. Ablation studies verify the effectiveness of the Phy-Sky-SA dataset, the PCKG, and the physics-consistency loss.
Abstract (translated)
URL
https://arxiv.org/abs/2603.27504
PDF
-
Make It Up: Fake Images, Real Gains in Generalized Few-shot Semantic Segmentation
2026-03-28 09:22:50Guohuan Xie, Xin He, Dingying Fan, Le Zhang, Ming-Ming Cheng, Yun LiuAbstract
Generalized few-shot semantic segmentation (GFSS) is fundamentally limited by the coverage of novel-class appearances under scarce annotations. While diffusion models can synthesize novel-class images at scale, practical gains are often hindered by insufficient coverage and noisy supervision when masks are unavailable or unreliable. We propose Syn4Seg, a generation-enhanced GFSS framework designed to expand novel-class coverage while improving pseudo-label quality. Syn4Seg first maximizes prompt-space coverage by constructing an embedding-deduplicated prompt bank for each novel class, yielding diverse yet class-consistent synthetic images. It then performs support-guided pseudo-label estimation via a two-stage refinement that i) filters low-consistency regions to obtain high-precision seeds and ii) relabels uncertain pixels with image-adaptive prototypes that combine global (support) and local (image) statistics. Finally, we refine only boundary-band and unlabeled pixels using a constrained SAM-based update to improve contour fidelity without overwriting high-confidence interiors. Extensive experiments on PASCAL-$5^i$ and COCO-$20^i$ demonstrate consistent improvements in both 1-shot and 5-shot settings, highlighting synthetic data as a scalable path for GFSS with reliable masks and precise boundaries.
Abstract (translated)
广义小样本语义分割（GFSS）从根本上受限于稀缺标注下新类外观的覆盖范围。尽管扩散模型能够大规模合成新类图像，但当掩码不可用或不可靠时，覆盖不足和噪声监督往往制约实际效果。我们提出Syn4Seg，一个生成增强型GFSS框架，旨在扩展新类覆盖范围同时提升伪标签质量。Syn4Seg首先通过为每个新类构建嵌入去重的提示词库，最大化提示空间覆盖，生成多样且类别一致的合成图像。随后执行支持引导的伪标签估计，采用两阶段细化：i）过滤低一致性区域以获得高精度种子点；ii）利用结合全局（支持集）与局部（图像）统计的图像自适应原型，对不确定像素进行重标注。最后，仅对边界带和未标注像素执行基于约束SAM的更新，在不覆盖高置信度内部区域的前提下提升轮廓精度。在PASCAL-$5^i$和COCO-$20^i$上的大量实验表明，该方法在1-shot和5-shot设置下均实现稳定提升，凸显了合成数据作为GFSS可扩展路径的潜力，可同时保证可靠掩码与精确边界。
URL
https://arxiv.org/abs/2603.27206
PDF
-
PRUE: A Practical Recipe for Field Boundary Segmentation at Scale
2026-03-28 02:47:46Gedeon Muhawenayo, Caleb Robinson, Subash Khanal, Zhanpei Fang, Isaac Corley, Alexander Wollam, Tianyi Gao, Leonard Strnad, Ryan Avery, Lyndon Estes, Ana M. T\'arano, Nathan Jacobs, Hannah KernerAbstract
Large-scale maps of field boundaries are essential for agricultural monitoring tasks. Existing deep learning approaches for satellite-based field mapping are sensitive to illumination, spatial scale, and changes in geographic location. We conduct the first systematic evaluation of segmentation and geospatial foundation models (GFMs) for global field boundary delineation using the Fields of The World (FTW) benchmark. We evaluate 18 models under unified experimental settings, showing that a U-Net semantic segmentation model outperforms instance-based and GFM alternatives on a suite of performance and deployment metrics. We propose a new segmentation approach that combines a U-Net backbone, composite loss functions, and targeted data augmentations to enhance performance and robustness under real-world conditions. Our model achieves a 76\% IoU and 47\% object-F1 on FTW, an increase of 6\% and 9\% over the previous baseline. Our approach provides a practical framework for reliable, scalable, and reproducible field boundary delineation across model design, training, and inference. We release all models and model-derived field boundary datasets for five countries.
Abstract (translated)
大规模的地块边界地图对于农业监测任务至关重要。现有的基于卫星的地块制图深度学习方法对光照、空间尺度和地理位置变化较为敏感。我们首次利用《世界农田》（FTW）基准数据集，系统评估了用于全球地块边界勾绘的分割模型与地理空间基础模型（GFMs）。在统一的实验设置下，我们对18个模型进行了评估，结果表明U-Net语义分割模型在一系列性能与部署指标上均优于基于实例的方法和GFMs。我们提出了一种新的分割方法，该方法结合U-Net主干网络、复合损失函数和定向数据增强技术，以提升模型在真实场景下的性能与鲁棒性。我们的模型在FTW数据集上达到76%的IoU和47%的目标F1分数，较此前基准分别提升了6%和9%。该方法为跨模型设计、训练与推理的可靠、可扩展且可复现的地块边界勾绘提供了实用框架。我们已发布涵盖五个国家的全部模型及基于模型生成的地块边界数据集。
URL
https://arxiv.org/abs/2603.27101
PDF
-
Tunable Soft Equivariance with Guarantees
2026-03-27 17:56:25Md Ashiqur Rahman, Lim Jun Hao, Jeremiah Jiang, Teck-Yian Lim, Raymond A. YehAbstract
Equivariance is a fundamental property in computer vision models, yet strict equivariance is rarely satisfied in real-world data, which can limit a model's performance. Controlling the degree of equivariance is therefore desirable. We propose a general framework for constructing soft equivariant models by projecting the model weights into a designed subspace. The method applies to any pre-trained architecture and provides theoretical bounds on the induced equivariance error. Empirically, we demonstrate the effectiveness of our method on multiple pre-trained backbones, including ViT and ResNet, across image classification, semantic segmentation, and human-trajectory prediction tasks. Notably, our approach improves the performance while simultaneously reducing equivariance error on the competitive ImageNet benchmark.
Abstract (translated)
等变性是计算机视觉模型中的一个基本属性，然而在真实数据中严格等变性很少被满足，这可能限制模型的性能。因此，控制等变性的程度是可取的。我们提出了一种通过将模型权重投影到设计好的子空间来构建软等变模型的通用框架。该方法适用于任何预训练架构，并能对诱导的等变性误差提供理论上界。实验证明，我们的方法在多个预训练骨干网络（包括ViT和ResNet）上，在图像分类、语义分割和人体轨迹预测任务中均有效。值得注意的是，在竞争激烈的ImageNet基准测试中，我们的方法在提升性能的同时降低了等变性误差。
URL
https://arxiv.org/abs/2603.26657
PDF
-
Beyond MACs: Hardware Efficient Architecture Design for Vision Backbones
2026-03-27 16:05:20Moritz Nottebaum, Matteo Dunnhofer, Christian MicheloniAbstract
Vision backbone networks play a central role in modern computer vision. Enhancing their efficiency directly benefits a wide range of downstream applications. To measure efficiency, many publications rely on MACs (Multiply Accumulate operations) as a predictor of execution time. In this paper, we experimentally demonstrate the shortcomings of such a metric, especially in the context of edge devices. By contrasting the MAC count and execution time of common architectural design elements, we identify key factors for efficient execution and provide insights to optimize backbone design. Based on these insights, we present LowFormer, a novel vision backbone family. LowFormer features a streamlined macro and micro design that includes Lowtention, a lightweight alternative to Multi-Head Self-Attention. Lowtention not only proves more efficient, but also enables superior results on ImageNet. Additionally, we present an edge GPU version of LowFormer, that can further improve upon its baseline's speed on edge GPU and desktop GPU. We demonstrate LowFormer's wide applicability by evaluating it on smaller image classification datasets, as well as adapting it to several downstream tasks, such as object detection, semantic segmentation, image retrieval, and visual object tracking. LowFormer models consistently achieve remarkable speed-ups across various hardware platforms compared to recent state-of-the-art backbones. Code and models are available at this https URL.
Abstract (translated)
视觉骨干网络在现代计算机视觉中扮演着核心角色。提升其效率可直接惠及众多下游应用。为衡量效率，许多出版物依赖乘加运算次数（MACs）作为执行时间的预测指标。本文通过实验证明该指标存在缺陷，尤其在边缘设备场景下。通过对比常见架构设计元素的MACs数量与执行时间，我们识别出高效执行的关键因素，并为骨干网络设计优化提供见解。基于这些见解，我们提出LowFormer——一种新型视觉骨干网络家族。LowFormer采用精简的宏观与微观设计，包含Lowtention（多头自注意力机制的轻量化替代方案）。Lowtention不仅效率更高，还在ImageNet上取得更优结果。此外，我们推出了LowFormer的边缘GPU版本，可在边缘GPU和桌面GPU上进一步超越基线速度。通过在较小图像分类数据集上的评估，以及将其适配到目标检测、语义分割、图像检索、视觉目标跟踪等多个下游任务，我们验证了LowFormer的广泛适用性。与近期最先进骨干网络相比，LowFormer模型在各类硬件平台上均实现了显著加速。代码与模型详见链接。
URL
https://arxiv.org/abs/2603.26551
PDF
-
Learnable Quantum Efficiency Filters for Urban Hyperspectral Segmentation
2026-03-27 15:40:12Imad Ali Shah, Jiarong Li, Ethan Delaney, Enda Ward, Martin Glavin, Edward Jones, Brian DeeganAbstract
Hyperspectral sensing provides rich spectral information for scene understanding in urban driving, but its high dimensionality poses challenges for interpretation and efficient learning. We introduce Learnable Quantum Efficiency (LQE), a physics-inspired, interpretable dimensionality reduction (DR) method that parameterizes smooth high-order spectral response functions that emulate plausible sensor quantum efficiency curves. Unlike conventional methods or unconstrained learnable layers, LQE enforces physically motivated constraints, including a single dominant peak, smooth responses, and bounded bandwidth. This formulation yields a compact spectral representation that preserves discriminative information while remaining fully differentiable and end-to-end trainable within semantic segmentation models (SSMs). We conduct systematic evaluations across three publicly available multi-class hyperspectral urban driving datasets, comparing LQE against six conventional and seven learnable baseline DR methods across six SSMs. Averaged across all SSMs and configurations, LQE achieves the highest average mIoU, improving over conventional methods by 2.45\%, 0.45\%, and 1.04\%, and over learnable methods by 1.18\%, 1.56\%, and 0.81\% on HyKo, HSI-Drive, and Hyperspectral City, respectively. LQE maintains strong parameter efficiency (12--36 parameters compared to 51--22K for competing learnable approaches) and competitive inference latency. Ablation studies show that low-order configurations are optimal, while the learned spectral filters converge to dataset-intrinsic wavelength patterns. These results demonstrate that physics-informed spectral learning can improve both performance and interpretability, providing a principled bridge between hyperspectral perception and data-driven multispectral sensor design for automotive vision systems.
Abstract (translated)
高光谱感知为城市驾驶场景理解提供了丰富的光谱信息，但其高维度特性给解释和高效学习带来了挑战。我们提出了受物理启发的、可解释的降维方法——可学习量子效率（LQE），该方法通过参数化平滑的高阶光谱响应函数来模拟合理的传感器量子效率曲线。与传统方法或无约束的可学习层不同，LQE 强制执行物理动机约束，包括单一主导峰值、平滑响应和受限带宽。这种公式化产生了一种紧凑的光谱表示，在保留判别信息的同时，仍可在语义分割模型（SSMs）中实现完全可微和端到端训练。我们在三个公开可用的多类高光谱城市驾驶数据集上进行了系统评估，将 LQE 与六种传统方法和七种可学习基线降维方法在六个 SSMs 上进行了比较。平均所有 SSMs 和配置，LQE 实现了最高的平均 mIoU，在 HyKo、HSI-Drive 和 Hyperspectral City 数据集上分别比传统方法提升 2.45%、0.45% 和 1.04%，比可学习方法提升 1.18%、1.56% 和 0.81%。LQE 保持了强大的参数效率（12-36 个参数，而竞争的可学习方法为 51-22K）和具有竞争力的推理延迟。消融研究表明，低阶配置是最优的，而学习到的光谱滤波器收敛于数据集固有的波长模式。这些结果证明，物理信息光谱学习可以同时提升性能和可解释性，为高光谱感知与数据驱动的汽车视觉系统多光谱传感器设计之间架起了一座原则性的桥梁。
URL
https://arxiv.org/abs/2603.26528
PDF
-
CPUBone: Efficient Vision Backbone Design for Devices with Low Parallelization Capabilities
2026-03-27 13:55:06Moritz Nottebaum, Matteo Dunnhofer, Christian MicheloniAbstract
Recent research on vision backbone architectures has predominantly focused on optimizing efficiency for hardware platforms with high parallel processing capabilities. This category increasingly includes embedded systems such as mobile phones and embedded AI accelerator modules. In contrast, CPUs do not have the possibility to parallelize operations in the same manner, wherefore models benefit from a specific design philosophy that balances amount of operations (MACs) and hardware-efficient execution by having high MACs per second (MACpS). In pursuit of this, we investigate two modifications to standard convolutions, aimed at reducing computational cost: grouping convolutions and reducing kernel sizes. While both adaptations substantially decrease the total number of MACs required for inference, sustaining low latency necessitates preserving hardware-efficiency. Our experiments across diverse CPU devices confirm that these adaptations successfully retain high hardware-efficiency on CPUs. Based on these insights, we introduce CPUBone, a new family of vision backbone models optimized for CPU-based inference. CPUBone achieves state-of-the-art Speed-Accuracy Trade-offs (SATs) across a wide range of CPU devices and effectively transfers its efficiency to downstream tasks such as object detection and semantic segmentation. Models and code are available at this https URL.
Abstract (translated)
近期关于视觉骨干架构的研究主要集中于优化适用于高并行处理硬件平台的效率，此类平台日益包括移动设备和嵌入式AI加速模块等嵌入式系统。相比之下，CPU无法以相同方式实现操作并行化，因此模型需采用特定的设计理念——通过高单位时间乘累加操作数（MACpS）来平衡运算总量（MACs）与硬件高效执行。为实现这一目标，我们研究了两种针对标准卷积的改进方案，旨在降低计算成本：分组卷积与减小核尺寸。尽管这两种调整均显著减少了推理所需的MACs总量，但维持低延迟仍需保持硬件效率。我们在多种CPU设备上的实验证实，这些调整能成功在CPU上保持高硬件效率。基于这些发现，我们提出了CPUBone——一个专为CPU推理优化的新型视觉骨干模型家族。CPUBone在各类CPU设备上实现了先进的速度-精度权衡（SAT），并能将效率有效迁移至目标检测、语义分割等下游任务。模型与代码已公开于此链接。
URL
https://arxiv.org/abs/2603.26425
PDF
-
DRUM: Diffusion-based Raydrop-aware Unpaired Mapping for Sim2Real LiDAR Segmentation
2026-03-27 10:30:49Tomoya Miyawaki, Kazuto Nakashima, Yumi Iwashita, Ryo KurazumeAbstract
LiDAR-based semantic segmentation is a key component for autonomous mobile robots, yet large-scale annotation of LiDAR point clouds is prohibitively expensive and time-consuming. Although simulators can provide labeled synthetic data, models trained on synthetic data often underperform on real-world data due to a data-level domain gap. To address this issue, we propose DRUM, a novel Sim2Real translation framework. We leverage a diffusion model pre-trained on unlabeled real-world data as a generative prior and translate synthetic data by reproducing two key measurement characteristics: reflectance intensity and raydrop noise. To improve sample fidelity, we introduce a raydrop-aware masked guidance mechanism that selectively enforces consistency with the input synthetic data while preserving realistic raydrop noise induced by the diffusion prior. Experimental results demonstrate that DRUM consistently improves Sim2Real performance across multiple representations of LiDAR data. The project page is available at this https URL.
Abstract (translated)
基于激光雷达的语义分割是自主移动机器人的关键组成部分，然而对激光雷达点云进行大规模标注的成本极高且耗时。尽管模拟器可提供带标注的合成数据，但由于数据层面的域差异，在合成数据上训练的模型在真实数据上往往表现不佳。为应对此问题，我们提出了DRUM——一种新颖的Sim2Real（仿真到现实）翻译框架。我们利用在无标注真实数据上预训练的扩散模型作为生成先验，并通过复现两种关键测量特性（反射强度与雨滴噪声）来转换合成数据。为提升样本保真度，我们引入了雨滴感知掩码引导机制，该机制在保留扩散先验所引入的真实雨滴噪声的同时，选择性地强化与输入合成数据的一致性。实验结果表明，DRUM在多种激光雷达数据表示形式上均持续提升了Sim2Real性能。项目页面请访问此链接。
URL
https://arxiv.org/abs/2603.26263
PDF
-
GeoGuide: Hierarchical Geometric Guidance for Open-Vocabulary 3D Semantic Segmentation
2026-03-27 10:29:19Xujing Tao, Chuxin Wang, Yubo Ai, Zhixin Cheng, Zhuoyuan Li, Liangsheng Liu, Yujia Chen, Xinjun Li, Qiao Li, Wenfei Yang, Tianzhu ZhangAbstract
Open-vocabulary 3D semantic segmentation aims to segment arbitrary categories beyond the training set. Existing methods predominantly rely on distilling knowledge from 2D open-vocabulary models. However, aligning 3D features to the 2D representation space restricts intrinsic 3D geometric learning and inherits errors from 2D predictions. To address these limitations, we propose GeoGuide, a novel framework that leverages pretrained 3D models to integrate hierarchical geometry-semantic consistency for open-vocabulary 3D segmentation. Specifically, we introduce an Uncertainty-based Superpoint Distillation module to fuse geometric and semantic features for estimating per-point uncertainty, adaptively weighting 2D features within superpoints to suppress noise while preserving discriminative information to enhance local semantic consistency. Furthermore, our Instance-level Mask Reconstruction module leverages geometric priors to enforce semantic consistency within instances by reconstructing complete instance masks. Additionally, our Inter-Instance Relation Consistency module aligns geometric and semantic similarity matrices to calibrate cross-instance consistency for same-category objects, mitigating viewpoint-induced semantic drift. Extensive experiments on ScanNet v2, Matterport3D, and nuScenes demonstrate the superior performance of GeoGuide.
Abstract (translated)
开放词汇3D语义分割旨在对训练集之外的任意类别进行分割。现有方法主要依赖从2D开放词汇模型中蒸馏知识，然而将3D特征对齐到2D表示空间会限制3D几何本质学习，并继承2D预测的误差。为应对这些局限，我们提出GeoGuide新颖框架，利用预训练3D模型整合分层几何-语义一致性以实现开放词汇3D分割。具体而言，我们引入基于不确定性的超点蒸馏模块，融合几何与语义特征以估计逐点不确定性，在超点内自适应加权2D特征，在抑制噪声的同时保留判别信息以增强局部语义一致性。此外，我们的实例级掩码重建模块利用几何先验，通过重建完整实例掩码来强化实例内语义一致性。同时，实例间关系一致性模块对齐几何与语义相似度矩阵，对同类物体的跨实例一致性进行校准，缓解视角引起的语义漂移。在ScanNet v2、Matterport3D和nuScenes上的大量实验验证了GeoGuide的优越性能。
URL
https://arxiv.org/abs/2603.26260
PDF
-
LEMMA: Laplacian pyramids for Efficient Marine SeMAntic Segmentation
2026-03-26 17:38:16Ishaan Gakhar, Laven Srivastava, Sankarshanaa Sagaram, Aditya Kasliwal, Ujjwal VermaAbstract
Semantic segmentation in marine environments is crucial for the autonomous navigation of unmanned surface vessels (USVs) and coastal Earth Observation events such as oil spills. However, existing methods, often relying on deep CNNs and transformer-based architectures, face challenges in deployment due to their high computational costs and resource-intensive nature. These limitations hinder the practicality of real-time, low-cost applications in real-world marine settings. To address this, we propose LEMMA, a lightweight semantic segmentation model designed specifically for accurate remote sensing segmentation under resource constraints. The proposed architecture leverages Laplacian Pyramids to enhance edge recognition, a critical component for effective feature extraction in complex marine environments for disaster response, environmental surveillance, and coastal monitoring. By integrating edge information early in the feature extraction process, LEMMA eliminates the need for computationally expensive feature map computations in deeper network layers, drastically reducing model size, complexity and inference time. LEMMA demonstrates state-of-the-art performance across datasets captured from diverse platforms while reducing trainable parameters and computational requirements by up to 71x, GFLOPs by up to 88.5\%, and inference time by up to 84.65\%, as compared to existing models. Experimental results highlight its effectiveness and real-world applicability, including 93.42\% IoU on the Oil Spill dataset and 98.97\% mIoU on Mastr1325.
Abstract (translated)
海洋环境中的语义分割对于无人水面艇（USV）的自主导航以及海岸带地球观测事件（如油污泄漏）至关重要。然而，现有方法通常依赖深度卷积神经网络（CNN）和基于Transformer的架构，因其高昂的计算成本和资源密集型特性，在部署时面临挑战。这些限制阻碍了其在真实海洋场景中实现实时、低成本应用的实用性。为解决此问题，我们提出了LEMMA——一个专门针对资源受限条件下精确遥感分割而设计的轻量级语义分割模型。该架构利用拉普拉斯金字塔增强边缘识别能力，这是复杂海洋环境中灾难响应、环境监测和海岸监控有效特征提取的关键环节。通过在特征提取早期整合边缘信息，LEMMA消除了深层网络层中对计算量大的特征图运算的需求，显著降低了模型规模、复杂度和推理时间。与现有模型相比，LEMMA在来自不同平台采集的数据集上展现出最先进性能，同时将可训练参数和计算需求最多减少71倍，GFLOPs最多降低88.5%，推理时间最多缩短84.65%。实验结果凸显了其有效性和实际应用价值，在油污数据集上达到93.42%的IoU，在Mastr1325数据集上达到98.97%的mIoU。
URL
https://arxiv.org/abs/2603.25689
PDF
-
Hyperspectral Trajectory Image for Multi-Month Trajectory Anomaly Detection
2026-03-26 09:59:17Md Awsafur Rahman, Chandrakanth Gudavalli, Hardik Prajapati, B. S. ManjunathAbstract
Trajectory anomaly detection underpins applications from fraud detection to urban mobility analysis. Dense GPS methods preserve fine-grained evidence such as abnormal speeds and short-duration events, but their quadratic cost makes multi-month analysis intractable; consequently, no existing approach detects anomalies over multi-month dense GPS trajectories. The field instead relies on scalable sparse stay-point methods that discard this evidence, forcing separate architectures for each regime and preventing knowledge transfer. We argue this bottleneck is unnecessary: human trajectories, dense or sparse, share a natural two-dimensional cyclic structure along within-day and across-day axes. We therefore propose TITAnD (Trajectory Image Transformer for Anomaly Detection), which reformulates trajectory anomaly detection as a vision problem by representing trajectories as a Hyperspectral Trajectory Image (HTI): a day x time-of-day grid whose channels encode spatial, semantic, temporal, and kinematic information from either modality, unifying both under a single representation. Under this formulation, agent-level detection reduces to image classification and temporal localization to semantic segmentation. To model this representation, we introduce the Cyclic Factorized Transformer (CFT), which factorizes attention along the two temporal axes, encoding the cyclic inductive bias of human routines, while reducing attention cost by orders of magnitude and enabling dense multi-month anomaly detection for the first time. Empirically, TITAnD achieves the best AUC-PR across sparse and dense benchmarks, surpassing vision models like UNet while being 11-75x faster than the Transformer with comparable memory, demonstrating that vision reformulation and structure-aware modeling are jointly essential. Code will be made public soon.
Abstract (translated)
轨迹异常检测是欺诈检测、城市移动性分析等应用的基础。密集GPS方法能保留细粒度证据（如异常速度、短时事件），但其二次方计算成本使得多个月份的分析难以处理；因此，现有方法无法对多个月份的密集GPS轨迹进行异常检测。该领域转而依赖可扩展的稀疏停留点方法，但这些方法丢弃了上述证据，导致每种场景需要独立的架构，并阻碍知识迁移。我们认为这一瓶颈并非必要：人类轨迹无论密集或稀疏，在日内和跨日轴上均共享自然的二维周期性结构。因此，我们提出了TITAnD（用于异常检测的轨迹图像变换器），通过将轨迹表示为超光谱轨迹图像（HTI）——一种日×时刻的网格，其通道编码来自任一模态的空间、语义、时间与运动学信息，从而在单一表示下统一两者——将轨迹异常检测重新定义为视觉问题。在此框架下，智能体级别检测简化为图像分类，时间定位简化为语义分割。为建模该表示，我们引入了循环因子化变换器（CFT），其沿两个时间轴分解注意力，编码人类日常活动的周期性归纳偏置，同时将注意力成本降低数个数量级，并首次实现了密集多月份异常检测。实验表明，TITAnD在稀疏与密集基准上均达到最佳AUC-PR，超越UNet等视觉模型，且比同等内存占用的标准变换器快11-75倍，证明了视觉重构与结构感知建模的协同必要性。代码即将公开。
URL
https://arxiv.org/abs/2603.25255
PDF
-
A Unified Spatial Alignment Framework for Highly Transferable Transformation-Based Attacks on Spatially Structured Tasks
2026-03-26 09:31:46Jiaming Liang, Chi-Man PunAbstract
Transformation-based adversarial attacks (TAAs) demonstrate strong transferability when deceiving classification models. However, existing TAAs often perform unsatisfactorily or even fail when applied to structured tasks such as semantic segmentation and object detection. Encouragingly, recent studies that categorize transformations into non-spatial and spatial transformations inspire us to address this challenge. We find that for non-structured tasks, labels are spatially non-structured, and thus TAAs are not required to adjust labels when applying spatial transformations. In contrast, for structured tasks, labels are spatially structured, and failing to transform labels synchronously with inputs can cause spatial misalignment and yield erroneous gradients. To address these issues, we propose a novel unified Spatial Alignment Framework (SAF) for highly transferable TAAs on spatially structured tasks, where the TAAs spatially transform labels synchronously with the input using the proposed Spatial Alignment (SA) algorithm. Extensive experiments demonstrate the crucial role of our SAF for TAAs on structured tasks. Specifically, in non-targeted attacks, our SAF degrades the average mIoU on Cityscapes from 24.50 to 11.34, and on Kvasir-SEG from 49.91 to 31.80, while reducing the average mAP of COCO from 17.89 to 5.25.
Abstract (translated)
基于变换的对抗攻击（TAAs）在欺骗分类模型时展现出强迁移性。然而，当应用于语义分割和目标检测等结构化任务时，现有TAAs往往表现不佳甚至失效。令人鼓舞的是，近期研究将变换分为非空间变换与空间变换的分类方式启发了我们应对这一挑战。我们发现：对于非结构化任务，其标签在空间上呈非结构化分布，因此TAAs在应用空间变换时无需调整标签；而对于结构化任务，标签具有空间结构性，若不同步变换标签与输入会导致空间错位并产生错误梯度。针对这些问题，我们提出了一种新颖的统一空间对齐框架（SAF），用于在空间结构化任务中实现高迁移性TAAs，其中TAAs通过所提出的空间对齐（SA）算法同步变换标签与输入。大量实验验证了SAF对结构化任务中TAAs的关键作用。具体而言，在非目标攻击中，SAF使Cityscapes数据集上的平均mIoU从24.50降至11.34，Kvasir-SEG数据集上从49.91降至31.80，同时将COCO数据集的平均mAP从17.89降至5.25。
URL
https://arxiv.org/abs/2603.25230
PDF
-
Denoise and Align: Towards Source-Free UDA for Robust Panoramic Semantic Segmentation
2026-03-26 07:54:43Yaowen Chang, Zhen Cao, Xu Zheng, Xiaoxin Mi, Zhen DongAbstract
Panoramic semantic segmentation is pivotal for comprehensive 360° scene understanding in critical applications like autonomous driving and virtual reality. However, progress in this domain is constrained by two key challenges: the severe geometric distortions inherent in panoramic projections and the prohibitive cost of dense annotation. While Unsupervised Domain Adaptation (UDA) from label-rich pinhole-camera datasets offers a viable alternative, many real-world tasks impose a stricter source-free (SFUDA) constraint where source data is inaccessible for privacy or proprietary reasons. This constraint significantly amplifies the core problems of domain shift, leading to unreliable pseudo-labels and dramatic performance degradation, particularly for minority classes. To overcome these limitations, we propose the DAPASS framework. DAPASS introduces two synergistic modules to robustly transfer knowledge without source data. First, our Panoramic Confidence-Guided Denoising (PCGD) module generates high-fidelity, class-balanced pseudo-labels by enforcing perturbation consistency and incorporating neighborhood-level confidence to filter noise. Second, a Contextual Resolution Adversarial Module (CRAM) explicitly addresses scale variance and distortion by adversarially aligning fine-grained details from high-resolution crops with global semantics from low-resolution contexts. DAPASS achieves state-of-the-art performances on outdoor (Cityscapes-to-DensePASS) and indoor (Stanford2D3D) benchmarks, yielding 55.04% (+2.05%) and 70.38% (+1.54%) mIoU, respectively.
Abstract (translated)
全景语义分割对于自动驾驶和虚拟现实等关键应用中的全方位360°场景理解至关重要。然而，该领域的发展受限于两个核心挑战：全景投影固有的严重几何畸变，以及密集标注的高昂成本。尽管从标注丰富的针孔相机数据集进行无监督域适应（UDA）提供了一种可行方案，但许多实际任务施加了更严格的源自由（SFUDA）约束——因隐私或专有原因无法访问源数据。这一约束显著加剧了域偏移的核心问题，导致伪标签不可靠且性能急剧下降，尤其对少数类影响显著。为克服这些限制，我们提出了DAPASS框架。该框架引入两个协同模块以在无源数据条件下稳健迁移知识：首先，全景置信度引导去噪（PCGD）模块通过强制扰动一致性并引入邻域级置信度过滤噪声，生成高保真、类别平衡的伪标签；其次，上下文分辨率对抗模块（CRAM）通过对抗性对齐高分辨率裁剪的细粒度细节与低分辨率上下文的全局语义，显式解决尺度变异和畸变问题。DAPASS在户外（Cityscapes-to-DensePASS）和室内（Stanford2D3D）基准测试中均达到最先进性能，分别取得55.04%（+2.05%）和70.38%（+1.54%）的mIoU。
URL
https://arxiv.org/abs/2603.25131
PDF
-
Heuristic Self-Paced Learning for Domain Adaptive Semantic Segmentation under Adverse Conditions
2026-03-25 14:01:55Shiqin Wang, Haoyang Chen, Huaizhou Huang, Yinkan He, Dongfang Sun, Xiaoqing Chen, Xingyu Liu, Zheng Wang, Kaiyan ZhaoAbstract
The learning order of semantic classes significantly impacts unsupervised domain adaptation for semantic segmentation, especially under adverse weather conditions. Most existing curricula rely on handcrafted heuristics (e.g., fixed uncertainty metrics) and follow a static schedule, which fails to adapt to a model's evolving, high-dimensional training dynamics, leading to category bias. Inspired by Reinforcement Learning, we cast curriculum learning as a sequential decision problem and propose an autonomous class scheduler. This scheduler consists of two components: (i) a high-dimensional state encoder that maps the model's training status into a latent space and distills key features indicative of progress, and (ii) a category-fair policy-gradient objective that ensures balanced improvement across classes. Coupled with mixed source-target supervision, the learned class rankings direct the network's focus to the most informative classes at each stage, enabling more adaptive and dynamic learning. It is worth noting that our method achieves state-of-the-art performance on three widely used benchmarks (e.g., ACDC, Dark Zurich, and Nighttime Driving) and shows generalization ability in synthetic-to-real semantic segmentation.
Abstract (translated)
语义类别的学习顺序显著影响无监督领域适应在语义分割中的表现，尤其在恶劣天气条件下。大多数现有课程依赖手工设计的启发式方法（如固定的不确定性指标）并遵循静态计划，这无法适应模型不断变化的高维训练动态，导致类别偏差。受强化学习启发，我们将课程学习构建为序列决策问题，并提出一种自主类别调度器。该调度器包含两个组件：(i) 高维状态编码器，将模型训练状态映射到潜在空间并提炼指示进展的关键特征；(ii) 类别公平的策略梯度目标，确保各类别间的平衡提升。结合混合源-目标监督，学习到的类别排序在每个阶段引导网络关注最具信息量的类别，实现更自适应和动态的学习。值得注意的是，我们的方法在三个广泛使用的基准数据集（如ACDC、Dark Zurich和Nighttime Driving）上达到最先进性能，并在合成到真实的语义分割中展现出泛化能力。
URL
https://arxiv.org/abs/2603.24322
PDF
