Semantic_Segmentation
-
Open-Vocabulary Domain Generalization in Urban-Scene Segmentation
2026-02-21 14:32:27Dong Zhao, Qi Zang, Nan Pu, Wenjing Li, Nicu Sebe, Zhun ZhongAbstract
Domain Generalization in Semantic Segmentation (DG-SS) aims to enable segmentation models to perform robustly in unseen environments. However, conventional DG-SS methods are restricted to a fixed set of known categories, limiting their applicability in open-world scenarios. Recent progress in Vision-Language Models (VLMs) has advanced Open-Vocabulary Semantic Segmentation (OV-SS) by enabling models to recognize a broader range of concepts. Yet, these models remain sensitive to domain shifts and struggle to maintain robustness when deployed in unseen environments, a challenge that is particularly severe in urban-driving scenarios. To bridge this gap, we introduce Open-Vocabulary Domain Generalization in Semantic Segmentation (OVDG-SS), a new setting that jointly addresses unseen domains and unseen categories. We introduce the first benchmark for OVDG-SS in autonomous driving, addressing a previously unexplored problem and covering both synthetic-to-real and real-to-real generalization across diverse unseen domains and unseen categories. In OVDG-SS, we observe that domain shifts often distort text-image correlations in pre-trained VLMs, which hinders the performance of OV-SS models. To tackle this challenge, we propose S2-Corr, a state-space-driven text-image correlation refinement mechanism that mitigates domain-induced distortions and produces more consistent text-image correlations under distribution changes. Extensive experiments on our constructed benchmark demonstrate that the proposed method achieves superior cross-domain performance and efficiency compared to existing OV-SS approaches.
Abstract (translated)
领域泛化在语义分割（Domain Generalization in Semantic Segmentation，DG-SS）旨在使分割模型能够在未见过的环境中稳健地运行。然而，传统的DG-SS方法局限于一组已知类别，这限制了它们在开放世界场景中的应用能力。近期视觉语言模型（Vision-Language Models, VLMs）的进步推动了开放词汇语义分割（Open-Vocabulary Semantic Segmentation，OV-SS），使模型能够识别更广泛的概念范围。然而，这些模型仍然对领域偏移敏感，并且难以保持在未见过环境中部署的稳健性，在城市驾驶场景中这一问题尤为严重。 为弥合这一差距，我们提出了开放词汇领域泛化的语义分割（Open-Vocabulary Domain Generalization in Semantic Segmentation, OVDG-SS），这是一个同时处理未见领域和未见类别的新设定。我们在自主驾驶环境中引入了第一个针对OVDG-SS的基准测试，解决了之前未探索的问题，并涵盖了从合成到现实以及真实环境间多样化的未见过领域的泛化。 在OVDG-SS中，我们观察到域偏移常常扭曲预训练VLM中的文本图像相关性，这阻碍了OV-SS模型的表现。为解决这一挑战，我们提出了S2-Corr机制，这是一种基于状态空间的文本图像相关性改进方法，能够减轻领域引起的扭曲，并在分布变化下产生更一致的文本图像相关性。 在构建的基准上进行的大量实验表明，所提出的方法相比现有的OV-SS方法，在跨域性能和效率方面都表现出了显著的优势。
URL
https://arxiv.org/abs/2602.18853
PDF
-
Benchmarking Computational Pathology Foundation Models For Semantic Segmentation
2026-02-21 08:00:46Lavish Ramchandani, Aashay Tinaikar, Dev Kumar Das, Rohit Garg, Tijo ThomasAbstract
In recent years, foundation models such as CLIP, DINO,and CONCH have demonstrated remarkable domain generalization and unsupervised feature extraction capabilities across diverse imaging tasks. However, systematic and independent evaluations of these models for pixel-level semantic segmentation in histopathology remain scarce. In this study, we propose a robust benchmarking approach to asses 10 foundational models on four histopathological datasets covering both morphological tissue-region and cellular/nuclear segmentation tasks. Our method leverages attention maps of foundation models as pixel-wise features, which are then classified using a machine learning algorithm, XGBoost, enabling fast, interpretable, and model-agnostic evaluation without finetuning. We show that the vision language foundation model, CONCH performed the best across datasets when compared to vision-only foundation models, with PathDino as close second. Further analysis shows that models trained on distinct histopathology cohorts capture complementary morphological representations, and concatenating their features yields superior segmentation performance. Concatenating features from CONCH, PathDino and CellViT outperformed individual models across all the datasets by 7.95% (averaged across the datasets), suggesting that ensembles of foundation models can better generalize to diverse histopathological segmentation tasks.
Abstract (translated)
近年来，基础模型（如CLIP、DINO和CONCH）在各种成像任务中展现了出色的领域泛化能力和无监督特征提取能力。然而，在像素级语义分割的组织病理学应用方面，这些模型系统的独立评估仍然很少见。在这项研究中，我们提出了一种稳健的基准测试方法，用于评估10个基础模型在四个涵盖形态组织区域和细胞/核分割任务的组织病理学数据集上的性能。我们的方法利用了基础模型的注意力图作为像素级特征，并使用机器学习算法XGBoost进行分类，这使得无需微调即可快速、可解释且与模型无关地评估这些模型。我们发现，在所有数据集中，视觉语言基础模型CONCH的表现优于仅基于视觉的基础模型，而PathDino紧随其后。进一步的分析表明，针对不同的组织病理学队列训练出的模型捕捉到了互补的形态特征表示，并将它们的特征合并可以显著提高分割性能。结合CONCH、PathDino和CellViT的特征在所有数据集上均超越了单一模型的表现（平均高出7.95%），表明基础模型的集成能够更好地泛化到各种组织病理学分割任务中。
URL
https://arxiv.org/abs/2602.18747
PDF
-
Faster Training, Fewer Labels: Self-Supervised Pretraining for Fine-Grained BEV Segmentation
2026-02-20 08:37:58Daniel Busch, Christian Bohn, Thomas Kurbiel, Klaus Friedrichs, Richard Meyes, Tobias MeisenAbstract
Dense Bird's Eye View (BEV) semantic maps are central to autonomous driving, yet current multi-camera methods depend on costly, inconsistently annotated BEV ground truth. We address this limitation with a two-phase training strategy for fine-grained road marking segmentation that removes full supervision during pretraining and halves the amount of training data during fine-tuning while still outperforming the comparable supervised baseline model. During the self-supervised pretraining, BEVFormer predictions are differentiably reprojected into the image plane and trained against multi-view semantic pseudo-labels generated by the widely used semantic segmentation model Mask2Former. A temporal loss encourages consistency across frames. The subsequent supervised fine-tuning phase requires only 50% of the dataset and significantly less training time. With our method, the fine-tuning benefits from rich priors learned during pretraining boosting the performance and BEV segmentation quality (up to +2.5pp mIoU over the fully supervised baseline) on nuScenes. It simultaneously halves the usage of annotation data and reduces total training time by up to two thirds. The results demonstrate that differentiable reprojection plus camera perspective pseudo labels yields transferable BEV features and a scalable path toward reduced-label autonomous perception.
Abstract (translated)
稠密的鸟瞰图（BEV）语义地图对于自动驾驶至关重要，然而现有的多相机方法依赖于成本高昂且标注不一致的BEV地面真实数据。我们通过一个两阶段训练策略解决了这一局限性，该策略用于细粒度的道路标记分割任务，在预训练过程中取消了完全监督，并在微调阶段将训练数据量减半，但仍优于可比较的有监督基线模型。在自我监督的预训练期间，BEVFormer预测被差异性地重新投影到图像平面上，并通过广泛使用的语义分割模型Mask2Former生成的多视角语义伪标签进行训练。时间损失鼓励帧之间的连贯性。后续的监督微调阶段仅需要50%的数据集和显著更少的训练时间。使用我们的方法，微调能够从预训练期间学习到的丰富先验中获益，从而提升性能并提高BEV分割质量（在nuScenes数据集中，平均交并比mIoU提高了多达+2.5个百分点）。同时，这种方法将标注数据的使用量减半，并最多减少总训练时间三分之二。实验结果表明，差异性重新投影加上相机视角伪标签能够生成可迁移的BEV特征和一条向低标签自动驾驶感知发展的可扩展路径。
URL
https://arxiv.org/abs/2602.18066
PDF
-
Neural Prior Estimation: Learning Class Priors from Latent Representations
2026-02-19 21:36:34Masoud Yavari, Payman MoallemAbstract
Class imbalance induces systematic bias in deep neural networks by imposing a skewed effective class prior. This work introduces the Neural Prior Estimator (NPE), a framework that learns feature-conditioned log-prior estimates from latent representations. NPE employs one or more Prior Estimation Modules trained jointly with the backbone via a one-way logistic loss. Under the Neural Collapse regime, NPE is analytically shown to recover the class log-prior up to an additive constant, providing a theoretically grounded adaptive signal without requiring explicit class counts or distribution-specific hyperparameters. The learned estimate is incorporated into logit adjustment, forming NPE-LA, a principled mechanism for bias-aware prediction. Experiments on long-tailed CIFAR and imbalanced semantic segmentation benchmarks (STARE, ADE20K) demonstrate consistent improvements, particularly for underrepresented classes. NPE thus offers a lightweight and theoretically justified approach to learned prior estimation and imbalance-aware prediction.
Abstract (translated)
类不平衡会在深度神经网络中引入系统偏差，通过施加扭曲的效用类别先验。这项工作介绍了神经优先估计器（NPE），这是一种框架，能够从潜在表示中学习到特征条件下的对数先验估计值。NPE 使用一个或多个与主干模型共同训练的优先估计模块，并采用单向逻辑损失函数。在神经崩溃制度下，NPE 被理论证明可以恢复类别对数先验（除了一个可加常量），从而提供一种无需显式类计数或特定分布超参数的自适应信号。所学得的估计值被整合进 logits 调整中，形成 NPE-LA，这是一种偏见感知预测的原则机制。在长尾 CIFAR 和不平衡语义分割基准测试（如 STARE、ADE20K）上的实验显示了对代表性不足类别的持续改进效果。因此，NPE 提供了一种轻量级且理论基础坚实的类先验估计和偏差感知预测方法。
URL
https://arxiv.org/abs/2602.17853
PDF
-
Enabling Training-Free Text-Based Remote Sensing Segmentation
2026-02-19 20:05:56Jose Sosa, Danila Rukhovich, Anis Kacem, Djamila AouadaAbstract
Recent advances in Vision Language Models (VLMs) and Vision Foundation Models (VFMs) have opened new opportunities for zero-shot text-guided segmentation of remote sensing imagery. However, most existing approaches still rely on additional trainable components, limiting their generalisation and practical applicability. In this work, we investigate to what extent text-based remote sensing segmentation can be achieved without additional training, by relying solely on existing foundation models. We propose a simple yet effective approach that integrates contrastive and generative VLMs with the Segment Anything Model (SAM), enabling a fully training-free or lightweight LoRA-tuned pipeline. Our contrastive approach employs CLIP as mask selector for SAM's grid-based proposals, achieving state-of-the-art open-vocabulary semantic segmentation (OVSS) in a completely zero-shot setting. In parallel, our generative approach enables reasoning and referring segmentation by generating click prompts for SAM using GPT-5 in a zero-shot setting and a LoRA-tuned Qwen-VL model, with the latter yielding the best results. Extensive experiments across 19 remote sensing benchmarks, including open-vocabulary, referring, and reasoning-based tasks, demonstrate the strong capabilities of our approach. Code will be released at this https URL.
Abstract (translated)
最近在视觉语言模型（VLM）和视觉基础模型（VFMs）方面的进展为零样本文本引导的遥感图像分割带来了新的机遇。然而，大多数现有的方法仍然依赖于额外可训练的组件，这限制了它们的泛化能力和实际应用性。在这项工作中，我们探讨了在仅依靠现有基础模型的情况下实现基于文本的遥感分割到什么程度是可能的。我们提出了一种简单而有效的方法，该方法将对比和生成型VLM与Segment Anything Model（SAM）相结合，从而实现了完全无需训练或轻量级LoRA微调的流水线。 我们的对比方法利用CLIP作为SAM网格化提议的掩码选择器，在完全零样本设置下实现了最先进的开放式词汇语义分割(OVSS)。同时，我们生成的方法通过使用GPT-5在零样本环境下为SAM生成点击提示，并用经过LoRA微调的Qwen-VL模型来实现推理和参考分割任务，后者取得了最佳效果。 我们在包括开放词汇、引用及基于推理的任务在内的19个遥感基准数据集上进行了广泛的实验，展示了我们方法的强大能力。代码将在[提供的URL]发布。
URL
https://arxiv.org/abs/2602.17799
PDF
-
RA-Nav: A Risk-Aware Navigation System Based on Semantic Segmentation for Aerial Robots in Unpredictable Environments
2026-02-19 16:26:43Ziyi Zong, Xin Dong, Jinwu Xiang, Daochun Li, Zhan TuAbstract
Existing aerial robot navigation systems typically plan paths around static and dynamic obstacles, but fail to adapt when a static obstacle suddenly moves. Integrating environmental semantic awareness enables estimation of potential risks posed by suddenly moving obstacles. In this paper, we propose RA- Nav, a risk-aware navigation framework based on semantic segmentation. A lightweight multi-scale semantic segmentation network identifies obstacle categories in real time. These obstacles are further classified into three types: stationary, temporarily static, and dynamic. For each type, corresponding risk estimation functions are designed to enable real-time risk prediction, based on which a complete local risk map is constructed. Based on this map, the risk-informed path search algorithm is designed to guarantee planning that balances path efficiency and safety. Trajectory optimization is then applied to generate trajectories that are safe, smooth, and dynamically feasible. Comparative simulations demonstrate that RA-Nav achieves higher success rates than baselines in sudden obstacle state transition scenarios. Its effectiveness is further validated in simulations using real- world data.
Abstract (translated)
URL
https://arxiv.org/abs/2602.17515
PDF
-
Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation
2026-02-17 06:37:00Shuwei Li, Lei Tan, Robby T. TanAbstract
Day-to-night unpaired image translation is important to downstream tasks but remains challenging due to large appearance shifts and the lack of direct pixel-level supervision. Existing methods often introduce semantic hallucinations, where objects from target classes such as traffic signs and vehicles, as well as man-made light effects, are incorrectly synthesized. These hallucinations significantly degrade downstream performance. We propose a novel framework that detects and suppresses hallucinations of target-class features during unpaired translation. To detect hallucination, we design a dual-head discriminator that additionally performs semantic segmentation to identify hallucinated content in background regions. To suppress these hallucinations, we introduce class-specific prototypes, constructed by aggregating features of annotated target-domain objects, which act as semantic anchors for each class. Built upon a Schrodinger Bridge-based translation model, our framework performs iterative refinement, where detected hallucination features are explicitly pushed away from class prototypes in feature space, thus preserving object semantics across the translation this http URL show that our method outperforms existing approaches both qualitatively and quantitatively. On the BDD100K dataset, it improves mAP by 15.5% for day-to-night domain adaptation, with a notable 31.7% gain for classes such as traffic lights that are prone to hallucinations.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15383
PDF
-
SAILS: Segment Anything with Incrementally Learned Semantics for Task-Invariant and Training-Free Continual Learning
2026-02-16 14:14:02Shishir Muralidhara, Didier Stricker, Ren\'e SchusterAbstract
Continual learning remains constrained by the need for repeated retraining, high computational costs, and the persistent challenge of forgetting. These factors significantly limit the applicability of continuous learning in real-world settings, as iterative model updates require significant computational resources and inherently exacerbate forgetting. We present SAILS -- Segment Anything with Incrementally Learned Semantics, a training-free framework for Class-Incremental Semantic Segmentation (CISS) that sidesteps these challenges entirely. SAILS leverages foundational models to decouple CISS into two stages: Zero-shot region extraction using Segment Anything Model (SAM), followed by semantic association through prototypes in a fixed feature space. SAILS incorporates selective intra-class clustering, resulting in multiple prototypes per class to better model intra-class variability. Our results demonstrate that, despite requiring no incremental training, SAILS typically surpasses the performance of existing training-based approaches on standard CISS datasets, particularly in long and challenging task sequences where forgetting tends to be most severe. By avoiding parameter updates, SAILS completely eliminates forgetting and maintains consistent, task-invariant performance. Furthermore, SAILS exhibits positive backward transfer, where the introduction of new classes can enhance performance on previous classes.
Abstract (translated)
URL
https://arxiv.org/abs/2602.14767
PDF
-
Cross-view Domain Generalization via Geometric Consistency for LiDAR Semantic Segmentation
2026-02-16 07:19:46Jindong Zhao, Yuan Gao, Yang Xia, Sheng Nie, Jun Yue, Weiwei Sun, Shaobo XiaAbstract
Domain-generalized LiDAR semantic segmentation (LSS) seeks to train models on source-domain point clouds that generalize reliably to multiple unseen target domains, which is essential for real-world LiDAR applications. However, existing approaches assume similar acquisition views (e.g., vehicle-mounted) and struggle in cross-view scenarios, where observations differ substantially due to viewpoint-dependent structural incompleteness and non-uniform point density. Accordingly, we formulate cross-view domain generalization for LiDAR semantic segmentation and propose a novel framework, termed CVGC (Cross-View Geometric Consistency). Specifically, we introduce a cross-view geometric augmentation module that models viewpoint-induced variations in visibility and sampling density, generating multiple cross-view observations of the same scene. Subsequently, a geometric consistency module enforces consistent semantic and occupancy predictions across geometrically augmented point clouds of the same scene. Extensive experiments on six public LiDAR datasets establish the first systematic evaluation of cross-view domain generalization for LiDAR semantic segmentation, demonstrating that CVGC consistently outperforms state-of-the-art methods when generalizing from a single source domain to multiple target domains with heterogeneous acquisition viewpoints. The source code will be publicly available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2602.14525
PDF
-
DenseMLLM: Standard Multimodal LLMs are Intrinsic Dense Predictors
2026-02-15 13:12:28Yi Li, Hongze Shen, Lexiang Tang, Xin Li, Xinpeng Ding, Yinsong Liu, Deqiang Jiang, Xing Sun, Xiaomeng LiAbstract
Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in high-level visual understanding. However, extending these models to fine-grained dense prediction tasks, such as semantic segmentation and depth estimation, typically necessitates the incorporation of complex, task-specific decoders and other customizations. This architectural fragmentation increases model complexity and deviates from the generalist design of MLLMs, ultimately limiting their practicality. In this work, we challenge this paradigm by accommodating standard MLLMs to perform dense predictions without requiring additional task-specific decoders. The proposed model is called DenseMLLM, grounded in the standard architecture with a novel vision token supervision strategy for multiple labels and tasks. Despite its minimalist design, our model achieves highly competitive performance across a wide range of dense prediction and vision-language benchmarks, demonstrating that a standard, general-purpose MLLM can effectively support dense perception without architectural specialization.
Abstract (translated)
URL
https://arxiv.org/abs/2602.14134
PDF
-
Restoration Adaptation for Semantic Segmentation on Low Quality Images
2026-02-15 08:13:23Kai Guan, Rongyuan Wu, Shuai Li, Wentao Zhu, Wenjun Zeng, Lei ZhangAbstract
In real-world scenarios, the performance of semantic segmentation often deteriorates when processing low-quality (LQ) images, which may lack clear semantic structures and high-frequency details. Although image restoration techniques offer a promising direction for enhancing degraded visual content, conventional real-world image restoration (Real-IR) models primarily focus on pixel-level fidelity and often fail to recover task-relevant semantic cues, limiting their effectiveness when directly applied to downstream vision tasks. Conversely, existing segmentation models trained on high-quality data lack robustness under real-world degradations. In this paper, we propose Restoration Adaptation for Semantic Segmentation (RASS), which effectively integrates semantic image restoration into the segmentation process, enabling high-quality semantic segmentation on the LQ images directly. Specifically, we first propose a Semantic-Constrained Restoration (SCR) model, which injects segmentation priors into the restoration model by aligning its cross-attention maps with segmentation masks, encouraging semantically faithful image reconstruction. Then, RASS transfers semantic restoration knowledge into segmentation through LoRA-based module merging and task-specific fine-tuning, thereby enhancing the model's robustness to LQ images. To validate the effectiveness of our framework, we construct a real-world LQ image segmentation dataset with high-quality annotations, and conduct extensive experiments on both synthetic and real-world LQ benchmarks. The results show that SCR and RASS significantly outperform state-of-the-art methods in segmentation and restoration tasks. Code, models, and datasets will be available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2602.14042
PDF
-
Attention-gated U-Net model for semantic segmentation of brain tumors and feature extraction for survival prognosis
2026-02-14 07:48:58Rut Pate, Snehal Rajput, Mehul S. Raval, Rupal A. Kapdi, Mohendra RoyAbstract
Gliomas, among the most common primary brain tumors, vary widely in aggressiveness, prognosis, and histology, making treatment challenging due to complex and time-intensive surgical interventions. This study presents an Attention-Gated Recurrent Residual U-Net (R2U-Net) based Triplanar (2.5D) model for improved brain tumor segmentation. The proposed model enhances feature representation and segmentation accuracy by integrating residual, recurrent, and triplanar architectures while maintaining computational efficiency, potentially aiding in better treatment planning. The proposed method achieves a Dice Similarity Score (DSC) of 0.900 for Whole Tumor (WT) segmentation on the BraTS2021 validation set, demonstrating performance comparable to leading models. Additionally, the triplanar network extracts 64 features per planar model for survival days prediction, which are reduced to 28 using an Artificial Neural Network (ANN). This approach achieves an accuracy of 45.71%, a Mean Squared Error (MSE) of 108,318.128, and a Spearman Rank Correlation Coefficient (SRC) of 0.338 on the test dataset.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15067
PDF
-
Privacy-Concealing Cooperative Perception for BEV Scene Segmentation
2026-02-14 02:11:29Song Wang, Lingling Li, Marcus Santos, Guanghui WangAbstract
Cooperative perception systems for autonomous driving aim to overcome the limited perception range of a single vehicle by communicating with adjacent agents to share sensing information. While this improves perception performance, these systems also face a significant privacy-leakage issue, as sensitive visual content can potentially be reconstructed from the shared data. In this paper, we propose a novel Privacy-Concealing Cooperation (PCC) framework for Bird's Eye View (BEV) semantic segmentation. Based on commonly shared BEV features, we design a hiding network to prevent an image reconstruction network from recovering the input images from the shared features. An adversarial learning mechanism is employed to train the network, where the hiding network works to conceal the visual clues in the BEV features while the reconstruction network attempts to uncover these clues. To maintain segmentation performance, the perception network is integrated with the hiding network and optimized end-to-end. The experimental results demonstrate that the proposed PCC framework effectively degrades the quality of the reconstructed images with minimal impact on segmentation performance, providing privacy protection for cooperating vehicles. The source code will be made publicly available upon publication.
Abstract (translated)
URL
https://arxiv.org/abs/2602.13555
PDF
-
DynaGuide: A Generalizable Dynamic Guidance Framework for Unsupervised Semantic Segmentation
2026-02-13 15:25:34Boujemaa Guermazi, Riadh Ksantini, Naimul KhanAbstract
Unsupervised image segmentation is a critical task in computer vision. It enables dense scene understanding without human annotations, which is especially valuable in domains where labelled data is scarce. However, existing methods often struggle to reconcile global semantic structure with fine-grained boundary accuracy. This paper introduces DynaGuide, an adaptive segmentation framework that addresses these challenges through a novel dual-guidance strategy and dynamic loss optimization. Building on our previous work, DynaSeg, DynaGuide combines global pseudo-labels from zero-shot models such as DiffSeg or SegFormer with local boundary refinement using a lightweight CNN trained from scratch. This synergy allows the model to correct coarse or noisy global predictions and produce high-precision segmentations. At the heart of DynaGuide is a multi-component loss that dynamically balances feature similarity, Huber-smoothed spatial continuity, including diagonal relationships, and semantic alignment with the global pseudo-labels. Unlike prior approaches, DynaGuide trains entirely without ground-truth labels in the target domain and supports plug-and-play integration of diverse guidance sources. Extensive experiments on BSD500, PASCAL VOC2012, and COCO demonstrate that DynaGuide achieves state-of-the-art performance, improving mIoU by 17.5% on BSD500, 3.1% on PASCAL VOC2012, and 11.66% on COCO. With its modular design, strong generalization, and minimal computational footprint, DynaGuide offers a scalable and practical solution for unsupervised segmentation in real-world settings. Code available at: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2602.13020
PDF
-
An Integrated Causal Inference Framework for Traffic Safety Modeling with Semantic Street-View Visual Features
2026-02-12 13:47:56Lishan Sun, Yujia Cheng, Pengfei Cui, Lei Han, Mohamed Abdel-Aty, Yunhan Zheng, Xingchen ZhangAbstract
Macroscopic traffic safety modeling aims to identify critical risk factors for regional crashes, thereby informing targeted policy interventions for safety improvement. However, current approaches rely heavily on static sociodemographic and infrastructure metrics, frequently overlooking the impacts from drivers' visual perception of driving environment. Although visual environment features have been found to impact driving and traffic crashes, existing evidence remains largely observational, failing to establish the robust causality for traffic policy evaluation under complex spatial environment. To fill these gaps, we applied semantic segmentation on Google Street View imageries to extract visual environmental features and proposed a Double Machine Learning framework to quantify their causal effects on regional crashes. Meanwhile, we utilized SHAP values to characterize the nonlinear influence mechanisms of confounding variables in the models and applied causal forests to estimate conditional average treatment effects. Leveraging crash records from the Miami metropolitan area, Florida, and 220,000 street view images, evidence shows that greenery proportion exerts a significant and robust negative causal effect on traffic crashes (Average Treatment Effect = -6.38, p = 0.005). This protective effect exhibits spatial heterogeneity, being most pronounced in densely populated and socially vulnerable urban cores. While greenery significantly mitigates angle and rear-end crashes, its protective benefit for vulnerable road users (VRUs) remains limited. Our findings provide causal evidence for greening as a potential safety intervention, prioritizing hazardous visual environments while highlighting the need for distinct design optimizations to protect VRUs.
Abstract (translated)
URL
https://arxiv.org/abs/2602.13339
PDF
-
Viewpoint Recommendation for Point Cloud Labeling through Interaction Cost Modeling
2026-02-11 13:59:42Yu Zhang, Xinyi Zhao, Chongke Bi, Siming ChenAbstract
Semantic segmentation of 3D point clouds is important for many applications, such as autonomous driving. To train semantic segmentation models, labeled point cloud segmentation datasets are essential. Meanwhile, point cloud labeling is time-consuming for annotators, which typically involves tuning the camera viewpoint and selecting points by lasso. To reduce the time cost of point cloud labeling, we propose a viewpoint recommendation approach to reduce annotators' labeling time costs. We adapt Fitts' law to model the time cost of lasso selection in point clouds. Using the modeled time cost, the viewpoint that minimizes the lasso selection time cost is recommended to the annotator. We build a data labeling system for semantic segmentation of 3D point clouds that integrates our viewpoint recommendation approach. The system enables users to navigate to recommended viewpoints for efficient annotation. Through an ablation study, we observed that our approach effectively reduced the data labeling time cost. We also qualitatively compare our approach with previous viewpoint selection approaches on different datasets.
Abstract (translated)
三维点云的语义分割对于许多应用（如自动驾驶）非常重要。为了训练语义分割模型，标注良好的点云分割数据集是必不可少的。然而，点云标注对注释者来说非常耗时，通常需要调整相机视角并使用套索工具选择点。为减少点云标注的时间成本，我们提出了一种视角推荐方法来降低注释者的工作量。我们将Fitts定律应用于建模点云中套索选择的操作时间成本，并利用该模型确定最小化套索选择时间成本的视角。我们构建了一个用于三维点云语义分割的数据标注系统，该系统集成了我们的视角推荐方法。系统允许用户导航至推荐视角以实现高效的注释工作。通过消融研究，我们观察到这种方法显著降低了数据标注的时间成本，并且在不同数据集上与先前的视角选择方法进行了定性比较。
URL
https://arxiv.org/abs/2602.10871
PDF
-
AMAP-APP: Efficient Segmentation and Morphometry Quantification of Fluorescent Microscopy Images of Podocytes
2026-02-11 09:07:03Arash Fatehi, David Unnersj\"o-Jess, Linus Butt, No\'emie Moreau, Thomas Benzing, Katarzyna BozekAbstract
Background: Automated podocyte foot process quantification is vital for kidney research, but the established "Automatic Morphological Analysis of Podocytes" (AMAP) method is hindered by high computational demands, a lack of a user interface, and Linux dependency. We developed AMAP-APP, a cross-platform desktop application designed to overcome these barriers. Methods: AMAP-APP optimizes efficiency by replacing intensive instance segmentation with classic image processing while retaining the original semantic segmentation model. It introduces a refined Region of Interest (ROI) algorithm to improve precision. Validation involved 365 mouse and human images (STED and confocal), benchmarking performance against the original AMAP via Pearson correlation and Two One-Sided T-tests (TOST). Results: AMAP-APP achieved a 147-fold increase in processing speed on consumer hardware. Morphometric outputs (area, perimeter, circularity, and slit diaphragm density) showed high correlation (r>0.90) and statistical equivalence (TOST P<0.05) to the original method. Additionally, the new ROI algorithm demonstrated superior accuracy compared to the original, showing reduced deviation from manual delineations. Conclusion: AMAP-APP democratizes deep learning-based podocyte morphometry. By eliminating the need for high-performance computing clusters and providing a user-friendly interface for Windows, macOS, and Linux, it enables widespread adoption in nephrology research and potential clinical diagnostics.
Abstract (translated)
背景：自动足突定量在肾脏研究中至关重要，但现有的“自动足突形态学分析”（AMAP）方法由于计算需求高、缺乏用户界面以及依赖Linux系统而受到限制。我们开发了AMAP-APP，这是一个跨平台的桌面应用程序，旨在克服这些障碍。 **方法**：AMAP-APP通过用传统的图像处理技术替代资源密集型实例分割，并保留原有的语义分割模型来提高效率。它还引入了一种改进的兴趣区域（ROI）算法以提高精度。验证过程包括使用365张来自小鼠和人类的STED和共聚焦图像，与原始AMAP方法进行皮尔逊相关性和双侧单边t检验（TOST）性能对比。 **结果**：在消费者硬件上，AMAP-APP实现了147倍的速度提升。形态学输出参数（面积、周长、圆度及裂隙隔膜密度）显示出与原始方法的高度相关性（r>0.90）和统计等效性（TOST P<0.05）。此外，新的ROI算法在精度方面优于原始版本，显示了其结果更接近于手动描绘。 **结论**：AMAP-APP使基于深度学习的足突形态学分析更加普及化。通过消除对高性能计算集群的需求，并为Windows、macOS和Linux系统提供用户友好的界面，它能够在肾病学研究中得到广泛应用，并具有潜在的临床诊断价值。
URL
https://arxiv.org/abs/2602.10663
PDF
-
Spatio-Temporal Attention for Consistent Video Semantic Segmentation in Automated Driving
2026-02-10 18:18:37Serin Varghese, Kevin Ross, Fabian Hueger, Kira MaagAbstract
Deep neural networks, especially transformer-based architectures, have achieved remarkable success in semantic segmentation for environmental perception. However, existing models process video frames independently, thus failing to leverage temporal consistency, which could significantly improve both accuracy and stability in dynamic scenes. In this work, we propose a Spatio-Temporal Attention (STA) mechanism that extends transformer attention blocks to incorporate multi-frame context, enabling robust temporal feature representations for video semantic segmentation. Our approach modifies standard self-attention to process spatio-temporal feature sequences while maintaining computational efficiency and requiring minimal changes to existing architectures. STA demonstrates broad applicability across diverse transformer architectures and remains effective across both lightweight and larger-scale models. A comprehensive evaluation on the Cityscapes and BDD100k datasets shows substantial improvements of 9.20 percentage points in temporal consistency metrics and up to 1.76 percentage points in mean intersection over union compared to single-frame baselines. These results demonstrate STA as an effective architectural enhancement for video-based semantic segmentation applications.
Abstract (translated)
深度神经网络，特别是基于变压器的架构，在环境感知中的语义分割方面取得了显著的成功。然而，现有的模型处理视频帧时是独立进行的，从而未能利用时间一致性，而这在动态场景中可以大幅提高准确性和稳定性。在此项工作中，我们提出了一种空间-时间注意（STA）机制，它扩展了变压器注意力块，以纳入多帧上下文，从而使视频语义分割具备稳健的时间特征表示能力。我们的方法将标准的自注意力处理方式修改为能够同时处理时空特征序列，同时保持计算效率，并且只需要对现有架构做出最小改动。STA在各种Transformer架构中具有广泛的适用性，并且无论模型是轻量级还是大规模，在所有情况下均能有效运行。在Cityscapes和BDD100k数据集上的全面评估显示，与单帧基线相比，时间一致性指标提高了9.20个百分点，平均交并比（mean Intersection over Union）最高提升了1.76个百分点。这些结果表明STA作为一种架构增强手段，在基于视频的语义分割应用中具有有效性。
URL
https://arxiv.org/abs/2602.10052
PDF
-
VersaViT: Enhancing MLLM Vision Backbones via Task-Guided Optimization
2026-02-10 16:08:19Yikun Liu, Yuan Liu, Shangzhe Di, Haicheng Wang, Zhongyin Zhao, Le Tian, Xiao Zhou, Jie Zhou, Jiangchao Yao, Yanfeng Wang, Weidi XieAbstract
Multimodal Large Language Models (MLLMs) have recently achieved remarkable success in visual-language understanding, demonstrating superior high-level semantic alignment within their vision encoders. An important question thus arises: Can these encoders serve as versatile vision backbones, capable of reliably performing classic vision-centric tasks as well? To address the question, we make the following contributions: (i) we identify that the vision encoders within MLLMs exhibit deficiencies in their dense feature representations, as evidenced by their suboptimal performance on dense prediction tasks (e.g., semantic segmentation, depth estimation); (ii) we propose VersaViT, a well-rounded vision transformer that instantiates a novel multi-task framework for collaborative post-training. This framework facilitates the optimization of the vision backbone via lightweight task heads with multi-granularity supervision; (iii) extensive experiments across various downstream tasks demonstrate the effectiveness of our method, yielding a versatile vision backbone suited for both language-mediated reasoning and pixel-level understanding.
Abstract (translated)
多模态大型语言模型（MLLM）近期在视觉-语言理解方面取得了显著成功，展示了其视觉编码器中出色的高级语义对齐能力。由此引发了一个重要的问题：这些编码器能否作为通用的视觉骨干网络，可靠地执行经典的以视觉为中心的任务？为了解决这个问题，我们做出了以下贡献： (i) 我们识别出多模态大型语言模型中的视觉编码器在密集特征表示方面存在不足，在密集预测任务（如语义分割、深度估计）中表现出性能不佳。 (ii) 我们提出了一种全面的视觉变换器——VersaViT，该变换器实例化了一个新颖的多任务框架，用于协作式后期训练。此框架通过带有多种粒度监督的轻量级任务头优化视觉骨干网络。 (iii) 在各种下游任务中的广泛实验表明了我们方法的有效性，生成了一种适用于语言介导推理和像素级理解的通用视觉骨干网络。 通过以上贡献，VersaViT不仅提升了MLLM在视觉密集预测方面的性能，还为开发能够同时支持高层次语义理解和精确像素级分析的多功能视觉模型铺平了道路。
URL
https://arxiv.org/abs/2602.09934
PDF
-
Time2General: Learning Spatiotemporal Invariant Representations for Domain-Generalization Video Semantic Segmentation
2026-02-10 10:55:25Siyu Chen, Ting Han, Haoling Huang, Chaolei Wang, Chengzheng Fu, Duxin Zhu, Guorong Cai, Jinhe SuAbstract
Domain Generalized Video Semantic Segmentation (DGVSS) is trained on a single labeled driving domain and is directly deployed on unseen domains without target labels and test-time adaptation while maintaining temporally consistent predictions over video streams. In practice, both domain shift and temporal-sampling shift break correspondence-based propagation and fixed-stride temporal aggregation, causing severe frame-to-frame flicker even in label-stable regions. We propose Time2General, a DGVSS framework built on Stability Queries. Time2General introduces a Spatio-Temporal Memory Decoder that aggregates multi-frame context into a clip-level spatio-temporal memory and decodes temporally consistent per-frame masks without explicit correspondence propagation. To further suppress flicker and improve robustness to varying sampling rates, the Masked Temporal Consistency Loss is proposed to regularize temporal prediction discrepancies across different strides, and randomize training strides to expose the model to diverse temporal gaps. Extensive experiments on multiple driving benchmarks show that Time2General achieves a substantial improvement in cross-domain accuracy and temporal stability over prior DGSS and VSS baselines while running at up to 18 FPS. Code will be released after the review process.
Abstract (translated)
领域泛化的视频语义分割（Domain Generalized Video Semantic Segmentation, DGVSS）是在单一标注的驾驶域上训练，并直接在无目标标签和测试时间适应的未知域上部署，同时保持视频流中一致的时间预测。实践中，无论是领域的转变还是采样率的变化都会打破基于对应传播和固定步幅的时间聚合，即使在标签稳定的区域也会导致严重的帧间闪烁问题。 我们提出了Time2General框架，它建立在稳定性查询的基础上，并针对DGVSS设计了一个时空记忆解码器（Spatio-Temporal Memory Decoder），该解码器可以将多帧上下文信息聚集为一段视频级别的时空内存，并无需显式的对应传播来解码出一致的时间预测的每帧掩模。为了进一步抑制闪烁并提高对不同采样率变化的鲁棒性，我们提出了遮罩时间一致性损失（Masked Temporal Consistency Loss），以规范不同时步中的时间预测差异，并随机化训练步骤以使模型接触各种各样的时间间隔。 在多个驾驶基准测试中进行的广泛实验表明，Time2General在跨域精度和时间稳定性上相对于之前的DGSS和VSS基线取得了显著改进，并且可以在高达18 FPS的速度下运行。代码将在审查过程后发布。
URL
https://arxiv.org/abs/2602.09648
PDF
