Segmentation
-
Deep mineralogical segmentation of thin section images based on QEMSCAN maps
2025-05-22 17:58:34Jean Pablo Vieira de Mello, Matheus Augusto Alves Cuglieri, Leandro P. de Figueiredo, Fernando Bordignon, Marcelo Ramalho Albuquerque, Rodrigo Surmas, Bruno Cavalcanti de PaulaAbstract
Interpreting the mineralogical aspects of rock thin sections is an important task for oil and gas reservoirs evaluation. However, human analysis tend to be subjective and laborious. Technologies like QEMSCAN(R) are designed to automate the mineralogical mapping process, but also suffer from limitations like high monetary costs and time-consuming analysis. This work proposes a Convolutional Neural Network model for automatic mineralogical segmentation of thin section images of carbonate rocks. The model is able to mimic the QEMSCAN mapping itself in a low-cost, generalized and efficient manner. For this, the U-Net semantic segmentation architecture is trained on plane and cross polarized thin section images using the corresponding QEMSCAN maps as target, which is an approach not widely explored. The model was instructed to differentiate occurrences of Calcite, Dolomite, Mg-Clay Minerals, Quartz, Pores and the remaining mineral phases as an unique class named "Others", while it was validated on rock facies both seen and unseen during training, in order to address its generalization capability. Since the images and maps are provided in different resolutions, image registration was applied to align then spatially. The study reveals that the quality of the segmentation is very much dependent on these resolution differences and on the variety of learnable rock textures. However, it shows promising results, especially with regard to the proper delineation of minerals boundaries on solid textures and precise estimation of the minerals distributions, describing a nearly linear relationship between expected and predicted distributions, with coefficient of determination (R^2) superior to 0.97 for seen facies and 0.88 for unseen.
Abstract (translated)
岩石薄片的矿物学分析对于油气储层评价是一项重要任务。然而，人工分析往往主观且耗时。虽然诸如QEMSCAN(R)等技术旨在自动化矿物学测绘过程，但它们也存在高成本和耗时的问题。本研究提出了一种基于卷积神经网络（Convolutional Neural Network, CNN）的模型，用于自动分割碳酸盐岩薄片图像中的矿物区域，该方法以低成本、通用且高效的方式模拟了QEMSCAN制图流程。 为了实现这一目标，使用U-Net语义分割架构对平面偏光和交叉偏光下的岩石薄片图像进行训练，并将对应的QEMSCAN地图作为目标。这种基于QEMSCAN地图的训练方式在研究中不常被采用。模型被设定为区分方解石、白云石、镁质粘土矿物、石英、孔隙以及其他矿物相（标记为“Others”）。为了评估其泛化能力，该模型不仅对训练期间见过的岩相进行了验证，还测试了未见过的岩相。 由于图像和地图提供时分辨率不同，应用图像配准技术使它们在空间上对齐。研究表明，分割质量很大程度上取决于这些分辨率差异以及可学习岩石纹理的多样性。然而，研究结果显示出令人鼓舞的结果，特别是在固体纹理中矿物边界划定得当，并且能精确估计矿物分布。该模型预测与实际分布之间呈现出近似线性关系，对于见过和未见岩相分别获得了0.97以上的决定系数（R^2）值。 总的来说，这项工作通过利用深度学习技术提供了低成本、高效率的岩石薄片矿物学分析方法，展示了在油气储层评价中的潜在应用价值。
URL
https://arxiv.org/abs/2505.17008
PDF
-
Native Segmentation Vision Transformers
2025-05-22 17:55:20Guillem Bras\'o, Aljo\v{s}a O\v{s}ep, Laura Leal-Taix\'eAbstract
Uniform downsampling remains the de facto standard for reducing spatial resolution in vision backbones. In this work, we propose an alternative design built around a content-aware spatial grouping layer, that dynamically assigns tokens to a reduced set based on image boundaries and their semantic content. Stacking our grouping layer across consecutive backbone stages results in hierarchical segmentation that arises natively in the feature extraction process, resulting in our coined Native Segmentation Vision Transformer. We show that a careful design of our architecture enables the emergence of strong segmentation masks solely from grouping layers, that is, without additional segmentation-specific heads. This sets the foundation for a new paradigm of native, backbone-level segmentation, which enables strong zero-shot results without mask supervision, as well as a minimal and efficient standalone model design for downstream segmentation tasks. Our project page is this https URL.
Abstract (translated)
均匀下采样一直是降低视觉骨干网络空间分辨率的事实标准。在本工作中，我们提出了一种基于内容感知的空间分组层的替代设计方案，该设计可以根据图像边界及其语义内容动态地将标记分配给一个缩小的集合中。在整个连续骨干阶段堆叠我们的分组层会产生一种层次化分割，这种分割自然出现在特征提取过程中，从而形成了我们提出的原生分割视觉变换器（Native Segmentation Vision Transformer）。我们展示了对架构进行精心设计可以使仅通过分组层就能产生强大的分割掩码，而无需额外的特定于分割的头部。这为新的原生骨干级分割范式奠定了基础，该范式可以在没有掩码监督的情况下实现强大的零样本结果，并且对于下游分割任务具有最小和高效的独立模型设计。我们的项目页面在此 [URL]。 注：原文中的项目页面链接（https URL）未给出具体网址，在实际引用时需要提供完整的网址信息。
URL
https://arxiv.org/abs/2505.16993
PDF
-
Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
2025-05-22 17:54:30Moru Liu, Hao Dong, Jessica Kelly, Olga Fink, Mario TrappAbstract
Out-of-distribution (OOD) detection and segmentation are crucial for deploying machine learning models in safety-critical applications such as autonomous driving and robot-assisted surgery. While prior research has primarily focused on unimodal image data, real-world applications are inherently multimodal, requiring the integration of multiple modalities for improved OOD detection. A key challenge is the lack of supervision signals from unknown data, leading to overconfident predictions on OOD samples. To address this challenge, we propose Feature Mixing, an extremely simple and fast method for multimodal outlier synthesis with theoretical support, which can be further optimized to help the model better distinguish between in-distribution (ID) and OOD data. Feature Mixing is modality-agnostic and applicable to various modality combinations. Additionally, we introduce CARLA-OOD, a novel multimodal dataset for OOD segmentation, featuring synthetic OOD objects across diverse scenes and weather conditions. Extensive experiments on SemanticKITTI, nuScenes, CARLA-OOD datasets, and the MultiOOD benchmark demonstrate that Feature Mixing achieves state-of-the-art performance with a $10 \times$ to $370 \times$ speedup. Our source code and dataset will be available at this https URL.
Abstract (translated)
出界（Out-of-distribution，OOD）检测和分割对于在自动驾驶和机器人辅助手术等安全关键应用中部署机器学习模型至关重要。尽管之前的大多数研究主要集中在单模态图像数据上，但现实世界的应用本质上是多模态的，需要整合多种模态以提高OOD检测的效果。一个关键挑战是没有来自未知数据的监督信号，导致模型在处理OOD样本时过于自信。为解决这一挑战，我们提出了特征混合（Feature Mixing）方法，这是一种极其简单且快速的方法，用于生成具有理论支持的多模态异常值，可以通过进一步优化帮助模型更好地区分已知分布（in-distribution，ID）和OOD数据。特征混合与模式无关，并适用于各种模态组合。 此外，我们还介绍了CARLA-OOD，这是一个新颖的多模态数据集，用于OOD分割任务，其中包含在不同场景和天气条件下合成的OOD物体。在SemanticKITTI、nuScenes、CARLA-OOD以及MultiOOD基准测试上进行的大量实验表明，特征混合方法能够实现最先进的性能，并且速度提高了10倍到370倍。我们的源代码和数据集将在[此处](https://this https URL)提供。 该段落翻译为中文后清晰地介绍了研究背景、提出的方法及其优势，以及用于验证新方法的数据集和实验结果。
URL
https://arxiv.org/abs/2505.16985
PDF
-
OpenSeg-R: Improving Open-Vocabulary Segmentation via Step-by-Step Visual Reasoning
2025-05-22 17:51:48Zongyan Han, Jiale Cao, Shuo Chen, Tong Wang, Jorma Laaksonen, Rao Muhammad AnwerAbstract
Open-Vocabulary Segmentation (OVS) has drawn increasing attention for its capacity to generalize segmentation beyond predefined categories. However, existing methods typically predict segmentation masks with simple forward inference, lacking explicit reasoning and interpretability. This makes it challenging for OVS model to distinguish similar categories in open-world settings due to the lack of contextual understanding and discriminative visual cues. To address this limitation, we propose a step-by-step visual reasoning framework for open-vocabulary segmentation, named OpenSeg-R. The proposed OpenSeg-R leverages Large Multimodal Models (LMMs) to perform hierarchical visual reasoning before segmentation. Specifically, we generate both generic and image-specific reasoning for each image, forming structured triplets that explain the visual reason for objects in a coarse-to-fine manner. Based on these reasoning steps, we can compose detailed description prompts, and feed them to the segmentor to produce more accurate segmentation masks. To the best of our knowledge, OpenSeg-R is the first framework to introduce explicit step-by-step visual reasoning into OVS. Experimental results demonstrate that OpenSeg-R significantly outperforms state-of-the-art methods on open-vocabulary semantic segmentation across five benchmark datasets. Moreover, it achieves consistent gains across all metrics on open-vocabulary panoptic segmentation. Qualitative results further highlight the effectiveness of our reasoning-guided framework in improving both segmentation precision and interpretability. Our code is publicly available at this https URL.
Abstract (translated)
开放词汇分割（OVS）因其能够将分割任务推广到预定义类别之外的能力而越来越受到关注。然而，现有的方法通常通过简单的前向推理来预测分割掩码，缺乏明确的推理过程和可解释性。这使得OVS模型在开放式环境中难以区分相似的类别，因为缺少上下文理解和具有判别性的视觉线索。 为了解决这一局限性，我们提出了一种逐步视觉推理框架，用于开放词汇分割，并将其命名为OpenSeg-R。所提出的OpenSeg-R利用大型多模态模型（LMMs）来进行分层视觉推理，在进行分割之前完成该步骤。具体而言，针对每张图像生成通用和特定于图像的推理内容，形成结构化三元组，以从粗到细的方式解释对象的视觉原因。基于这些推理步骤，我们可以合成详细的描述提示，并将其输入到分割器中，从而产生更准确的分割掩码。 据我们所知，OpenSeg-R是第一个将明确的逐步视觉推理引入OVS框架的方法。实验结果表明，在五个基准数据集上的开放词汇语义分割任务上，OpenSeg-R显著优于当前最佳方法。此外，在开放词汇全景分割的所有指标中也取得了持续性的改进。定性结果显示了我们推理引导框架在提高分割精度和可解释性方面的有效性。 我们的代码可在以下链接获取：[提供URL的地方]（请替换为实际的代码公开链接）。
URL
https://arxiv.org/abs/2505.16974
PDF
-
BP-Seg: A graphical model approach to unsupervised and non-contiguous text segmentation using belief propagation
2025-05-22 17:46:23Fengyi Li, Kayhan Behdin, Natesh Pillai, Xiaofeng Wang, Zhipeng Wang, Ercan YildizAbstract
Text segmentation based on the semantic meaning of sentences is a fundamental task with broad utility in many downstream applications. In this paper, we propose a graphical model-based unsupervised learning approach, named BP-Seg for efficient text segmentation. Our method not only considers local coherence, capturing the intuition that adjacent sentences are often more related, but also effectively groups sentences that are distant in the text yet semantically similar. This is achieved through belief propagation on the carefully constructed graphical models. Experimental results on both an illustrative example and a dataset with long-form documents demonstrate that our method performs favorably compared to competing approaches.
Abstract (translated)
基于句子语义意义的文本分段是一项具有广泛应用价值的基本任务。在本文中，我们提出了一种基于图模型的无监督学习方法，命名为BP-Seg，用于高效的文本分割。我们的方法不仅考虑了局部连贯性（即相邻句子通常关系更紧密），还能够有效归类那些虽然在文本中相距较远但语义相似的句子。这是通过在精心构建的图形模型上进行信念传播来实现的。实验结果表明，在一个示例数据集和包含长篇文档的数据集上，我们的方法相较于竞争方法表现出色。
URL
https://arxiv.org/abs/2505.16965
PDF
-
NovelSeek: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification
2025-05-22 17:27:43NovelSeek Team, Bo Zhang, Shiyang Feng, Xiangchao Yan, Jiakang Yuan, Zhiyin Yu, Xiaohan He, Songtao Huang, Shaowei Hou, Zheng Nie, Zhilong Wang, Jinyao Liu, Runmin Ma, Tianshuo Peng, Peng Ye, Dongzhan Zhou, Shufei Zhang, Xiaosong Wang, Yilan Zhang, Meng Li, Zhongying Tu, Xiangyu Yue, Wangli Ouyang, Bowen Zhou, Lei BaiAbstract
Artificial Intelligence (AI) is accelerating the transformation of scientific research paradigms, not only enhancing research efficiency but also driving innovation. We introduce NovelSeek, a unified closed-loop multi-agent framework to conduct Autonomous Scientific Research (ASR) across various scientific research fields, enabling researchers to tackle complicated problems in these fields with unprecedented speed and precision. NovelSeek highlights three key advantages: 1) Scalability: NovelSeek has demonstrated its versatility across 12 scientific research tasks, capable of generating innovative ideas to enhance the performance of baseline code. 2) Interactivity: NovelSeek provides an interface for human expert feedback and multi-agent interaction in automated end-to-end processes, allowing for the seamless integration of domain expert knowledge. 3) Efficiency: NovelSeek has achieved promising performance gains in several scientific fields with significantly less time cost compared to human efforts. For instance, in reaction yield prediction, it increased from 27.6% to 35.4% in just 12 hours; in enhancer activity prediction, accuracy rose from 0.52 to 0.79 with only 4 hours of processing; and in 2D semantic segmentation, precision advanced from 78.8% to 81.0% in a mere 30 hours.
Abstract (translated)
人工智能（AI）正在加速科研范式的转变，不仅提升了研究效率，还推动了创新。我们推出了NovelSeek，这是一个统一的闭环多智能体框架，用于在多个科学领域中进行自主科学研究（ASR），使研究人员能够以前所未有的速度和精度解决这些领域的复杂问题。NovelSeek突出三大优势： 1. **可扩展性**：NovelSeek已在12项科研任务中展示了其适应能力，能够在多种基线代码的性能提升方面生成创新想法。 2. **交互性**：NovelSeek提供了一个接口，支持人类专家反馈和多智能体互动，在自动化端到端过程中能够无缝集成领域专业知识。 3. **效率**：相比人工努力，NovelSeek在多个科学领域中实现了显著的时间成本节约，并取得了令人瞩目的性能提升。例如，在反应产率预测方面，其性能从27.6%提升至35.4%，仅耗时12小时；在增强子活性预测上，准确度从0.52升至0.79，仅需4小时的处理时间；而在二维语义分割领域，精度提升了近三个百分点，在短短30小时内由78.8%提高到81.0%。
URL
https://arxiv.org/abs/2505.16938
PDF
-
Hypergraph Tversky-Aware Domain Incremental Learning for Brain Tumor Segmentation with Missing Modalities
2025-05-22 15:49:25Junze Wang (College of Computer and Control Engineering, Northeast Forestry University, Harbin, China), Lei Fan (The Centre for Healthy Brain Ageing, School of Computer Science and Engineering, University of New South Wales, Sydney, Australia), Weipeng Jing (College of Computer and Control Engineering, Northeast Forestry University, Harbin, China), Donglin Di (Space AI, Li Auto, Beijing, China), Yang Song (School of Computer Science and Engineering, University of New South Wales, Sydney, Australia), Sidong Liu (Centre for Health Informatics, Macquarie University, Sydney, Australia), Cong Cong (Centre for Health Informatics, Macquarie University, Sydney, Australia)Abstract
Existing methods for multimodal MRI segmentation with missing modalities typically assume that all MRI modalities are available during training. However, in clinical practice, some modalities may be missing due to the sequential nature of MRI acquisition, leading to performance degradation. Furthermore, retraining models to accommodate newly available modalities can be inefficient and may cause overfitting, potentially compromising previously learned knowledge. To address these challenges, we propose Replay-based Hypergraph Domain Incremental Learning (ReHyDIL) for brain tumor segmentation with missing modalities. ReHyDIL leverages Domain Incremental Learning (DIL) to enable the segmentation model to learn from newly acquired MRI modalities without forgetting previously learned information. To enhance segmentation performance across diverse patient scenarios, we introduce the Cross-Patient Hypergraph Segmentation Network (CHSNet), which utilizes hypergraphs to capture high-order associations between patients. Additionally, we incorporate Tversky-Aware Contrastive (TAC) loss to effectively mitigate information imbalance both across and within different modalities. Extensive experiments on the BraTS2019 dataset demonstrate that ReHyDIL outperforms state-of-the-art methods, achieving an improvement of over 2\% in the Dice Similarity Coefficient across various tumor regions. Our code is available at ReHyDIL.
Abstract (translated)
现有的多模态MRI分割方法通常假设在训练期间所有MRI模态都可用。然而，在临床实践中，由于MRI采集的顺序特性，某些模态可能会缺失，从而导致性能下降。重新训练模型以适应新出现的模态既低效又可能导致过拟合，这可能破坏之前学到的知识。为了解决这些挑战，我们提出了基于重播的超图领域增量学习（ReHyDIL），用于具有缺失模态的大脑肿瘤分割。ReHyDIL利用领域增量学习（DIL）使分割模型能够从新获取的MRI模态中学习而不忘记先前学到的信息。 为了增强在各种患者场景中的分割性能，我们引入了跨患者的超图分割网络（CHSNet），该网络使用超图来捕捉不同患者之间的高阶关联。此外，我们还采用了Tversky感知对比损失(TAC)以有效缓解不同模态之间以及内部的信息不平衡问题。 在BraTS2019数据集上的广泛实验表明，ReHyDIL优于最先进的方法，在各个肿瘤区域的Dice相似系数上取得了超过2%的改进。我们的代码可在ReHyDIL项目中获取。
URL
https://arxiv.org/abs/2505.16809
PDF
-
Learning Beyond Limits: Multitask Learning and Synthetic Data for Low-Resource Canonical Morpheme Segmentation
2025-05-22 15:40:09Changbing Yang, Garrett NicolaiAbstract
We introduce a transformer-based morpheme segmentation system that augments a low-resource training signal through multitask learning and LLM-generated synthetic data. Our framework jointly predicts morphological segments and glosses from orthographic input, leveraging shared linguistic representations obtained through a common documentary process to enhance model generalization. To further address data scarcity, we integrate synthetic training data generated by large language models (LLMs) using in-context learning. Experimental results on the SIGMORPHON 2023 dataset show that our approach significantly improves word-level segmentation accuracy and morpheme-level F1-score across multiple low-resource languages.
Abstract (translated)
我们介绍了一种基于变压器的词素分割系统，该系统通过多任务学习和大型语言模型生成的合成数据来增强低资源训练信号。我们的框架从正写形式输入中同时预测形态段落和释义，利用共同文档过程中获得的共享语言表示来提升模型的泛化能力。为了进一步解决数据稀缺问题，我们整合了使用上下文学习由大型语言模型（LLM）生成的合成训练数据。在SIGMORPHON 2023数据集上的实验结果表明，我们的方法显著提高了多种低资源语言中的词级别分割准确性和词素级别的F1值。
URL
https://arxiv.org/abs/2505.16800
PDF
-
Towards Texture- And Shape-Independent 3D Keypoint Estimation in Birds
2025-05-22 13:04:24Valentin Schmuker, Alex Hoi Hang Chan, Bastian Goldluecke, Urs WaldmannAbstract
In this paper, we present a texture-independent approach to estimate and track 3D joint positions of multiple pigeons. For this purpose, we build upon the existing 3D-MuPPET framework, which estimates and tracks the 3D poses of up to 10 pigeons using a multi-view camera setup. We extend this framework by using a segmentation method that generates silhouettes of the individuals, which are then used to estimate 2D keypoints. Following 3D-MuPPET, these 2D keypoints are triangulated to infer 3D poses, and identities are matched in the first frame and tracked in 2D across subsequent frames. Our proposed texture-independent approach achieves comparable accuracy to the original texture-dependent 3D-MuPPET framework. Additionally, we explore our approach's applicability to other bird species. To do that, we infer the 2D joint positions of four bird species without additional fine-tuning the model trained on pigeons and obtain preliminary promising results. Thus, we think that our approach serves as a solid foundation and inspires the development of more robust and accurate texture-independent pose estimation frameworks.
Abstract (translated)
在这篇论文中，我们提出了一种基于纹理无关的方法来估计和追踪多只鸽子的三维关节位置。为此，我们在现有的3D-MuPPET框架基础上进行扩展，该框架使用一个多视角摄像头设置，可以估算并跟踪多达10只鸽子的三维姿态。我们将此框架扩展为一种分割方法，该方法生成个体轮廓，并将这些轮廓用于估计二维关键点。接着，遵循3D-MuPPET的方法，我们对这些二维关键点进行三角测量以推断出三维姿势，并在第一帧中匹配身份并在后续帧中对其进行二维跟踪。 我们的纹理无关方法达到了与原始的基于纹理的3D-MuPPET框架相当的准确度。此外，我们还探讨了该方法在其他鸟类物种上的应用性。为此，我们在不额外微调训练鸽子模型的情况下推断出四种不同鸟类的二维关节位置，并获得了初步有前景的结果。 因此，我们认为我们的方法为开发更稳健且精确的纹理无关姿势估计框架奠定了坚实的基础，并具有启发意义。
URL
https://arxiv.org/abs/2505.16633
PDF
-
Background Matters: A Cross-view Bidirectional Modeling Framework for Semi-supervised Medical Image Segmentation
2025-05-22 12:59:45Luyang Cao, Jianwei Li, Yinghuan ShiAbstract
Semi-supervised medical image segmentation (SSMIS) leverages unlabeled data to reduce reliance on manually annotated images. However, current SOTA approaches predominantly focus on foreground-oriented modeling (i.e., segmenting only the foreground region) and have largely overlooked the potential benefits of explicitly modeling the background region. Our study theoretically and empirically demonstrates that highly certain predictions in background modeling enhance the confidence of corresponding foreground modeling. Building on this insight, we propose the Cross-view Bidirectional Modeling (CVBM) framework, which introduces a novel perspective by incorporating background modeling to improve foreground modeling performance. Within CVBM, background modeling serves as an auxiliary perspective, providing complementary supervisory signals to enhance the confidence of the foreground model. Additionally, CVBM introduces an innovative bidirectional consistency mechanism, which ensures mutual alignment between foreground predictions and background-guided predictions. Extensive experiments demonstrate that our approach achieves SOTA performance on the LA, Pancreas, ACDC, and HRF datasets. Notably, on the Pancreas dataset, CVBM outperforms fully supervised methods (i.e., DSC: 84.57% vs. 83.89%) while utilizing only 20% of the labeled data. Our code is publicly available at this https URL.
Abstract (translated)
半监督医学图像分割（SSMIS）利用未标记的数据来减少对手动标注图像的依赖。然而，当前最先进的方法主要集中在前景导向建模上（即仅分割前景区域），并且忽略了显式建模背景区域的巨大潜力。我们的研究表明，在背景建模中做出高确定性的预测可以增强相应前景建模的信心。基于这一见解，我们提出了跨视角双向建模（CVBM）框架，该框架通过将背景建模纳入其中，以创新的方式提高了前景建模的性能。在CVBM中，背景建模作为一个辅助视角发挥作用，提供互补监督信号来增强前景模型的信心。此外，CVBM引入了一种创新的双向一致性机制，确保了前景预测与基于背景指导的预测之间的相互对齐。广泛的实验表明，我们的方法在LA、胰腺、ACDC和HRF数据集上实现了最先进的性能。值得注意的是，在胰腺数据集中，CVBM在仅使用20%标签数据的情况下，优于完全监督的方法（即DSC：84.57％ vs 83.89％）。我们的代码可在以下网址公开获取：[此URL]。 请注意，上述引用的“此URL”需要您手动填写实际发布的代码链接。
URL
https://arxiv.org/abs/2505.16625
PDF
-
Auto-nnU-Net: Towards Automated Medical Image Segmentation
2025-05-22 11:52:16Jannis Becktepe, Leona Hennig, Steffen Oeltze-Jafra, Marius LindauerAbstract
Medical Image Segmentation (MIS) includes diverse tasks, from bone to organ segmentation, each with its own challenges in finding the best segmentation model. The state-of-the-art AutoML-related MIS-framework nnU-Net automates many aspects of model configuration but remains constrained by fixed hyperparameters and heuristic design choices. As a full-AutoML framework for MIS, we propose Auto-nnU-Net, a novel nnU-Net variant enabling hyperparameter optimization (HPO), neural architecture search (NAS), and hierarchical NAS (HNAS). Additionally, we propose Regularized PriorBand to balance model accuracy with the computational resources required for training, addressing the resource constraints often faced in real-world medical settings that limit the feasibility of extensive training procedures. We evaluate our approach across diverse MIS datasets from the well-established Medical Segmentation Decathlon, analyzing the impact of AutoML techniques on segmentation performance, computational efficiency, and model design choices. The results demonstrate that our AutoML approach substantially improves the segmentation performance of nnU-Net on 6 out of 10 datasets and is on par on the other datasets while maintaining practical resource requirements. Our code is available at this https URL.
Abstract (translated)
医学图像分割（MIS）涵盖了从骨骼到器官的各种任务，每种任务在寻找最佳分割模型时都有其独特的挑战。目前最先进的与AutoML相关的MIS框架nnU-Net自动配置了许多模型参数，但仍受限于固定的超参数和启发式设计选择。作为面向MIS的全自动化机器学习（AutoML）框架，我们提出了一种新的nnU-Net变体——Auto-nnU-Net，该框架支持超参数优化（HPO）、神经架构搜索（NAS）以及分层NAS（HNAS）。此外，我们还提出了Regularized PriorBand方法，在保证模型精度的同时考虑训练所需的计算资源，解决了实际医疗环境中由于资源限制而使广泛培训程序不可行的问题。我们在Medical Segmentation Decathlon中一系列多样化的MIS数据集上评估了我们的方法，分析了AutoML技术对分割性能、计算效率以及模型设计选择的影响。结果表明，在10个数据集中，我们的AutoML方法在6个数据集中显著提高了nnU-Net的分割性能，并且在其他四个数据集中的表现与现有方法相当，同时保持实用的资源需求。 我们提出的代码可以在以下网址获取：[请在此处插入链接]。
URL
https://arxiv.org/abs/2505.16561
PDF
-
TextureSAM: Towards a Texture Aware Foundation Model for Segmentation
2025-05-22 11:31:56Inbal Cohen, Boaz Meivar, Peihan Tu, Shai Avidan, Gal OrenAbstract
Segment Anything Models (SAM) have achieved remarkable success in object segmentation tasks across diverse datasets. However, these models are predominantly trained on large-scale semantic segmentation datasets, which introduce a bias toward object shape rather than texture cues in the image. This limitation is critical in domains such as medical imaging, material classification, and remote sensing, where texture changes define object boundaries. In this study, we investigate SAM's bias toward semantics over textures and introduce a new texture-aware foundation model, TextureSAM, which performs superior segmentation in texture-dominant scenarios. To achieve this, we employ a novel fine-tuning approach that incorporates texture augmentation techniques, incrementally modifying training images to emphasize texture features. By leveraging a novel texture-alternation of the ADE20K dataset, we guide TextureSAM to prioritize texture-defined regions, thereby mitigating the inherent shape bias present in the original SAM model. Our extensive experiments demonstrate that TextureSAM significantly outperforms SAM-2 on both natural (+0.2 mIoU) and synthetic (+0.18 mIoU) texture-based segmentation datasets. The code and texture-augmented dataset will be publicly available.
Abstract (translated)
段落翻译如下： 片段化模型（Segment Anything Models，SAM）在各种数据集上的对象分割任务中取得了显著的成功。然而，这些模型主要是在大规模语义分割数据集上进行训练的，这导致了对物体形状的偏见，而不是图像中的纹理线索。这种限制在医学影像、材料分类和遥感等域尤为关键，在这些领域中，纹理变化定义了对象边界。在这项研究中，我们探讨了SAM模型在语义方面的偏差，并引入了一个新的以纹理感知为基础的基础模型——TextureSAM，该模型在以纹理为主导的场景中表现出色。为实现这一目标，我们采用了一种新颖的微调方法，结合了纹理增强技术，逐步修改训练图像以强调纹理特征。通过利用ADE20K数据集的新颖纹理替代版本，我们指导TextureSAM优先考虑由纹理定义的区域，从而缓解原始SAM模型中存在的固有形状偏见。我们的大量实验表明，在基于自然纹理和合成纹理的分割数据集中，TextureSAM在mIoU指标上均显著优于SAM-2（自然场景+0.2 mIoU，合成场景+0.18 mIoU）。代码及纹理增强的数据集将公开提供。
URL
https://arxiv.org/abs/2505.16540
PDF
-
Detailed Evaluation of Modern Machine Learning Approaches for Optic Plastics Sorting
2025-05-22 10:48:30Vaishali Maheshkar, Aadarsh Anantha Ramakrishnan, Charuvahan Adhivarahan, Karthik DantuAbstract
According to the EPA, only 25% of waste is recycled, and just 60% of U.S. municipalities offer curbside recycling. Plastics fare worse, with a recycling rate of only 8%; an additional 16% is incinerated, while the remaining 76% ends up in landfills. The low plastic recycling rate stems from contamination, poor economic incentives, and technical difficulties, making efficient recycling a challenge. To improve recovery, automated sorting plays a critical role. Companies like AMP Robotics and Greyparrot utilize optical systems for sorting, while Materials Recovery Facilities (MRFs) employ Near-Infrared (NIR) sensors to detect plastic types. Modern optical sorting uses advances in computer vision such as object recognition and instance segmentation, powered by machine learning. Two-stage detectors like Mask R-CNN use region proposals and classification with deep backbones like ResNet. Single-stage detectors like YOLO handle detection in one pass, trading some accuracy for speed. While such methods excel under ideal conditions with a large volume of labeled training data, challenges arise in realistic scenarios, emphasizing the need to further examine the efficacy of optic detection for automated sorting. In this study, we compiled novel datasets totaling 20,000+ images from varied sources. Using both public and custom machine learning pipelines, we assessed the capabilities and limitations of optical recognition for sorting. Grad-CAM, saliency maps, and confusion matrices were employed to interpret model behavior. We perform this analysis on our custom trained models from the compiled datasets. To conclude, our findings are that optic recognition methods have limited success in accurate sorting of real-world plastics at MRFs, primarily because they rely on physical properties such as color and shape.
Abstract (translated)
根据美国环境保护署（EPA）的数据，只有25%的废物被回收利用，并且仅有60%的美国城市提供路边回收服务。塑料的表现更糟，其回收率仅为8%，另有16%被焚烧，而剩余的76%最终进入垃圾填埋场。低塑料回收率的原因包括污染、经济激励不足以及技术难度，这些因素使得高效的塑料回收变得极具挑战性。为了改善材料回收效率，自动分拣系统发挥着关键作用。 一些公司如AMP Robotics和Greyparrot利用光学系统进行分类，而物料回收设施（MRFs）则使用近红外（NIR）传感器来检测塑料类型。现代光学分类技术运用了计算机视觉的进步，比如目标识别和实例分割，并通过机器学习支持这些技术。Mask R-CNN等两阶段探测器采用区域提议与深层骨干网如ResNet相结合的方法进行分类。而像YOLO这样的单阶段探测器则能在一次处理中完成检测任务，尽管牺牲了一定的精度以换取速度。 在理想的条件和大量标记训练数据的情况下，这些方法表现出色。然而，在实际应用中，由于各种挑战，例如光线变化、标签不一致等问题，光学识别技术的有效性受到了限制。为了进一步探究这些问题，本研究收集了来自不同来源共计20,000多张图像的新型数据集，并使用公共和定制机器学习管道评估了光学识别在分拣中的能力和局限。 我们利用Grad-CAM（Gradient-weighted Class Activation Mapping）、热图以及混淆矩阵来解释模型的行为。我们在从编译的数据集中自训练的模型上进行了这种分析。最终，我们的研究发现表明，在物料回收设施中准确分类真实世界的塑料方面，光学识别方法仅取得了有限的成功，其主要原因在于它们依赖于颜色和形状等物理特性。 这项研究表明了尽管现有技术在理论上具有潜力，但在实际操作环境中有效应用仍面临诸多挑战，特别是在处理复杂多样的环境条件时。这强调了进一步开发更加智能且适应性强的分类系统的必要性，以便能够更好地应对这些现实世界的难题。
URL
https://arxiv.org/abs/2505.16513
PDF
-
ALTo: Adaptive-Length Tokenizer for Autoregressive Mask Generation
2025-05-22 10:26:51Lingfeng Wang, Hualing Lin, Senda Chen, Tao Wang, Changxu Cheng, Yangyang Zhong, Dong Zheng, Wuyue ZhaoAbstract
While humans effortlessly draw visual objects and shapes by adaptively allocating attention based on their complexity, existing multimodal large language models (MLLMs) remain constrained by rigid token representations. Bridging this gap, we propose ALTo, an adaptive length tokenizer for autoregressive mask generation. To achieve this, a novel token length predictor is designed, along with a length regularization term and a differentiable token chunking strategy. We further build ALToLLM that seamlessly integrates ALTo into MLLM. Preferences on the trade-offs between mask quality and efficiency is implemented by group relative policy optimization (GRPO). Experiments demonstrate that ALToLLM achieves state-of-the-art performance with adaptive token cost on popular segmentation benchmarks. Code and models are released at this https URL.
Abstract (translated)
当人类能够轻松地根据视觉对象和形状的复杂性灵活分配注意力时，现有的多模态大型语言模型（MLLMs）仍然受到刚性的令牌表示形式的限制。为弥合这一差距，我们提出了ALTo，这是一种自回归掩码生成的自适应长度标记器。为此设计了一个新颖的令牌长度预测器，并引入了长度正则化项和可微分的令牌分块策略。进一步地，我们构建了ALToLLM，它将ALTo无缝集成到MLLM中。通过群体相对政策优化（GRPO）实现对掩码质量和效率之间权衡偏好的实施。实验表明，在流行的分割基准测试上，ALToLLM在自适应令牌成本方面达到了最先进的性能。代码和模型可在[此链接](https://this-url.com)获取。
URL
https://arxiv.org/abs/2505.16495
PDF
-
AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformer
2025-05-22 09:44:44Jiquan Shan, Junxiao Wang, Lifeng Zhao, Liang Cai, Hongyuan Zhang, Ioannis LiritzisAbstract
Recently, vision transformers (ViTs) have achieved excellent performance on vision tasks by measuring the global self-attention among the image patches. Given $n$ patches, they will have quadratic complexity such as $\mathcal{O}(n^2)$ and the time cost is high when splitting the input image with a small granularity. Meanwhile, the pivotal information is often randomly gathered in a few regions of an input image, some tokens may not be helpful for the downstream tasks. To handle this problem, we introduce an anchor-based efficient vision transformer (AnchorFormer), which employs the anchor tokens to learn the pivotal information and accelerate the inference. Firstly, by estimating the bipartite attention between the anchors and tokens, the complexity will be reduced from $\mathcal{O}(n^2)$ to $\mathcal{O}(mn)$, where $m$ is an anchor number and $m < n$. Notably, by representing the anchors with the neurons in a neural layer, we can differentiable learn these distributions and approximate global self-attention through the Markov process. Moreover, we extend the proposed model to three downstream tasks including classification, detection, and segmentation. Extensive experiments show the effectiveness of our AnchorFormer, e.g., achieving up to a 9.0% higher accuracy or 46.7% FLOPs reduction on ImageNet classification, 81.3% higher mAP on COCO detection under comparable FLOPs, as compared to the current baselines.
Abstract (translated)
最近，视觉变换器（ViT）通过测量图像块之间的全局自注意力，在视觉任务中取得了卓越的性能。给定 $n$ 个图像块时，这类模型具有二次复杂度如 $\mathcal{O}(n^2)$，当输入图像以小颗粒度进行分割时时间成本会变得非常高。同时，关键信息往往随机分布在输入图像中的几个区域里，有些标记对于下游任务来说可能并不重要。为了解决这些问题，我们引入了一种基于锚点的高效视觉变换器（AnchorFormer），该模型通过使用锚点来学习关键信息并加速推理过程。 首先，通过估计锚点与标记之间的双部图注意力，复杂度可以从 $\mathcal{O}(n^2)$ 减少到 $\mathcal{O}(mn)$，其中 $m$ 是锚点的数量且满足 $m < n$。值得注意的是，通过用神经网络层中的神经元表示这些锚点，我们可以对它们进行可微学习，并通过马尔科夫过程近似全局自注意力。 此外，我们将提出的模型扩展到了包括分类、检测和分割在内的三个下游任务中。广泛的实验表明了我们AnchorFormer的有效性，例如，在ImageNet分类上达到了比现有基准高出最多9.0%的准确率或减少46.7%的FLOPs（浮点运算次数），在COCO检测中的mAP值也比当前基线高出了81.3%，并且是在与之相当的计算量条件下实现的。
URL
https://arxiv.org/abs/2505.16463
PDF
-
CMRINet: Joint Groupwise Registration and Segmentation for Cardiac Function Quantification from Cine-MRI
2025-05-22 09:36:42Mohamed S. Elmahdy, Marius Staring, Patrick J. H. de Koning, Samer Alabed, Mahan Salehi, Faisal Alandejani, Michael Sharkey, Ziad Aldabbagh, Andrew J. Swift, Rob J. van der GeestAbstract
Accurate and efficient quantification of cardiac function is essential for the estimation of prognosis of cardiovascular diseases (CVDs). One of the most commonly used metrics for evaluating cardiac pumping performance is left ventricular ejection fraction (LVEF). However, LVEF can be affected by factors such as inter-observer variability and varying pre-load and after-load conditions, which can reduce its reproducibility. Additionally, cardiac dysfunction may not always manifest as alterations in LVEF, such as in heart failure and cardiotoxicity diseases. An alternative measure that can provide a relatively load-independent quantitative assessment of myocardial contractility is myocardial strain and strain rate. By using LVEF in combination with myocardial strain, it is possible to obtain a thorough description of cardiac function. Automated estimation of LVEF and other volumetric measures from cine-MRI sequences can be achieved through segmentation models, while strain calculation requires the estimation of tissue displacement between sequential frames, which can be accomplished using registration models. These tasks are often performed separately, potentially limiting the assessment of cardiac function. To address this issue, in this study we propose an end-to-end deep learning (DL) model that jointly estimates groupwise (GW) registration and segmentation for cardiac cine-MRI images. The proposed anatomically-guided Deep GW network was trained and validated on a large dataset of 4-chamber view cine-MRI image series of 374 subjects. A quantitative comparison with conventional GW registration using elastix and two DL-based methods showed that the proposed model improved performance and substantially reduced computation time.
Abstract (translated)
心脏功能的准确和高效量化对于心血管疾病（CVD）预后的评估至关重要。评价心脏泵血性能最常用的指标之一是左心室射血分数（LVEF）。然而，LVEF可能会受到如观察者间变异性和不同前后负荷条件等因素的影响，这可能降低其可重复性。此外，并非所有的心脏功能障碍都会表现为LVEF的变化，例如在心脏衰竭和心毒性疾病中。另一种能够提供相对独立于负荷的定量评估心肌收缩力的指标是心肌应变和应变速率。通过结合使用LVEF与心肌应变，可以全面描述心脏的功能状态。 自动从电影磁共振成像（cine-MRI）序列估算LVEF和其他容积测量可以通过分割模型实现，而应变计算则需要估计连续帧之间组织的位移，这可以通过配准模型完成。这些任务通常分别进行，可能会限制对心脏功能评估的效果。为解决这一问题，在本研究中我们提出了一种端到端深度学习（DL）模型，该模型能够同时联合估算心脏cine-MRI图像的组内注册和分割。所提出的基于解剖学引导的深层组配网络在374名受试者的四腔心电影MRI序列的大数据集上进行了训练和验证。与使用Elastix的传统组内配准以及两种深度学习方法进行的定量比较显示，提出的方法提高了性能，并大幅减少了计算时间。
URL
https://arxiv.org/abs/2505.16452
PDF
-
Sketchy Bounding-box Supervision for 3D Instance Segmentation
2025-05-22 08:49:49Qian Deng, Le Hui, Jin Xie, Jian YangAbstract
Bounding box supervision has gained considerable attention in weakly supervised 3D instance segmentation. While this approach alleviates the need for extensive point-level annotations, obtaining accurate bounding boxes in practical applications remains challenging. To this end, we explore the inaccurate bounding box, named sketchy bounding box, which is imitated through perturbing ground truth bounding box by adding scaling, translation, and rotation. In this paper, we propose Sketchy-3DIS, a novel weakly 3D instance segmentation framework, which jointly learns pseudo labeler and segmentator to improve the performance under the sketchy bounding-box supervisions. Specifically, we first propose an adaptive box-to-point pseudo labeler that adaptively learns to assign points located in the overlapped parts between two sketchy bounding boxes to the correct instance, resulting in compact and pure pseudo instance labels. Then, we present a coarse-to-fine instance segmentator that first predicts coarse instances from the entire point cloud and then learns fine instances based on the region of coarse instances. Finally, by using the pseudo instance labels to supervise the instance segmentator, we can gradually generate high-quality instances through joint training. Extensive experiments show that our method achieves state-of-the-art performance on both the ScanNetV2 and S3DIS benchmarks, and even outperforms several fully supervised methods using sketchy bounding boxes. Code is available at this https URL.
Abstract (translated)
边界框监督在弱监督下的三维实例分割中受到了广泛的关注。尽管这种方法减轻了对大量点级别标注的需求，但在实际应用中获取准确的边界框仍然具有挑战性。为此，我们探索了一种不精确的边界框，称为草图边界框，它是通过对地面真实边界框添加缩放、平移和旋转扰动来模仿得到的。 在本文中，我们提出了一种新颖的弱监督三维实例分割框架——Sketchy-3DIS，该框架通过联合学习伪标签生成器（pseudo labeler）和分割器（segmentator），以改善草图边界框指导下的性能。具体而言，首先我们提出了一个自适应的框到点的伪标签生成器，它能够自适应地将位于两个不精确边界框重叠部分中的点分配给正确的实例，从而产生紧凑且纯净的伪实例标签。然后，我们提出了一种由粗到精的实例分割器，该分割器首先从整个点云中预测粗略实例，然后再基于这些粗略实例区域学习精细实例。最后，通过使用伪实例标签监督实例分割器，并进行联合训练，可以逐步生成高质量的实例。 大量的实验表明，我们的方法在ScanNetV2和S3DIS基准测试上取得了最先进的性能，甚至超过了几个采用草图边界框的完全监督的方法。代码可在提供的链接中获取。
URL
https://arxiv.org/abs/2505.16399
PDF
-
Style Transfer with Diffusion Models for Synthetic-to-Real Domain Adaptation
2025-05-22 08:11:10Estelle Chigot, Dennis G. Wilson, Meriem Ghrib, Thomas OberlinAbstract
Semantic segmentation models trained on synthetic data often perform poorly on real-world images due to domain gaps, particularly in adverse conditions where labeled data is scarce. Yet, recent foundation models enable to generate realistic images without any training. This paper proposes to leverage such diffusion models to improve the performance of vision models when learned on synthetic data. We introduce two novel techniques for semantically consistent style transfer using diffusion models: Class-wise Adaptive Instance Normalization and Cross-Attention (CACTI) and its extension with selective attention Filtering (CACTIF). CACTI applies statistical normalization selectively based on semantic classes, while CACTIF further filters cross-attention maps based on feature similarity, preventing artifacts in regions with weak cross-attention correspondences. Our methods transfer style characteristics while preserving semantic boundaries and structural coherence, unlike approaches that apply global transformations or generate content without constraints. Experiments using GTA5 as source and Cityscapes/ACDC as target domains show that our approach produces higher quality images with lower FID scores and better content preservation. Our work demonstrates that class-aware diffusion-based style transfer effectively bridges the synthetic-to-real domain gap even with minimal target domain data, advancing robust perception systems for challenging real-world applications. The source code is available at: this https URL.
Abstract (translated)
基于合成数据训练的语义分割模型在真实世界图像上的表现通常较差，特别是在标签数据稀缺的恶劣条件下。然而，最近的基础模型能够在不进行训练的情况下生成逼真的图像。本文提出利用这些扩散模型（diffusion models）来改进仅通过合成数据学习的视觉模型的表现。 我们介绍了两种新的用于语义一致风格迁移的技术：基于类别的自适应实例归一化与交叉注意力（CACTI，Class-wise Adaptive Instance Normalization and Cross-Attention），以及具有选择性注意过滤功能的其扩展版本（CACTIF）。CACTI技术根据语义类别进行统计标准化处理，而CACTIF进一步根据特征相似度对跨注意力图进行过滤，从而避免在对应关系较弱区域出现伪影。我们的方法可以转移风格特性并保持语义边界和结构一致性，与应用全局变换或无约束内容生成的方法不同。 使用GTA5作为源域，Cityscapes/ACDC作为目标域的实验表明，我们提出的方法能够产生质量更高、FID得分更低且内容保存更好的图像。我们的研究证明了类别感知扩散基风格转换技术可以有效缩小合成数据与真实世界之间的差距，并在目标领域数据量极少的情况下推进鲁棒性感知系统的发展，以应对具有挑战性的现实应用。 源代码可在以下链接获取：[此链接]（请将实际的URL地址插入此处）。
URL
https://arxiv.org/abs/2505.16360
PDF
-
SAMba-UNet: Synergizing SAM2 and Mamba in UNet with Heterogeneous Aggregation for Cardiac MRI Segmentation
2025-05-22 06:57:03Guohao Huo, Ruiting Dai, Hao TangAbstract
To address the challenge of complex pathological feature extraction in automated cardiac MRI segmentation, this study proposes an innovative dual-encoder architecture named SAMba-UNet. The framework achieves cross-modal feature collaborative learning by integrating the vision foundation model SAM2, the state-space model Mamba, and the classical UNet. To mitigate domain discrepancies between medical and natural images, a Dynamic Feature Fusion Refiner is designed, which enhances small lesion feature extraction through multi-scale pooling and a dual-path calibration mechanism across channel and spatial dimensions. Furthermore, a Heterogeneous Omni-Attention Convergence Module (HOACM) is introduced, combining global contextual attention with branch-selective emphasis mechanisms to effectively fuse SAM2's local positional semantics and Mamba's long-range dependency modeling capabilities. Experiments on the ACDC cardiac MRI dataset demonstrate that the proposed model achieves a Dice coefficient of 0.9103 and an HD95 boundary error of 1.0859 mm, significantly outperforming existing methods, particularly in boundary localization for complex pathological structures such as right ventricular anomalies. This work provides an efficient and reliable solution for automated cardiac disease diagnosis, and the code will be open-sourced.
Abstract (translated)
为了应对自动化心脏MRI分割中复杂的病理特征提取挑战，本研究提出了一种创新的双编码器架构，名为SAMba-UNet。该框架通过整合视觉基础模型SAM2、状态空间模型Mamba和经典的UNet实现了跨模态特征协作学习。为缓解医学图像与自然图像之间的领域差异问题，设计了一个动态特征融合精炼器，该精炼器通过多尺度池化以及通道和空间维度上的双路径校准机制增强了对小病灶的特征提取能力。 此外，还引入了异构全注意力收敛模块（HOACM），结合全局上下文注意力与分支选择性强调机制，有效融合SAM2的局部位置语义和Mamba的长程依赖建模能力。在ACDC心脏MRI数据集上的实验表明，所提出的模型实现了0.9103的Dice系数和1.0859毫米的HD95边界误差，在复杂病理结构（如右心室异常）的边界定位上显著优于现有方法。 这项工作为自动化心脏病诊断提供了一种高效可靠的解决方案，并将开放源代码。
URL
https://arxiv.org/abs/2505.16304
PDF
-
Efficient Prototype Consistency Learning in Medical Image Segmentation via Joint Uncertainty and Data Augmentation
2025-05-22 06:25:32Lijian Li, Yuanpeng He, Chi-Man PunAbstract
Recently, prototype learning has emerged in semi-supervised medical image segmentation and achieved remarkable performance. However, the scarcity of labeled data limits the expressiveness of prototypes in previous methods, potentially hindering the complete representation of prototypes for class embedding. To overcome this issue, we propose an efficient prototype consistency learning via joint uncertainty quantification and data augmentation (EPCL-JUDA) to enhance the semantic expression of prototypes based on the framework of Mean-Teacher. The concatenation of original and augmented labeled data is fed into student network to generate expressive prototypes. Then, a joint uncertainty quantification method is devised to optimize pseudo-labels and generate reliable prototypes for original and augmented unlabeled data separately. High-quality global prototypes for each class are formed by fusing labeled and unlabeled prototypes, which are utilized to generate prototype-to-features to conduct consistency learning. Notably, a prototype network is proposed to reduce high memory requirements brought by the introduction of augmented data. Extensive experiments on Left Atrium, Pancreas-NIH, Type B Aortic Dissection datasets demonstrate EPCL-JUDA's superiority over previous state-of-the-art approaches, confirming the effectiveness of our framework. The code will be released soon.
Abstract (translated)
最近，在半监督医学图像分割领域，原型学习作为一种新兴方法取得了显著的性能。然而，标注数据的稀缺性限制了先前方法中原型的表现力，可能阻碍了原型对类嵌入的完全表示。为了解决这个问题，我们提出了一种通过联合不确定性量化和数据增强（EPCL-JUDA）进行高效的原型一致性学习的方法，以提高基于Mean-Teacher框架下原型的语义表达能力。原始标注数据与增强后的标注数据串联后输入学生网络生成具有表现力的原型。随后，设计了一个联合不确定性量化方法来优化伪标签，并为原始和增强的未标注数据分别生成可靠的原型。通过融合标注数据和未标注数据的原型，可以形成每个类别的高质量全局原型，这些全局原型被用来生成从原型到特征的映射，从而进行一致性学习。值得注意的是，我们提出了一种原型网络来减少引入增强数据后带来的高内存需求。 在左心房、胰腺-NIH以及B型主动脉夹层等数据集上的大量实验表明，EPCL-JUDA方法优于先前最先进的方法，证实了我们的框架的有效性。代码即将发布。
URL
https://arxiv.org/abs/2505.16283
PDF
