Weakly_Supervised
-
DBGroup: Dual-Branch Point Grouping for Weakly Supervised 3D Instance Segmentation
2025-11-13 06:12:13Xuexun Liu, Xiaoxu Xu, Qiudan Zhang, Lin Ma, Xu WangAbstract
Weakly supervised 3D instance segmentation is essential for 3D scene understanding, especially as the growing scale of data and high annotation costs associated with fully supervised approaches. Existing methods primarily rely on two forms of weak supervision: one-thing-one-click annotations and bounding box annotations, both of which aim to reduce labeling efforts. However, these approaches still encounter limitations, including labor-intensive annotation processes, high complexity, and reliance on expert annotators. To address these challenges, we propose \textbf{DBGroup}, a two-stage weakly supervised 3D instance segmentation framework that leverages scene-level annotations as a more efficient and scalable alternative. In the first stage, we introduce a Dual-Branch Point Grouping module to generate pseudo labels guided by semantic and mask cues extracted from multi-view images. To further improve label quality, we develop two refinement strategies: Granularity-Aware Instance Merging and Semantic Selection and Propagation. The second stage involves multi-round self-training on an end-to-end instance segmentation network using the refined pseudo-labels. Additionally, we introduce an Instance Mask Filter strategy to address inconsistencies within the pseudo labels. Extensive experiments demonstrate that DBGroup achieves competitive performance compared to sparse-point-level supervised 3D instance segmentation methods, while surpassing state-of-the-art scene-level supervised 3D semantic segmentation approaches. Code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2511.10003
PDF
-
TYrPPG: Uncomplicated and Enhanced Learning Capability rPPG for Remote Heart Rate Estimation
2025-11-08 03:46:58Taixi Chen, Yiu-ming CheungAbstract
Remote photoplethysmography (rPPG) can remotely extract physiological signals from RGB video, which has many advantages in detecting heart rate, such as low cost and no invasion to patients. The existing rPPG model is usually based on the transformer module, which has low computation efficiency. Recently, the Mamba model has garnered increasing attention due to its efficient performance in natural language processing tasks, demonstrating potential as a substitute for transformer-based algorithms. However, the Mambaout model and its variants prove that the SSM module, which is the core component of the Mamba model, is unnecessary for the vision task. Therefore, we hope to prove the feasibility of using the Mambaout-based module to remotely learn the heart rate. Specifically, we propose a novel rPPG algorithm called uncomplicated and enhanced learning capability rPPG (TYrPPG). This paper introduces an innovative gated video understanding block (GVB) designed for efficient analysis of RGB videos. Based on the Mambaout structure, this block integrates 2D-CNN and 3D-CNN to enhance video understanding for analysis. In addition, we propose a comprehensive supervised loss function (CSL) to improve the model's learning capability, along with its weakly supervised variants. The experiments show that our TYrPPG can achieve state-of-the-art performance in commonly used datasets, indicating its prospects and superiority in remote heart rate estimation. The source code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2511.05833
PDF
-
LG-NuSegHop: A Local-to-Global Self-Supervised Pipeline For Nuclei Instance Segmentation
2025-11-07 00:34:10Vasileios Magoulianitis, Catherine A. Alexander, Jiaxin Yang, C. -C. Jay KuoAbstract
Nuclei segmentation is the cornerstone task in histology image reading, shedding light on the underlying molecular patterns and leading to disease or cancer diagnosis. Yet, it is a laborious task that requires expertise from trained physicians. The large nuclei variability across different organ tissues and acquisition processes challenges the automation of this task. On the other hand, data annotations are expensive to obtain, and thus, Deep Learning (DL) models are challenged to generalize to unseen organs or different domains. This work proposes Local-to-Global NuSegHop (LG-NuSegHop), a self-supervised pipeline developed on prior knowledge of the problem and molecular biology. There are three distinct modules: (1) a set of local processing operations to generate a pseudolabel, (2) NuSegHop a novel data-driven feature extraction model and (3) a set of global operations to post-process the predictions of NuSegHop. Notably, even though the proposed pipeline uses { no manually annotated training data} or domain adaptation, it maintains a good generalization performance on other datasets. Experiments in three publicly available datasets show that our method outperforms other self-supervised and weakly supervised methods while having a competitive standing among fully supervised methods. Remarkably, every module within LG-NuSegHop is transparent and explainable to physicians.
Abstract (translated)
URL
https://arxiv.org/abs/2511.04892
PDF
-
WST: Weakly Supervised Transducer for Automatic Speech Recognition
2025-11-06 04:14:07Dongji Gao, Chenda Liao, Changliang Liu, Matthew Wiesner, Leibny Paola Garcia, Daniel Povey, Sanjeev Khudanpur, Jian WuAbstract
The Recurrent Neural Network-Transducer (RNN-T) is widely adopted in end-to-end (E2E) automatic speech recognition (ASR) tasks but depends heavily on large-scale, high-quality annotated data, which are often costly and difficult to obtain. To mitigate this reliance, we propose a Weakly Supervised Transducer (WST), which integrates a flexible training graph designed to robustly handle errors in the transcripts without requiring additional confidence estimation or auxiliary pre-trained models. Empirical evaluations on synthetic and industrial datasets reveal that WST effectively maintains performance even with transcription error rates of up to 70%, consistently outperforming existing Connectionist Temporal Classification (CTC)-based weakly supervised approaches, such as Bypass Temporal Classification (BTC) and Omni-Temporal Classification (OTC). These results demonstrate the practical utility and robustness of WST in realistic ASR settings. The implementation will be publicly available.
Abstract (translated)
URL
https://arxiv.org/abs/2511.04035
PDF
-
Open Source State-Of-the-Art Solution for Romanian Speech Recognition
2025-11-05 11:02:16Gabriel Pirlogeanu, Alexandru-Lucian Georgescu, Horia CucuAbstract
In this work, we present a new state-of-the-art Romanian Automatic Speech Recognition (ASR) system based on NVIDIA's FastConformer architecture--explored here for the first time in the context of Romanian. We train our model on a large corpus of, mostly, weakly supervised transcriptions, totaling over 2,600 hours of speech. Leveraging a hybrid decoder with both Connectionist Temporal Classification (CTC) and Token-Duration Transducer (TDT) branches, we evaluate a range of decoding strategies including greedy, ALSD, and CTC beam search with a 6-gram token-level language model. Our system achieves state-of-the-art performance across all Romanian evaluation benchmarks, including read, spontaneous, and domain-specific speech, with up to 27% relative WER reduction compared to previous best-performing systems. In addition to improved transcription accuracy, our approach demonstrates practical decoding efficiency, making it suitable for both research and deployment in low-latency ASR applications.
Abstract (translated)
URL
https://arxiv.org/abs/2511.03361
PDF
-
Resource-efficient Automatic Refinement of Segmentations via Weak Supervision from Light Feedback
2025-11-04 13:53:10Alix de Langlais, Benjamin Billot, Th\'eo Aguilar Vidal, Marc-Olivier Gauci, Herv\'e DelingetteAbstract
Delineating anatomical regions is a key task in medical image analysis. Manual segmentation achieves high accuracy but is labor-intensive and prone to variability, thus prompting the development of automated approaches. Recently, a breadth of foundation models has enabled automated segmentations across diverse anatomies and imaging modalities, but these may not always meet the clinical accuracy standards. While segmentation refinement strategies can improve performance, current methods depend on heavy user interactions or require fully supervised segmentations for training. Here, we present SCORE (Segmentation COrrection from Regional Evaluations), a weakly supervised framework that learns to refine mask predictions only using light feedback during training. Specifically, instead of relying on dense training image annotations, SCORE introduces a novel loss that leverages region-wise quality scores and over/under-segmentation error labels. We demonstrate SCORE on humerus CT scans, where it considerably improves initial predictions from TotalSegmentator, and achieves performance on par with existing refinement methods, while greatly reducing their supervision requirements and annotation time. Our code is available at: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2511.02576
PDF
-
Weakly Supervised Concept Learning with Class-Level Priors for Interpretable Medical Diagnosis
2025-11-03 00:43:04Md Nahiduzzaman, Steven Korevaar, Alireza Bab-Hadiashar, Ruwan TennakoonAbstract
Human-interpretable predictions are essential for deploying AI in medical imaging, yet most interpretable-by-design (IBD) frameworks require concept annotations for training data, which are costly and impractical to obtain in clinical contexts. Recent attempts to bypass annotation, such as zero-shot vision-language models or concept-generation frameworks, struggle to capture domain-specific medical features, leading to poor reliability. In this paper, we propose a novel Prior-guided Concept Predictor (PCP), a weakly supervised framework that enables concept answer prediction without explicit supervision or reliance on language models. PCP leverages class-level concept priors as weak supervision and incorporates a refinement mechanism with KL divergence and entropy regularization to align predictions with clinical reasoning. Experiments on PH2 (dermoscopy) and WBCatt (hematology) show that PCP improves concept-level F1-score by over 33% compared to zero-shot baselines, while delivering competitive classification performance on four medical datasets (PH2, WBCatt, HAM10000, and CXR4) relative to fully supervised concept bottleneck models (CBMs) and V-IP.
Abstract (translated)
URL
https://arxiv.org/abs/2511.01131
PDF
-
Weakly Supervised Pneumonia Localization from Chest X-Rays Using Deep Neural Network and Grad-CAM Explanations
2025-11-01 08:44:24Kiran Shahi, Anup BagaleAbstract
This study proposes a weakly supervised deep learning framework for pneumonia classification and localization from chest X-rays, utilizing Grad-CAM explanations. Instead of costly pixel-level annotations, our approach utilizes image-level labels to generate clinically meaningful heatmaps that highlight regions affected by pneumonia. We evaluate seven ImageNet-pretrained architectures ResNet-18/50, DenseNet-121, EfficientNet-B0, MobileNet-V2/V3, and ViT-B16 under identical training conditions with focal loss and patient-wise splits to prevent data leakage. Experimental results on the Kermany CXR dataset demonstrate that ResNet-18 and EfficientNet-B0 achieve the best overall test accuracy of 98\%, ROC-AUC = 0.997, and F1 = 0.987, while MobileNet-V2 provides an optimal trade-off between accuracy and computational cost. Grad-CAM visualizations confirm that the proposed models focus on clinically relevant lung regions, supporting the use of interpretable AI for radiological diagnostics. This work highlights the potential of weakly supervised explainable models that enhance pneumonia screening transparency, and clinical trust in AI-assisted medical imaging. this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2511.00456
PDF
-
Knowledge-Guided Textual Reasoning for Explainable Video Anomaly Detection via LLMs
2025-10-30 01:18:55Hari LeeAbstract
We introduce Text-based Explainable Video Anomaly Detection (TbVAD), a language-driven framework for weakly supervised video anomaly detection that performs anomaly detection and explanation entirely within the textual domain. Unlike conventional WSVAD models that rely on explicit visual features, TbVAD represents video semantics through language, enabling interpretable and knowledge-grounded reasoning. The framework operates in three stages: (1) transforming video content into fine-grained captions using a vision-language model, (2) constructing structured knowledge by organizing the captions into four semantic slots (action, object, context, environment), and (3) generating slot-wise explanations that reveal which semantic factors contribute most to the anomaly decision. We evaluate TbVAD on two public benchmarks, UCF-Crime and XD-Violence, demonstrating that textual knowledge reasoning provides interpretable and reliable anomaly detection for real-world surveillance scenarios.
Abstract (translated)
URL
https://arxiv.org/abs/2511.07429
PDF
-
Seeing, Signing, and Saying: A Vision-Language Model-Assisted Pipeline for Sign Language Data Acquisition and Curation from Social Media
2025-10-29 11:29:56Shakib Yazdani, Yasser Hamidullah, Cristina Espa\~na-Bonet, Josef van GenabithAbstract
Most existing sign language translation (SLT) datasets are limited in scale, lack multilingual coverage, and are costly to curate due to their reliance on expert annotation and controlled recording setup. Recently, Vision Language Models (VLMs) have demonstrated strong capabilities as evaluators and real-time assistants. Despite these advancements, their potential remains untapped in the context of sign language dataset acquisition. To bridge this gap, we introduce the first automated annotation and filtering framework that utilizes VLMs to reduce reliance on manual effort while preserving data quality. Our method is applied to TikTok videos across eight sign languages and to the already curated YouTube-SL-25 dataset in German Sign Language for the purpose of additional evaluation. Our VLM-based pipeline includes a face visibility detection, a sign activity recognition, a text extraction from video content, and a judgment step to validate alignment between video and text, implementing generic filtering, annotation and validation steps. Using the resulting corpus, TikTok-SL-8, we assess the performance of two off-the-shelf SLT models on our filtered dataset for German and American Sign Languages, with the goal of establishing baselines and evaluating the robustness of recent models on automatically extracted, slightly noisy data. Our work enables scalable, weakly supervised pretraining for SLT and facilitates data acquisition from social media.
Abstract (translated)
URL
https://arxiv.org/abs/2510.25413
PDF
-
Region-CAM: Towards Accurate Object Regions in Class Activation Maps for Weakly Supervised Learning Tasks
2025-10-29 03:28:18Qingdong Cai, Charith AbhayaratneAbstract
Class Activation Mapping (CAM) methods are widely applied in weakly supervised learning tasks due to their ability to highlight object regions. However, conventional CAM methods highlight only the most discriminative regions of the target. These highlighted regions often fail to cover the entire object and are frequently misaligned with object boundaries, thereby limiting the performance of downstream weakly supervised learning tasks, particularly Weakly Supervised Semantic Segmentation (WSSS), which demands pixel-wise accurate activation maps to get the best results. To alleviate the above problems, we propose a novel activation method, Region-CAM. Distinct from network feature weighting approaches, Region-CAM generates activation maps by extracting semantic information maps (SIMs) and performing semantic information propagation (SIP) by considering both gradients and features in each of the stages of the baseline classification model. Our approach highlights a greater proportion of object regions while ensuring activation maps to have precise boundaries that align closely with object edges. Region-CAM achieves 60.12% and 58.43% mean intersection over union (mIoU) using the baseline model on the PASCAL VOC training and validation datasets, respectively, which are improvements of 13.61% and 13.13% over the original CAM (46.51% and 45.30%). On the MS COCO validation set, Region-CAM achieves 36.38%, a 16.23% improvement over the original CAM (20.15%). We also demonstrate the superiority of Region-CAM in object localization tasks, using the ILSVRC2012 validation set. Region-CAM achieves 51.7% in Top-1 Localization accuracy Loc1. Compared with LayerCAM, an activation method designed for weakly supervised object localization, Region-CAM achieves 4.5% better performance in Loc1.
Abstract (translated)
URL
https://arxiv.org/abs/2510.25134
PDF
-
A Benchmark for Open-Domain Numerical Fact-Checking Enhanced by Claim Decomposition
Abstract
Fact-checking numerical claims is critical as the presence of numbers provide mirage of veracity despite being fake potentially causing catastrophic impacts on society. The prior works in automatic fact verification do not primarily focus on natural numerical claims. A typical human fact-checker first retrieves relevant evidence addressing the different numerical aspects of the claim and then reasons about them to predict the veracity of the claim. Hence, the search process of a human fact-checker is a crucial skill that forms the foundation of the verification process. Emulating a real-world setting is essential to aid in the development of automated methods that encompass such skills. However, existing benchmarks employ heuristic claim decomposition approaches augmented with weakly supervised web search to collect evidences for verifying claims. This sometimes results in less relevant evidences and noisy sources with temporal leakage rendering a less realistic retrieval setting for claim verification. Hence, we introduce QuanTemp++: a dataset consisting of natural numerical claims, an open domain corpus, with the corresponding relevant evidence for each claim. The evidences are collected through a claim decomposition process approximately emulating the approach of human fact-checker and veracity labels ensuring there is no temporal leakage. Given this dataset, we also characterize the retrieval performance of key claim decomposition paradigms. Finally, we observe their effect on the outcome of the verification pipeline and draw insights. The code for data pipeline along with link to data can be found at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2510.22055
PDF
-
MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection
2025-10-24 13:28:29Shengtian Yang, Yue Feng, Yingshi Liu, Jingrou Zhang, Jie QinAbstract
Video Anomaly Detection (VAD) aims to locate unusual activities or behaviors within videos. Recently, offline VAD has garnered substantial research attention, which has been invigorated by the progress in large language models (LLMs) and vision-language models (VLMs), offering the potential for a more nuanced understanding of anomalies. However, online VAD has seldom received attention due to real-time constraints and computational intensity. In this paper, we introduce a novel Memory-based online scoring queue scheme for Training-free VAD (MoniTor), to address the inherent complexities in online VAD. Specifically, MoniTor applies a streaming input to VLMs, leveraging the capabilities of pre-trained large-scale models. To capture temporal dependencies more effectively, we incorporate a novel prediction mechanism inspired by Long Short-Term Memory (LSTM) networks. This ensures the model can effectively model past states and leverage previous predictions to identify anomalous behaviors. Thereby, it better understands the current frame. Moreover, we design a scoring queue and an anomaly prior to dynamically store recent scores and cover all anomalies in the monitoring scenario, providing guidance for LLMs to distinguish between normal and abnormal behaviors over time. We evaluate MoniTor on two large datasets (i.e., UCF-Crime and XD-Violence) containing various surveillance and real-world scenarios. The results demonstrate that MoniTor outperforms state-of-the-art methods and is competitive with weakly supervised methods without training. Code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2510.21449
PDF
-
Learning from N-Tuple Data with M Positive Instances: Unbiased Risk Estimation and Theoretical Guarantees
2025-10-21 08:28:07Miao Zhang, Junpeng Li, ChangChun HUa, Yana YangAbstract
Weakly supervised learning often operates with coarse aggregate signals rather than instance labels. We study a setting where each training example is an $n$-tuple containing exactly m positives, while only the count m per tuple is observed. This NTMP (N-tuple with M positives) supervision arises in, e.g., image classification with region proposals and multi-instance measurements. We show that tuple counts admit a trainable unbiased risk estimator (URE) by linking the tuple-generation process to latent instance marginals. Starting from fixed (n,m), we derive a closed-form URE and extend it to variable tuple sizes, variable counts, and their combination. Identification holds whenever the effective mixing rate is separated from the class prior. We establish generalization bounds via Rademacher complexity and prove statistical consistency with standard rates under mild regularity assumptions. To improve finite-sample stability, we introduce simple ReLU corrections to the URE that preserve asymptotic correctness. Across benchmarks converted to NTMP tasks, the approach consistently outperforms representative weak-supervision baselines and yields favorable precision-recall and F1 trade-offs. It remains robust under class-prior imbalance and across diverse tuple configurations, demonstrating that count-only supervision can be exploited effectively through a theoretically grounded and practically stable objective.
Abstract (translated)
弱监督学习通常处理粗略的聚合信号，而不是实例标签。我们研究了一种设置，在这种设置中，每个训练样本是一个包含恰好m个正例的n元组，而我们只观察到每组中的计数m。这种NTMP（N-tuple with M positives）监督出现在例如带有区域提案和多实例测量的图像分类场景中。我们展示了元组计数可以提供一个可训练的无偏风险估计器（URE），通过将元组生成过程与潜在实例边缘联系起来。从固定的(n, m)开始，我们推导出一种封闭形式的URE，并将其扩展到变量大小、变量数量以及它们的组合。当有效的混合速率与类先验分离时，即可进行识别。我们通过Rademacher复杂度建立泛化界限，并在适度正则性假设下证明统计一致性及标准速度。为了提高有限样本稳定性，我们在URE中引入了简单的ReLU修正，以保持渐近正确性。在转换为NTMP任务的基准测试上，该方法始终优于代表性弱监督基线，在精度-召回率和F1指标权衡方面表现良好，并且即使在类先验不平衡的情况下以及面对各种元组配置时也能保持稳健性。这表明仅通过计数监督就可以有效地利用一个理论基础坚实、实践稳定性好的目标函数。
URL
https://arxiv.org/abs/2510.18406
PDF
-
Split-Fuse-Transport: Annotation-Free Saliency via Dual Clustering and Optimal Transport Alignment
2025-10-20 12:27:55Muhammad Umer Ramzan, Ali Zia, Abdelwahed Khamis, Noman Ali, Usman Ali, Wei XiangAbstract
Salient object detection (SOD) aims to segment visually prominent regions in images and serves as a foundational task for various computer vision applications. We posit that SOD can now reach near-supervised accuracy without a single pixel-level label, but only when reliable pseudo-masks are available. We revisit the prototype-based line of work and make two key observations. First, boundary pixels and interior pixels obey markedly different geometry; second, the global consistency enforced by optimal transport (OT) is underutilized if prototype quality is weak. To address this, we introduce POTNet, an adaptation of Prototypical Optimal Transport that replaces POT's single k-means step with an entropy-guided dual-clustering head: high-entropy pixels are organized by spectral clustering, low-entropy pixels by k-means, and the two prototype sets are subsequently aligned by OT. This split-fuse-transport design yields sharper, part-aware pseudo-masks in a single forward pass, without handcrafted priors. Those masks supervise a standard MaskFormer-style encoder-decoder, giving rise to AutoSOD, an end-to-end unsupervised SOD pipeline that eliminates SelfMask's offline voting yet improves both accuracy and training efficiency. Extensive experiments on five benchmarks show that AutoSOD outperforms unsupervised methods by up to 26% and weakly supervised methods by up to 36% in F-measure, further narrowing the gap to fully supervised models.
Abstract (translated)
显著对象检测（SOD）旨在分割图像中视觉上突出的区域，并作为各种计算机视觉应用的基础任务。我们提出，现在可以通过利用可靠的伪掩码而不需要单个像素级别的标签来实现接近监督精度的SOD。我们重新审视基于原型的工作，并提出了两个关键观察结果：第一，边界像素和内部像素遵循截然不同的几何特性；第二，在原型质量较弱的情况下，最优传输（OT）所强制执行的整体一致性被低估了。 为了应对这一挑战，我们引入了POTNet，这是对基于原型的最优传输方法的一个改进版本。在新的设计中，我们将POT中的单一k-means步骤替换为由熵引导的双聚类头：高熵像素通过谱聚类组织，低熵像素则通过k-means进行处理，然后使用OT将两个原型集对齐。这种分而治之的设计在一个前向传递过程中产生了更加清晰、具备局部感知能力的伪掩码，并且没有采用手工设计的先验知识。 这些掩码监督了一个标准的MaskFormer风格的编码器-解码器模型，从而诞生了AutoSOD——一种端到端无监督SOD流水线。与SelfMask的离线投票相比，它不仅提高了精度和训练效率，而且还消除了这种依赖性。广泛的实验证明，在五个基准测试上，AutoSOD在F-measure指标上分别比无监督方法高出26%，比弱监督方法高出36%，从而进一步缩小了与完全监督模型之间的差距。
URL
https://arxiv.org/abs/2510.17484
PDF
-
Few-Label Multimodal Modeling of SNP Variants and ECG Phenotypes Using Large Language Models for Cardiovascular Risk Stratification
2025-10-18 15:19:35Niranjana Arun Menon, Yulong Li, Iqra Farooq, Sara Ahmed, Muhammad Awais, Imran RazzakAbstract
Cardiovascular disease (CVD) risk stratification remains a major challenge due to its multifactorial nature and limited availability of high-quality labeled datasets. While genomic and electrophysiological data such as SNP variants and ECG phenotypes are increasingly accessible, effectively integrating these modalities in low-label settings is non-trivial. This challenge arises from the scarcity of well-annotated multimodal datasets and the high dimensionality of biological signals, which limit the effectiveness of conventional supervised models. To address this, we present a few-label multimodal framework that leverages large language models (LLMs) to combine genetic and electrophysiological information for cardiovascular risk stratification. Our approach incorporates a pseudo-label refinement strategy to adaptively distill high-confidence labels from weakly supervised predictions, enabling robust model fine-tuning with only a small set of ground-truth annotations. To enhance the interpretability, we frame the task as a Chain of Thought (CoT) reasoning problem, prompting the model to produce clinically relevant rationales alongside predictions. Experimental results demonstrate that the integration of multimodal inputs, few-label supervision, and CoT reasoning improves robustness and generalizability across diverse patient profiles. Experimental results using multimodal SNP variants and ECG-derived features demonstrated comparable performance to models trained on the full dataset, underscoring the promise of LLM-based few-label multimodal modeling for advancing personalized cardiovascular care.
Abstract (translated)
心血管疾病（CVD）的风险分层仍然是一个重大挑战，因为其多因素性质以及高质量标注数据的缺乏。尽管基因组和电生理学数据如单核苷酸多态性（SNP）变异和心电图表型越来越容易获得，但在标签较少的情况下有效地整合这些模态却并不简单。这一问题源于多模式注释数据稀缺及生物信号高维度的特点，这限制了传统监督模型的有效性。 为了解决这个问题，我们提出了一种少标签的多模态框架，该框架利用大型语言模型（LLM）结合基因和电生理信息进行心血管风险分层。我们的方法包括伪标签精炼策略，可以从弱监督预测中自适应地提炼出高置信度标签，从而使仅用少量真实标注的情况下也能实现稳健的模型微调。 为了提高可解释性，我们将任务构架为链式思维（CoT）推理问题，促使模型在给出预测的同时提供临床相关理由。实验结果表明，多模态输入、少标签监督和CoT推理可以增强模型在不同患者群体中的鲁棒性和泛化能力。使用多模态SNP变异和心电图派生特征进行的实验结果显示，该方法的表现与基于完整数据集训练的模型相当，这彰显了LLM为基础的少标签多模式建模技术在推进个性化心血管护理方面所展现的巨大潜力。
URL
https://arxiv.org/abs/2510.16536
PDF
-
Instance-Aware Pseudo-Labeling and Class-Focused Contrastive Learning for Weakly Supervised Domain Adaptive Segmentation of Electron Microscopy
2025-10-18 11:05:37Shan Xiong, Jiabao Chen, Ye Wang, Jialin PengAbstract
Annotation-efficient segmentation of the numerous mitochondria instances from various electron microscopy (EM) images is highly valuable for biological and neuroscience research. Although unsupervised domain adaptation (UDA) methods can help mitigate domain shifts and reduce the high costs of annotating each domain, they typically have relatively low performance in practical applications. Thus, we investigate weakly supervised domain adaptation (WDA) that utilizes additional sparse point labels on the target domain, which require minimal annotation effort and minimal expert knowledge. To take full use of the incomplete and imprecise point annotations, we introduce a multitask learning framework that jointly conducts segmentation and center detection with a novel cross-teaching mechanism and class-focused cross-domain contrastive learning. While leveraging unlabeled image regions is essential, we introduce segmentation self-training with a novel instance-aware pseudo-label (IPL) selection strategy. Unlike existing methods that typically rely on pixel-wise pseudo-label filtering, the IPL semantically selects reliable and diverse pseudo-labels with the help of the detection task. Comprehensive validations and comparisons on challenging datasets demonstrate that our method outperforms existing UDA and WDA methods, significantly narrowing the performance gap with the supervised upper bound. Furthermore, under the UDA setting, our method also achieves substantial improvements over other UDA techniques.
Abstract (translated)
从各种电子显微镜（EM）图像中高效注释众多线粒体实例对于生物和神经科学研究非常有价值。虽然无监督领域适应（UDA）方法可以帮助缓解域偏移并减少对每个域进行详细标注的高昂成本，但它们在实际应用中的表现通常相对较低。因此，我们研究了弱监督领域适应（WDA），该方法利用目标域上的稀疏点标签，这些标签只需要最小的努力和专业知识即可完成注释。为了充分利用不完整且不够精确的点注释，我们引入了一个多任务学习框架，该框架同时进行分割和中心检测，并结合了一种新颖的跨教学机制以及以类别为中心的跨领域对比学习方法。 在利用未标注图像区域方面至关重要，我们引入了基于实例感知伪标签（IPL）选择策略的分割自训练。不同于现有方法通常依赖像素级伪标签过滤，IPL通过检测任务的帮助进行语义上可靠和多样化的伪标签选取。在具有挑战性的数据集上的全面验证和比较表明，我们的方法优于现有的UDA和WDA方法，并显著缩小了与监督上限性能差距。此外，在UDA设置下，我们的方法也比其他UDA技术实现了重大改进。
URL
https://arxiv.org/abs/2510.16450
PDF
-
Uncertainty-Aware Extreme Point Tracing for Weakly Supervised Ultrasound Image Segmentation
2025-10-17 13:53:16Lei Shi, Gang Li, Junxing ZhangAbstract
Automatic medical image segmentation is a fundamental step in computer-aided diagnosis, yet fully supervised approaches demand extensive pixel-level annotations that are costly and time-consuming. To alleviate this burden, we propose a weakly supervised segmentation framework that leverages only four extreme points as annotation. Specifically, bounding boxes derived from the extreme points are used as prompts for the Segment Anything Model 2 (SAM2) to generate reliable initial pseudo labels. These pseudo labels are progressively refined by an enhanced Feature-Guided Extreme Point Masking (FGEPM) algorithm, which incorporates Monte Carlo dropout-based uncertainty estimation to construct a unified gradient uncertainty cost map for boundary tracing. Furthermore, a dual-branch Uncertainty-aware Scale Consistency (USC) loss and a box alignment loss are introduced to ensure spatial consistency and precise boundary alignment during training. Extensive experiments on two public ultrasound datasets, BUSI and UNS, demonstrate that our method achieves performance comparable to, and even surpassing fully supervised counterparts while significantly reducing annotation cost. These results validate the effectiveness and practicality of the proposed weakly supervised framework for ultrasound image segmentation.
Abstract (translated)
自动医学图像分割是计算机辅助诊断中的一个基本步骤，但完全监督的方法需要大量的像素级注释，这些注释成本高昂且耗时。为了减轻这一负担，我们提出了一种弱监督分割框架，该框架仅使用四个极端点作为标注。具体而言，从这四个极端点衍生出的边界框被用作提示，以引导Segment Anything Model 2 (SAM2)生成可靠的初始伪标签。随后，通过增强型特征指导极点掩膜（Feature-Guided Extreme Point Masking, FGEPM）算法对这些伪标签进行逐步优化，并结合基于蒙特卡洛丢弃法的不确定性估计来构建统一的梯度不确定性成本图以追踪边界。 此外，我们还引入了一种双分支不确定性感知尺度一致性损失 (Uncertainty-aware Scale Consistency, USC) 以及一个框对齐损失，确保在训练过程中空间一致性和精确边界对齐。在两个公开的超声数据集（BUSI和UNS）上的广泛实验表明，我们的方法在性能上与完全监督的方法相当甚至超越它们的同时显著降低了注释成本。这些结果验证了我们提出的弱监督框架对于超声图像分割的有效性和实用性。
URL
https://arxiv.org/abs/2510.15666
PDF
-
3D Weakly Supervised Semantic Segmentation via Class-Aware and Geometry-Guided Pseudo-Label Refinement
2025-10-17 03:53:43Xiaoxu Xu, Xuexun Liu, Jinlong Li, Yitian Yuan, Qiudan Zhang, Lin Ma, Nicu Sebe, Xu WangAbstract
3D weakly supervised semantic segmentation (3D WSSS) aims to achieve semantic segmentation by leveraging sparse or low-cost annotated data, significantly reducing reliance on dense point-wise annotations. Previous works mainly employ class activation maps or pre-trained vision-language models to address this challenge. However, the low quality of pseudo-labels and the insufficient exploitation of 3D geometric priors jointly create significant technical bottlenecks in developing high-performance 3D WSSS models. In this paper, we propose a simple yet effective 3D weakly supervised semantic segmentation method that integrates 3D geometric priors into a class-aware guidance mechanism to generate high-fidelity pseudo labels. Concretely, our designed methodology first employs Class-Aware Label Refinement module to generate more balanced and accurate pseudo labels for semantic categrories. This initial refinement stage focuses on enhancing label quality through category-specific optimization. Subsequently, the Geometry-Aware Label Refinement component is developed, which strategically integrates implicit 3D geometric constraints to effectively filter out low-confidence pseudo labels that fail to comply with geometric plausibility. Moreover, to address the challenge of extensive unlabeled regions, we propose a Label Update strategy that integrates Self-Training to propagate labels into these areas. This iterative process continuously enhances pseudo-label quality while expanding label coverage, ultimately fostering the development of high-performance 3D WSSS models. Comprehensive experimental validation reveals that our proposed methodology achieves state-of-the-art performance on both ScanNet and S3DIS benchmarks while demonstrating remarkable generalization capability in unsupervised settings, maintaining competitive accuracy through its robust design.
Abstract (translated)
3D弱监督语义分割（3D WSSS）的目标是通过利用稀疏或低成本的标注数据来实现语义分割，从而大大减少了对密集点注释的依赖。此前的工作主要采用类别激活图或预训练的视觉-语言模型来应对这一挑战。然而，伪标签的质量低和三维几何先验的利用不足共同构成了开发高性能3D WSSS模型的技术瓶颈。在本文中，我们提出了一种简单而有效的3D弱监督语义分割方法，该方法将3D几何先验整合到类感知引导机制中以生成高质量的伪标签。 具体而言，我们的设计方法首先采用类感知标签精炼模块来为语义类别生成更平衡且准确的伪标签。这一初步细化阶段侧重于通过特定类别的优化提升标签质量。随后开发了基于几何感知的标签精炼组件，该组件战略性地整合隐含的3D几何约束以有效过滤掉不符合几何合理性的低置信度伪标签。 此外，为了解决大量未标注区域的挑战，我们提出了一种结合自训练来将标签传播到这些区域的标签更新策略。这一迭代过程持续提升伪标签质量并扩大标签覆盖范围，最终促进高性能3D WSSS模型的发展。 全面的实验验证显示，我们的方法在ScanNet和S3DIS基准测试中均达到了最先进的性能，并展示了其在无监督环境中的卓越泛化能力，通过稳健的设计保持了竞争力的准确度。
URL
https://arxiv.org/abs/2510.17875
PDF
-
Towards Generalist Intelligence in Dentistry: Vision Foundation Models for Oral and Maxillofacial Radiology
2025-10-16 10:24:23Xinrui Huang, Fan Xiao, Dongming He, Anqi Gao, Dandan Li, Xiaofan Zhang, Shaoting Zhang, Xudong WangAbstract
Oral and maxillofacial radiology plays a vital role in dental healthcare, but radiographic image interpretation is limited by a shortage of trained professionals. While AI approaches have shown promise, existing dental AI systems are restricted by their single-modality focus, task-specific design, and reliance on costly labeled data, hindering their generalization across diverse clinical scenarios. To address these challenges, we introduce DentVFM, the first family of vision foundation models (VFMs) designed for dentistry. DentVFM generates task-agnostic visual representations for a wide range of dental applications and uses self-supervised learning on DentVista, a large curated dental imaging dataset with approximately 1.6 million multi-modal radiographic images from various medical centers. DentVFM includes 2D and 3D variants based on the Vision Transformer (ViT) architecture. To address gaps in dental intelligence assessment and benchmarks, we introduce DentBench, a comprehensive benchmark covering eight dental subspecialties, more diseases, imaging modalities, and a wide geographical distribution. DentVFM shows impressive generalist intelligence, demonstrating robust generalization to diverse dental tasks, such as disease diagnosis, treatment analysis, biomarker identification, and anatomical landmark detection and segmentation. Experimental results indicate DentVFM significantly outperforms supervised, self-supervised, and weakly supervised baselines, offering superior generalization, label efficiency, and scalability. Additionally, DentVFM enables cross-modality diagnostics, providing more reliable results than experienced dentists in situations where conventional imaging is unavailable. DentVFM sets a new paradigm for dental AI, offering a scalable, adaptable, and label-efficient model to improve intelligent dental healthcare and address critical gaps in global oral healthcare.
Abstract (translated)
口腔和颌面放射学在牙科医疗保健中扮演着至关重要的角色，但放射图像的解读受限于训练有素的专业人员短缺。虽然人工智能方法显示出巨大潜力，但现有的牙科AI系统因其单一模态聚焦、任务特异性设计以及对昂贵标记数据的依赖性，在不同临床场景中的泛化能力受到限制。为解决这些挑战，我们推出了DentVFM，这是首个专为牙科设计的视觉基础模型（VFMs）家族。DentVFM生成适用于广泛牙科应用的任务无关视觉表示，并通过在DentVista上进行自我监督学习来实现这一点——这是一个包含来自各种医疗机构的大约160万张多模态放射图像的大型策划数据库。 DentVFM包括基于视觉变换器（ViT）架构的2D和3D变体。为填补牙科智能评估与基准测试中的空白，我们推出了DentBench，这是一个涵盖八个牙科亚专科、更多疾病类型、成像模式以及广泛地理分布的全面基准。实验结果表明，DentVFM在多种牙科任务中展现了强大的通才智慧能力，如疾病诊断、治疗分析、生物标志物识别和解剖地标检测与分割。 实验证明，DentVFM在泛化性能、标签效率和可扩展性方面显著优于监督式学习、自我监督学习以及弱监督基础模型。此外，DentVFM支持跨模态诊断，在传统成像不可用的情况下提供比经验丰富的牙医更为可靠的诊断结果。DentVFM为牙科AI树立了新的范例，通过一个可扩展的、适应性强且标签高效的模型来改进智能牙科医疗保健，并填补全球口腔健康护理中的关键空白。
URL
https://arxiv.org/abs/2510.14532
PDF
