Weakly_Supervised
-
Towards Generalist Intelligence in Dentistry: Vision Foundation Models for Oral and Maxillofacial Radiology
2025-10-16 10:24:23Xinrui Huang, Fan Xiao, Dongming He, Anqi Gao, Dandan Li, Xiaofan Zhang, Shaoting Zhang, Xudong WangAbstract
Oral and maxillofacial radiology plays a vital role in dental healthcare, but radiographic image interpretation is limited by a shortage of trained professionals. While AI approaches have shown promise, existing dental AI systems are restricted by their single-modality focus, task-specific design, and reliance on costly labeled data, hindering their generalization across diverse clinical scenarios. To address these challenges, we introduce DentVFM, the first family of vision foundation models (VFMs) designed for dentistry. DentVFM generates task-agnostic visual representations for a wide range of dental applications and uses self-supervised learning on DentVista, a large curated dental imaging dataset with approximately 1.6 million multi-modal radiographic images from various medical centers. DentVFM includes 2D and 3D variants based on the Vision Transformer (ViT) architecture. To address gaps in dental intelligence assessment and benchmarks, we introduce DentBench, a comprehensive benchmark covering eight dental subspecialties, more diseases, imaging modalities, and a wide geographical distribution. DentVFM shows impressive generalist intelligence, demonstrating robust generalization to diverse dental tasks, such as disease diagnosis, treatment analysis, biomarker identification, and anatomical landmark detection and segmentation. Experimental results indicate DentVFM significantly outperforms supervised, self-supervised, and weakly supervised baselines, offering superior generalization, label efficiency, and scalability. Additionally, DentVFM enables cross-modality diagnostics, providing more reliable results than experienced dentists in situations where conventional imaging is unavailable. DentVFM sets a new paradigm for dental AI, offering a scalable, adaptable, and label-efficient model to improve intelligent dental healthcare and address critical gaps in global oral healthcare.
Abstract (translated)
口腔和颌面放射学在牙科医疗保健中扮演着至关重要的角色，但放射图像的解读受限于训练有素的专业人员短缺。虽然人工智能方法显示出巨大潜力，但现有的牙科AI系统因其单一模态聚焦、任务特异性设计以及对昂贵标记数据的依赖性，在不同临床场景中的泛化能力受到限制。为解决这些挑战，我们推出了DentVFM，这是首个专为牙科设计的视觉基础模型（VFMs）家族。DentVFM生成适用于广泛牙科应用的任务无关视觉表示，并通过在DentVista上进行自我监督学习来实现这一点——这是一个包含来自各种医疗机构的大约160万张多模态放射图像的大型策划数据库。 DentVFM包括基于视觉变换器（ViT）架构的2D和3D变体。为填补牙科智能评估与基准测试中的空白，我们推出了DentBench，这是一个涵盖八个牙科亚专科、更多疾病类型、成像模式以及广泛地理分布的全面基准。实验结果表明，DentVFM在多种牙科任务中展现了强大的通才智慧能力，如疾病诊断、治疗分析、生物标志物识别和解剖地标检测与分割。 实验证明，DentVFM在泛化性能、标签效率和可扩展性方面显著优于监督式学习、自我监督学习以及弱监督基础模型。此外，DentVFM支持跨模态诊断，在传统成像不可用的情况下提供比经验丰富的牙医更为可靠的诊断结果。DentVFM为牙科AI树立了新的范例，通过一个可扩展的、适应性强且标签高效的模型来改进智能牙科医疗保健，并填补全球口腔健康护理中的关键空白。
URL
https://arxiv.org/abs/2510.14532
PDF
-
Learning to Recognize Correctly Completed Procedure Steps in Egocentric Assembly Videos through Spatio-Temporal Modeling
2025-10-14 11:03:30Tim J. Schoonbeek, Shao-Hsuan Hung, Dan Lehman, Hans Onvlee, Jacek Kustra, Peter H. N. de With, Fons van der SommenAbstract
Procedure step recognition (PSR) aims to identify all correctly completed steps and their sequential order in videos of procedural tasks. The existing state-of-the-art models rely solely on detecting assembly object states in individual video frames. By neglecting temporal features, model robustness and accuracy are limited, especially when objects are partially occluded. To overcome these limitations, we propose Spatio-Temporal Occlusion-Resilient Modeling for Procedure Step Recognition (STORM-PSR), a dual-stream framework for PSR that leverages both spatial and temporal features. The assembly state detection stream operates effectively with unobstructed views of the object, while the spatio-temporal stream captures both spatial and temporal features to recognize step completions even under partial occlusion. This stream includes a spatial encoder, pre-trained using a novel weakly supervised approach to capture meaningful spatial representations, and a transformer-based temporal encoder that learns how these spatial features relate over time. STORM-PSR is evaluated on the MECCANO and IndustReal datasets, reducing the average delay between actual and predicted assembly step completions by 11.2% and 26.1%, respectively, compared to prior methods. We demonstrate that this reduction in delay is driven by the spatio-temporal stream, which does not rely on unobstructed views of the object to infer completed steps. The code for STORM-PSR, along with the newly annotated MECCANO labels, is made publicly available at this https URL .
Abstract (translated)
步骤识别（PSR）的目标是确定程序任务视频中所有正确完成的步骤及其顺序。现有最先进的模型仅依赖于检测单个视频帧中的装配物体状态。由于忽略了时间特征，这些模型在面对部分遮挡情况时的鲁棒性和准确性受到限制。为克服这一局限性，我们提出了时空抗遮挡模型（STORM-PSR），这是一种用于程序步骤识别的双流框架，结合了空间和时间特征的优势。 装配状态检测流能够有效地处理物体未受阻碍的情况，而时空流则捕捉了时间和空间特性，在部分遮挡的情况下也能识别出步骤完成情况。这一流程包括一个通过新型弱监督方法预训练的空间编码器，用于捕获有意义的空间表示；以及一个基于变压器的时间编码器，学习这些空间特征随时间的关系。 STORM-PSR在MECCANO和IndustReal数据集上进行了评估，在预测装配步骤完成的平均延迟方面相比先前的方法分别减少了11.2%和26.1%。我们展示了这种延迟减少是由时空流驱动的，该流程不需要物体未受阻碍的视角来推断已完成的步骤。 STORM-PSR的代码以及新标注的MECCANO标签已在以下URL公开提供：[这里应插入链接]（原文中的具体网址需要根据实际情况填写）。
URL
https://arxiv.org/abs/2510.12385
PDF
-
BEEP3D: Box-Supervised End-to-End Pseudo-Mask Generation for 3D Instance Segmentation
2025-10-14 06:23:18Youngju Yoo, Seho Kim, Changick KimAbstract
3D instance segmentation is crucial for understanding complex 3D environments, yet fully supervised methods require dense point-level annotations, resulting in substantial annotation costs and labor overhead. To mitigate this, box-level annotations have been explored as a weaker but more scalable form of supervision. However, box annotations inherently introduce ambiguity in overlapping regions, making accurate point-to-instance assignment challenging. Recent methods address this ambiguity by generating pseudo-masks through training a dedicated pseudo-labeler in an additional training stage. However, such two-stage pipelines often increase overall training time and complexity, hinder end-to-end optimization. To overcome these challenges, we propose BEEP3D-Box-supervised End-to-End Pseudo-mask generation for 3D instance segmentation. BEEP3D adopts a student-teacher framework, where the teacher model serves as a pseudo-labeler and is updated by the student model via an Exponential Moving Average. To better guide the teacher model to generate precise pseudo-masks, we introduce an instance center-based query refinement that enhances position query localization and leverages features near instance centers. Additionally, we design two novel losses-query consistency loss and masked feature consistency loss-to align semantic and geometric signals between predictions and pseudo-masks. Extensive experiments on ScanNetV2 and S3DIS datasets demonstrate that BEEP3D achieves competitive or superior performance compared to state-of-the-art weakly supervised methods while remaining computationally efficient.
Abstract (translated)
三维实例分割对于理解复杂的三维环境至关重要，然而完全监督的方法需要密集的点级别标注，这导致了大量的注释成本和劳动力投入。为了缓解这一问题，人们探索了使用箱级标注作为更弱但更具可扩展性的监督形式。然而，边界框标注在重叠区域内在固有地引入了模糊性，使得准确的点到实例分配变得具有挑战性。近期的方法通过训练一个专门的伪标签生成器来解决这种模糊性，但这通常需要额外的一个训练阶段，并且会产生伪掩码。这样的两阶段管道往往会增加整体训练时间和复杂度，阻碍端到端优化。 为了解决这些挑战，我们提出了BEEP3D——一种基于边界框监督的三维实例分割中的端到端伪掩码生成方法。BEEP3D采用了一种师生框架，在该框架中，教师模型充当伪标签生成器，并通过指数移动平均法由学生模型进行更新。为了更好地指导教师模型生成精确的伪掩码，我们引入了一个基于实例中心的查询细化机制，增强了位置查询的定位能力，并利用了靠近实例中心的特征信息。此外，我们设计了两种新颖的损失函数——查询一致性损失和掩码特征一致性损失，以便在预测与伪掩码之间对齐语义和几何信号。 在ScanNetV2和S3DIS数据集上的广泛实验表明，BEEP3D相比现有的弱监督方法取得了具有竞争力或更优的性能，并且保持了计算效率。
URL
https://arxiv.org/abs/2510.12182
PDF
-
ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer
2025-10-13 07:12:23Yuan Tian, Min Zhou, Yitong Chen, Fang Li, Lingzi Qi, Shuo Wang, Xieyang Xu, Yu Yu, Shiqiong Xu, Chaoyu Lei, Yankai Jiang, Rongzhao Zhang, Jia Tan, Li Wu, Hong Chen, Xiaowei Liu, Wei Lu, Lin Li, Huifang Zhou, Xuefei Song, Guangtao Zhai, Xianqun FanAbstract
Patient face images provide a convenient mean for evaluating eye diseases, while also raising privacy concerns. Here, we introduce ROFI, a deep learning-based privacy protection framework for ophthalmology. Using weakly supervised learning and neural identity translation, ROFI anonymizes facial features while retaining disease features (over 98\% accuracy, $\kappa > 0.90$). It achieves 100\% diagnostic sensitivity and high agreement ($\kappa > 0.90$) across eleven eye diseases in three cohorts, anonymizing over 95\% of images. ROFI works with AI systems, maintaining original diagnoses ($\kappa > 0.80$), and supports secure image reversal (over 98\% similarity), enabling audits and long-term care. These results show ROFI's effectiveness of protecting patient privacy in the digital medicine era.
Abstract (translated)
患者面部图像为评估眼科疾病提供了一种便捷的方法，同时也引发了隐私方面的担忧。在这里，我们介绍了ROFI，这是一个基于深度学习的眼科隐私保护框架。通过弱监督学习和神经身份翻译技术，ROFI能够在去除非必要面部特征的同时保留疾病的特征（超过98%的准确性，$\kappa > 0.90$）。在三个不同队列中的十一种眼科疾病上，ROFI实现了100%的诊断敏感度，并且具有高度的一致性（$\kappa > 0.90$），同时去除了超过95%图像中的非必要面部信息。此外，该框架能够与AI系统兼容，保持原始诊断结果（$\kappa > 0.80$），并支持安全的图像恢复功能（相似度超过98%），从而允许进行审核和长期跟踪护理。这些结果显示了ROFI在数字医疗时代有效保护患者隐私方面的效果。
URL
https://arxiv.org/abs/2510.11073
PDF
-
Benchmarking Deep Learning Models for Laryngeal Cancer Staging Using the LaryngealCT Dataset
2025-10-13 06:25:19Nivea Roy, Son Tran, Atul Sajjanhar, K. Devaraja, Prakashini Koteshwara, Yong Xiang, Divya RaoAbstract
Laryngeal cancer imaging research lacks standardised datasets to enable reproducible deep learning (DL) model development. We present LaryngealCT, a curated benchmark of 1,029 computed tomography (CT) scans aggregated from six collections from The Cancer Imaging Archive (TCIA). Uniform 1 mm isotropic volumes of interest encompassing the larynx were extracted using a weakly supervised parameter search framework validated by clinical experts. 3D DL architectures (3D CNN, ResNet18,50,101, DenseNet121) were benchmarked on (i) early (Tis,T1,T2) vs. advanced (T3,T4) and (ii) T4 vs. non-T4 classification tasks. 3D CNN (AUC-0.881, F1-macro-0.821) and ResNet18 (AUC-0.892, F1-macro-0.646) respectively outperformed the other models in the two tasks. Model explainability assessed using 3D GradCAMs with thyroid cartilage overlays revealed greater peri-cartilage attention in non-T4 cases and focal activations in T4 predictions. Through open-source data, pretrained models, and integrated explainability tools, LaryngealCT offers a reproducible foundation for AI-driven research to support clinical decisions in laryngeal oncology.
Abstract (translated)
喉癌的成像研究缺乏标准化的数据集，以支持可重复的深度学习（DL）模型开发。我们提出了LaryngealCT，这是一个由六个来自癌症影像档案馆（TCIA）的集合汇总而成的基准数据集，包含1,029份计算机断层扫描（CT）图像。使用弱监督参数搜索框架并经临床专家验证后，提取了涵盖喉部的统一1毫米等轴距感兴趣体积。 三维DL架构（包括3D CNN、ResNet18、50和101、DenseNet121）在两类任务上进行了基准测试：早期阶段（Tis, T1, T2）与晚期阶段（T3, T4），以及T4与非T4的分类任务。在两个任务中，3D CNN（AUC-0.881，F1-macro-0.821）和ResNet18（AUC-0.892，F1-macro-0.646）分别超过了其他模型的表现。 通过使用带有甲状软骨叠加的三维GradCAM进行模型解释性评估，非T4病例显示了更显著的周围软骨关注区域，而T4预测则显示出焦点激活。通过开源数据、预训练模型和集成可解释工具，LaryngealCT为支持喉癌临床决策的人工智能研究提供了可重复的基础。
URL
https://arxiv.org/abs/2510.11047
PDF
-
Training-Free In-Context Forensic Chain for Image Manipulation Detection and Localization
2025-10-11 08:42:31Rui Chen, Bin Liu, Changtao Miao, Xinghao Wang, Yi Li, Tao Gong, Qi Chu, Nenghai YuAbstract
Advances in image tampering pose serious security threats, underscoring the need for effective image manipulation localization (IML). While supervised IML achieves strong performance, it depends on costly pixel-level annotations. Existing weakly supervised or training-free alternatives often underperform and lack interpretability. We propose the In-Context Forensic Chain (ICFC), a training-free framework that leverages multi-modal large language models (MLLMs) for interpretable IML tasks. ICFC integrates an objectified rule construction with adaptive filtering to build a reliable knowledge base and a multi-step progressive reasoning pipeline that mirrors expert forensic workflows from coarse proposals to fine-grained forensics results. This design enables systematic exploitation of MLLM reasoning for image-level classification, pixel-level localization, and text-level interpretability. Across multiple benchmarks, ICFC not only surpasses state-of-the-art training-free methods but also achieves competitive or superior performance compared to weakly and fully supervised approaches.
Abstract (translated)
图像篡改技术的进步带来了严重的安全威胁，突显了有效图像操作定位（IML）的必要性。虽然监督下的IML取得了很好的性能，但它依赖于昂贵的像素级标注。现有的弱监督或无训练替代方法往往表现不佳且缺乏解释能力。我们提出了一个无训练框架——上下文取证链（ICFC），该框架利用多模态大型语言模型（MLLM）来进行可解释的图像操作定位任务。 ICFC结合了对象化规则构建和自适应过滤，建立了一个可靠的知识库，并形成一个多步骤渐进推理管道，这一设计模仿了专家级的取证工作流程，从粗略提案到细粒度的取证结果。这种设计能够系统地利用MLLM推理进行图像级别的分类、像素级别的定位以及文本级别的解释。 在多个基准测试中，ICFC不仅超越了现有的无训练方法，而且在性能上与弱监督和全监督方法相当或更优。
URL
https://arxiv.org/abs/2510.10111
PDF
-
Automatic Speech Recognition in the Modern Era: Architectures, Training, and Evaluation
2025-10-11 05:38:45Md. Nayeem, Md Shamse Tabrej, Kabbojit Jit Deb, Shaonti Goswami, Md. Azizul HakimAbstract
Automatic Speech Recognition (ASR) has undergone a profound transformation over the past decade, driven by advances in deep learning. This survey provides a comprehensive overview of the modern era of ASR, charting its evolution from traditional hybrid systems, such as Gaussian Mixture Model-Hidden Markov Models (GMM-HMMs) and Deep Neural Network-HMMs (DNN-HMMs), to the now-dominant end-to-end neural architectures. We systematically review the foundational end-to-end paradigms: Connectionist Temporal Classification (CTC), attention-based encoder-decoder models, and the Recurrent Neural Network Transducer (RNN-T), which established the groundwork for fully integrated speech-to-text systems. We then detail the subsequent architectural shift towards Transformer and Conformer models, which leverage self-attention to capture long-range dependencies with high computational efficiency. A central theme of this survey is the parallel revolution in training paradigms. We examine the progression from fully supervised learning, augmented by techniques like SpecAugment, to the rise of self-supervised learning (SSL) with foundation models such as wav2vec 2.0, which drastically reduce the reliance on transcribed data. Furthermore, we analyze the impact of largescale, weakly supervised models like Whisper, which achieve unprecedented robustness through massive data diversity. The paper also covers essential ecosystem components, including key datasets and benchmarks (e.g., LibriSpeech, Switchboard, CHiME), standard evaluation metrics (e.g., Word Error Rate), and critical considerations for real-world deployment, such as streaming inference, on-device efficiency, and the ethical imperatives of fairness and robustness. We conclude by outlining open challenges and future research directions.
Abstract (translated)
自动语音识别（ASR）在过去十年中经历了一场深刻的变革，这是由于深度学习的进步所推动的。本综述提供了现代ASR时代的全面概述，回顾了其从传统的混合系统（如高斯混合模型-隐马尔可夫模型（GMM-HMM）和深层神经网络-隐马尔可夫模型（DNN-HMM））到如今占主导地位的端到端神经架构的发展历程。我们系统地回顾了基础性的端到端范式：连接时间分类（CTC）、基于注意力的编码器-解码器模型以及循环神经网络转导器（RNN-T），这些范式为完全集成的语音到文本系统奠定了基础。随后，我们详细介绍了架构向变压器和卷积变压器模型转变的过程，这些模型利用自注意力机制来高效地捕捉长程依赖关系。本综述的一个中心主题是训练范式的平行革命。我们研究了从全监督学习（通过SpecAugment等技术增强）到基于wav2vec 2.0等基础模型的自我监督学习（SSL）的发展过程，这些方法极大地减少了对转录数据的依赖。此外，我们还分析了大规模弱监督模型（如Whisper）的影响，这些模型通过海量的数据多样性实现了前所未有的鲁棒性。论文还涵盖了关键生态系统组件，包括重要的数据集和基准测试（例如LibriSpeech、Switchboard、CHiME）、标准评估指标（例如单词错误率），以及在实际部署中至关重要的考虑因素，如流推理、设备效率及公平性和稳健性的伦理要求。最后，我们概述了开放性挑战与未来的研究方向。
URL
https://arxiv.org/abs/2510.12827
PDF
-
Rewiring Development in Brain Segmentation: Leveraging Adult Brain Priors for Enhancing Infant MRI Segmentation
2025-10-10 11:55:43Alemu Sisay Nigru, Michele Svanera, Austin Dibble, Connor Dalby, Mattia Savardi, Sergio BeniniAbstract
Accurate segmentation of infant brain MRI is critical for studying early neurodevelopment and diagnosing neurological disorders. Yet, it remains a fundamental challenge due to continuously evolving anatomy of the subjects, motion artifacts, and the scarcity of high-quality labeled data. In this work, we present LODi, a novel framework that utilizes prior knowledge from an adult brain MRI segmentation model to enhance the segmentation performance of infant scans. Given the abundance of publicly available adult brain MRI data, we pre-train a segmentation model on a large adult dataset as a starting point. Through transfer learning and domain adaptation strategies, we progressively adapt the model to the 0-2 year-old population, enabling it to account for the anatomical and imaging variability typical of infant scans. The adaptation of the adult model is carried out using weakly supervised learning on infant brain scans, leveraging silver-standard ground truth labels obtained with FreeSurfer. By introducing a novel training strategy that integrates hierarchical feature refinement and multi-level consistency constraints, our method enables fast, accurate, age-adaptive segmentation, while mitigating scanner and site-specific biases. Extensive experiments on both internal and external datasets demonstrate the superiority of our approach over traditional supervised learning and domain-specific models. Our findings highlight the advantage of leveraging adult brain priors as a foundation for age-flexible neuroimaging analysis, paving the way for more reliable and generalizable brain MRI segmentation across the lifespan.
Abstract (translated)
婴儿脑部MRI的精确分割对于研究早期神经发育和诊断神经系统疾病至关重要。然而，由于受试者解剖结构不断变化、运动伪影以及高质量标注数据稀缺等因素，这一任务仍面临重大挑战。在这项工作中，我们提出了LODi框架，利用成人脑部MRI分割模型中的先验知识来提高婴儿扫描的分割性能。鉴于公开可用的成人脑部MRI数据丰富，我们首先在一个大型成人数据集上预训练了一个分割模型作为起点。通过迁移学习和领域适应策略，逐步将该模型调整适用于0至2岁儿童群体，使其能够应对婴儿扫描中常见的解剖学和成像变异。 我们将成年人模型应用到婴儿脑部扫描的过程中采用了弱监督学习，并利用FreeSurfer获得的银标准地面真实标签进行训练。通过引入一种结合层级特征精炼和多级一致性约束的新颖训练策略，我们的方法实现了快速、精确且适应年龄变化的分割效果，同时减少了特定扫描仪和地点带来的偏差。 在内部和外部数据集上的广泛实验表明，相较于传统的监督学习和领域特化模型，我们所提出的方法表现更优。我们的发现强调了利用成人脑部先验知识作为跨年龄段灵活神经影像分析基础的重要性，并为整个生命周期内更加可靠且通用的脑部MRI分割铺平了道路。
URL
https://arxiv.org/abs/2510.09306
PDF
-
RASALoRE: Region Aware Spatial Attention with Location-based Random Embeddings for Weakly Supervised Anomaly Detection in Brain MRI Scans
2025-10-09 10:37:47Bheeshm Sharma, Karthikeyan Jaganathan, Balamurugan PalaniappanAbstract
Weakly Supervised Anomaly detection (WSAD) in brain MRI scans is an important challenge useful to obtain quick and accurate detection of brain anomalies when precise pixel-level anomaly annotations are unavailable and only weak labels (e.g., slice-level) are available. In this work, we propose RASALoRE: Region Aware Spatial Attention with Location-based Random Embeddings, a novel two-stage WSAD framework. In the first stage, we introduce a Discriminative Dual Prompt Tuning (DDPT) mechanism that generates high-quality pseudo weak masks based on slice-level labels, serving as coarse localization cues. In the second stage, we propose a segmentation network with a region-aware spatial attention mechanism that relies on fixed location-based random embeddings. This design enables the model to effectively focus on anomalous regions. Our approach achieves state-of-the-art anomaly detection performance, significantly outperforming existing WSAD methods while utilizing less than 8 million parameters. Extensive evaluations on the BraTS20, BraTS21, BraTS23, and MSD datasets demonstrate a substantial performance improvement coupled with a significant reduction in computational complexity. Code is available at: this https URL.
Abstract (translated)
弱监督异常检测（WSAD）在脑MRI扫描中的应用是一项重要的挑战，特别是在精确的像素级异常标注不可用且仅能获取到弱标签（如切片级别）的情况下。为了快速准确地识别大脑异常情况，在这项工作中，我们提出了一种新的两阶段WSAD框架——RASALoRE（基于位置随机嵌入的位置感知空间注意力机制）。该框架分为两个阶段： 在第一阶段，我们引入了判别性双提示调优（Discriminative Dual Prompt Tuning, DDPT）机制，能够根据切片级别的标签生成高质量的伪弱掩模，作为粗略定位线索。 第二阶段中，我们提出了一种基于位置感知空间注意力机制的分割网络，并使用固定的位置随机嵌入。这种设计使得模型能够在异常区域有效聚焦。 我们的方法在多种数据集上实现了最先进的异常检测性能，在参数量小于800万的情况下显著优于现有的WSAD方法。对BraTS20、BraTS21、BraTS23和MSD等数据集的广泛评估显示，该模型不仅提高了性能还大幅减少了计算复杂度。 代码可在以下链接获取：[提供URL]
URL
https://arxiv.org/abs/2510.08052
PDF
-
Efficient Universal Models for Medical Image Segmentation via Weakly Supervised In-Context Learning
2025-10-07 13:07:27Jiesi Hu, Yanwu Yang, Zhiyu Ye, Jinyan Zhou, Jianfeng Cao, Hanyang Peng, Ting MaAbstract
Universal models for medical image segmentation, such as interactive and in-context learning (ICL) models, offer strong generalization but require extensive annotations. Interactive models need repeated user prompts for each image, while ICL relies on dense, pixel-level labels. To address this, we propose Weakly Supervised In-Context Learning (WS-ICL), a new ICL paradigm that leverages weak prompts (e.g., bounding boxes or points) instead of dense labels for context. This approach significantly reduces annotation effort by eliminating the need for fine-grained masks and repeated user prompting for all images. We evaluated the proposed WS-ICL model on three held-out benchmarks. Experimental results demonstrate that WS-ICL achieves performance comparable to regular ICL models at a significantly lower annotation cost. In addition, WS-ICL is highly competitive even under the interactive paradigm. These findings establish WS-ICL as a promising step toward more efficient and unified universal models for medical image segmentation. Our code and model are publicly available at this https URL.
Abstract (translated)
通用医学图像分割模型，如交互式和上下文内学习（ICL）模型，虽然具有很强的泛化能力，但需要大量的标注数据。交互式模型要求用户对每个图像进行多次提示输入，而ICL则依赖于密集、像素级别的标签。为解决这一问题，我们提出了弱监督上下文内学习（WS-ICL），这是一种新的ICL范式，它利用弱提示（例如边界框或点）而不是密集标签来提供上下文信息。这种方法通过消除对精细标注掩模的需求以及所有图像的重复用户提示需求，显著减少了注释工作量。 我们在三个独立的基准测试集上评估了所提出的WS-ICL模型。实验结果表明，在大大减少标注成本的情况下，WS-ICL与常规ICL模型相比能够达到相似的效果表现。此外，即使在交互式范式下，WS-ICL也表现出很强的竞争性。 这些发现证明了WS-ICL作为更高效和统一的医学图像分割通用模型的重要一步。我们的代码和模型可以在以下链接公开获取：[此处插入URL]。
URL
https://arxiv.org/abs/2510.05899
PDF
-
Graph-based LLM over Semi-Structured Population Data for Dynamic Policy Response
2025-10-06 16:10:18Daqian Shi, Xiaolei Diao, Jinge Wu, Honghan Wu, Xiongfeng Tang, Felix Naughton, Paulina BondaronekAbstract
Timely and accurate analysis of population-level data is crucial for effective decision-making during public health emergencies such as the COVID-19 pandemic. However, the massive input of semi-structured data, including structured demographic information and unstructured human feedback, poses significant challenges to conventional analysis methods. Manual expert-driven assessments, though accurate, are inefficient, while standard NLP pipelines often require large task-specific labeled datasets and struggle with generalization across diverse domains. To address these challenges, we propose a novel graph-based reasoning framework that integrates large language models with structured demographic attributes and unstructured public feedback in a weakly supervised pipeline. The proposed approach dynamically models evolving citizen needs into a need-aware graph, enabling population-specific analyses based on key features such as age, gender, and the Index of Multiple Deprivation. It generates interpretable insights to inform responsive health policy decision-making. We test our method using a real-world dataset, and preliminary experimental results demonstrate its feasibility. This approach offers a scalable solution for intelligent population health monitoring in resource-constrained clinical and governmental settings.
Abstract (translated)
及时且准确地分析人口层面的数据对于在如新冠疫情等公共卫生紧急情况下的有效决策至关重要。然而，包括结构化的人口统计信息和非结构化的公众反馈在内的大量半结构化数据输入对传统分析方法构成了重大挑战。虽然专家驱动的手动评估尽管精确但效率低下，而标准的自然语言处理（NLP）流程通常需要大量的特定任务标签数据，并且在跨多种领域时难以实现泛化。为了解决这些问题，我们提出了一种基于图推理的新框架，该框架将大型语言模型与结构化的个人属性和非结构化的公共反馈相结合，在一个弱监督管道中运行。所提出的这种方法能够动态地将不断变化的公民需求建模成一个需要意识的图形，允许根据年龄、性别以及多种剥夺指数等关键特征进行特定人口群体的分析，并生成可解释的见解以支持响应性的卫生政策决策制定。 我们使用真实世界的数据集测试了该方法，并且初步实验结果表明其可行性。这一方法为资源有限的临床和政府环境中的人口健康智能监控提供了一个可扩展的解决方案。
URL
https://arxiv.org/abs/2510.05196
PDF
-
MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
2025-10-06 04:26:39Soo Yong Kim, Suin Cho, Vincent-Daniel Yun, Gyeongyeon HwangAbstract
Bridging clinical diagnostic reasoning with AI remains a central challenge in medical imaging. We introduce MedCLM, an automated pipeline that converts detection datasets into large-scale medical visual question answering (VQA) data with Chain-of-Thought (CoT) reasoning by linking lesion boxes to organ segmentation and structured rationales. These contextual signals enable medical vision-language models to generate question-answer pairs with step-by-step reasoning. To utilize this data effectively, we propose an Integrated CoT-Curriculum Strategy composed of an Easy stage with explicit lesion boxes for visual grounding, a Medium stage that encourages implicit localization, and a Hard stage for weakly supervised reasoning. Experimental results demonstrate that MedCLM attains state-of-the-art performance on several medical VQA benchmarks, providing a scalable framework for developing clinically aligned medical vision-language models.
Abstract (translated)
将临床诊断推理与人工智能相结合仍然是医学影像领域的一个核心挑战。我们引入了MedCLM，这是一个自动管道系统，它通过将病变框链接到器官分割和结构化理由来将检测数据集转换为具有链式思维（CoT）推理的大规模医疗视觉问答（VQA）数据。这些上下文信号使医学视觉语言模型能够生成带有逐步推理的问答对。 为了有效利用此类数据，我们提出了一种集成CoT-课程策略，包括一个使用明确病变框进行视觉定位的简单阶段、一个鼓励隐式定位的中等难度阶段以及一个用于弱监督推理的困难阶段。实验结果表明，MedCLM在多个医疗VQA基准测试上达到了最先进的性能水平，并提供了一个可扩展的框架来开发与临床实践相一致的医学视觉语言模型。
URL
https://arxiv.org/abs/2510.04477
PDF
-
Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Crops
2025-10-04 01:38:56Mattia ScardecchiaAbstract
Recent advances in self-supervised learning (SSL) have made it possible to learn general-purpose visual features that capture both the high-level semantics and the fine-grained spatial structure of images. Most notably, the recent DINOv2 has established a new state of the art by surpassing weakly supervised methods (WSL) like OpenCLIP on most benchmarks. In this survey, we examine the core ideas behind its approach, multi-crop view augmentation and self-distillation with a mean teacher, and trace their development in previous work. We then compare the performance of DINO and DINOv2 with other SSL and WSL methods across various downstream tasks, and highlight some remarkable emergent properties of their learned features with transformer backbones. We conclude by briefly discussing DINOv2's limitations, its impact, and future research directions.
Abstract (translated)
最近的自监督学习（SSL）进展使得能够学习到既捕捉图像高层语义又包含细微空间结构的一般视觉特征成为可能。尤为值得注意的是，近期推出的DINOv2通过在大多数基准测试中超越弱监督方法（WSL），如OpenCLIP，确立了新的性能标准。在这份综述中，我们将探讨其核心理念——多视角增强和带有均值教师的自我蒸馏，并追溯这些概念在先前工作中的发展。接着，我们将在各种下游任务上比较DINO与DINOv2和其他SSL及WSL方法的表现，并强调它们使用变换器（Transformer）作为骨干网络时学习到的一些显著特性。最后，我们将简要讨论DINOv2的局限性、其影响以及未来的研究方向。
URL
https://arxiv.org/abs/2510.03606
PDF
-
Learning Robust Diffusion Models from Imprecise Supervision
2025-10-03 14:00:32Dong-Dong Wu, Jiacheng Cui, Wei Wang, Zhiqiang She, Masashi SugiyamaAbstract
Conditional diffusion models have achieved remarkable success in various generative tasks recently, but their training typically relies on large-scale datasets that inevitably contain imprecise information in conditional inputs. Such supervision, often stemming from noisy, ambiguous, or incomplete labels, will cause condition mismatch and degrade generation quality. To address this challenge, we propose DMIS, a unified framework for training robust Diffusion Models from Imprecise Supervision, which is the first systematic study within diffusion models. Our framework is derived from likelihood maximization and decomposes the objective into generative and classification components: the generative component models imprecise-label distributions, while the classification component leverages a diffusion classifier to infer class-posterior probabilities, with its efficiency further improved by an optimized timestep sampling strategy. Extensive experiments on diverse forms of imprecise supervision, covering tasks of image generation, weakly supervised learning, and noisy dataset condensation demonstrate that DMIS consistently produces high-quality and class-discriminative samples.
Abstract (translated)
条件扩散模型在各种生成任务中最近取得了显著的成功，但它们的训练通常依赖于包含条件输入中不可避免地含有不精确信息的大规模数据集。这种监督往往来源于嘈杂、模糊或不完整的标签，会导致条件不匹配并降低生成质量。为了解决这一挑战，我们提出了DMIS（从含糊监督训练鲁棒扩散模型的统一框架），这是在扩散模型领域内的首个系统性研究。我们的框架源自于似然最大化，并将目标分解为生成和分类两个部分：生成部分建模模糊标签分布，而分类部分利用一个扩散分类器来推断类后验概率，并通过优化的时间步采样策略进一步提高其效率。在涵盖图像生成、弱监督学习及噪声数据集浓缩等各种形式的含糊监督上的广泛实验表明，DMIS能够持续产出高质量且具有类别区分能力的样本。
URL
https://arxiv.org/abs/2510.03016
PDF
-
Weakly Supervised Cloud Detection Combining Spectral Features and Multi-Scale Deep Network
2025-10-01 08:32:49Shaocong Zhu, Zhiwei Li, Xinghua Li, Huanfeng ShenAbstract
Clouds significantly affect the quality of optical satellite images, which seriously limits their precise application. Recently, deep learning has been widely applied to cloud detection and has achieved satisfactory results. However, the lack of distinctive features in thin clouds and the low quality of training samples limit the cloud detection accuracy of deep learning methods, leaving space for further improvements. In this paper, we propose a weakly supervised cloud detection method that combines spectral features and multi-scale scene-level deep network (SpecMCD) to obtain highly accurate pixel-level cloud masks. The method first utilizes a progressive training framework with a multi-scale scene-level dataset to train the multi-scale scene-level cloud detection network. Pixel-level cloud probability maps are then obtained by combining the multi-scale probability maps and cloud thickness map based on the characteristics of clouds in dense cloud coverage and large cloud-area coverage images. Finally, adaptive thresholds are generated based on the differentiated regions of the scene-level cloud masks at different scales and combined with distance-weighted optimization to obtain binary cloud masks. Two datasets, WDCD and GF1MS-WHU, comprising a total of 60 Gaofen-1 multispectral (GF1-MS) images, were used to verify the effectiveness of the proposed method. Compared to the other weakly supervised cloud detection methods such as WDCD and WSFNet, the F1-score of the proposed SpecMCD method shows an improvement of over 7.82%, highlighting the superiority and potential of the SpecMCD method for cloud detection under different cloud coverage conditions.
Abstract (translated)
云对光学卫星图像的质量有显著影响，这严重限制了它们的精确应用。近年来，深度学习在云检测方面得到了广泛应用，并取得了令人满意的结果。然而，薄云缺乏独特特征以及训练样本质量低的问题，仍然制约着深度学习方法在提高云检测准确性方面的潜力。 本文提出了一种结合光谱特性和多尺度场景级深层网络（SpecMCD）的弱监督云检测方法，以获取高度准确的像素级云掩膜。该方法首先使用一个包含多尺度场景数据集的渐进式训练框架来训练多尺度场景级云检测网络。随后通过组合多尺度概率图和基于密集云覆盖及大范围云区图像特征的云厚度图，得到像素级云概率图。最后根据不同尺度下的场景级云掩膜区域差异性生成自适应阈值，并结合距离加权优化获取二进制云掩膜。 为了验证所提出方法的有效性，我们使用了包含总计60张高分1号多光谱（GF1-MS）图像的两个数据集WDCD和GF1MS-WHU。与现有的其他弱监督云检测方法如WDCD和WSFNet相比，SpecMCD方法的F1分数提升了超过7.82%，这突显了SpecMCD方法在不同云覆盖条件下的优越性和潜力。
URL
https://arxiv.org/abs/2510.00654
PDF
-
LMILAtt: A Deep Learning Model for Depression Detection from Social Media Users Enhanced by Multi-Instance Learning Based on Attention Mechanism
2025-09-30 11:58:32Yukun YangAbstract
Depression is a major global public health challenge and its early identification is crucial. Social media data provides a new perspective for depression detection, but existing methods face limitations such as insufficient accuracy, insufficient utilization of time series features, and high annotation costs. To this end, this study proposes the LMILAtt model, which innovatively integrates Long Short-Term Memory autoencoders and attention mechanisms: firstly, the temporal dynamic features of user tweets (such as depressive tendency evolution patterns) are extracted through unsupervised LSTM autoencoders. Secondly, the attention mechanism is used to dynamically weight key texts (such as early depression signals) and construct a multi-example learning architecture to improve the accuracy of user-level detection. Finally, the performance was verified on the WU3D dataset labeled by professional medicine. Experiments show that the model is significantly better than the baseline model in terms of accuracy, recall and F1 score. In addition, the weakly supervised learning strategy significantly reduces the cost of labeling and provides an efficient solution for large-scale social media depression screening.
Abstract (translated)
抑郁症是全球公共健康面临的主要挑战，其早期识别至关重要。社交媒体数据为抑郁检测提供了新的视角，但现有方法存在诸如准确性不足、时间序列特征利用不充分以及标注成本高等限制。为此，本研究提出了一种名为LMILAtt的模型，该模型创新性地结合了长短期记忆自动编码器（LSTM autoencoders）和注意力机制：首先，通过无监督的LSTM自动编码器提取用户推文的时间动态特征（例如抑郁倾向演变模式）。其次，利用注意力机制对关键文本进行动态加权（如早期抑郁症信号），并构建多例学习架构以提高用户级别检测的准确性。最后，在由专业医学人员标注的WU3D数据集上验证了模型性能。实验表明，该模型在准确率、召回率和F1值方面均显著优于基准模型。此外，弱监督学习策略大幅降低了标注成本，并为大规模社交媒体抑郁筛查提供了高效的解决方案。
URL
https://arxiv.org/abs/2509.26145
PDF
-
UniPose: Unified Cross-modality Pose Prior Propagation towards RGB-D data for Weakly Supervised 3D Human Pose Estimation
2025-09-27 15:49:30Jinghong Zheng, Changlong Jiang, Jiaqi Li, Haohong Kuang, Hang Xu, Tingbing YanAbstract
In this paper, we present UniPose, a unified cross-modality pose prior propagation method for weakly supervised 3D human pose estimation (HPE) using unannotated single-view RGB-D sequences (RGB, depth, and point cloud data). UniPose transfers 2D HPE annotations from large-scale RGB datasets (e.g., MS COCO) to the 3D domain via self-supervised learning on easily acquired RGB-D sequences, eliminating the need for labor-intensive 3D keypoint annotations. This approach bridges the gap between 2D and 3D domains without suffering from issues related to multi-view camera calibration or synthetic-to-real data shifts. During training, UniPose leverages off-the-shelf 2D pose estimations as weak supervision for point cloud networks, incorporating spatial-temporal constraints like body symmetry and joint motion. The 2D-to-3D back-projection loss and cross-modality interaction further enhance this process. By treating the point cloud network's 3D HPE results as pseudo ground truth, our anchor-to-joint prediction method performs 3D lifting on RGB and depth networks, making it more robust against inaccuracies in 2D HPE results compared to state-of-the-art methods. Experiments on CMU Panoptic and ITOP datasets show that UniPose achieves comparable performance to fully supervised methods. Incorporating large-scale unlabeled data (e.g., NTU RGB+D 60) enhances its performance under challenging conditions, demonstrating its potential for practical applications. Our proposed 3D lifting method also achieves state-of-the-art results.
Abstract (translated)
在这篇论文中，我们介绍了UniPose，这是一种统一的跨模态姿态先验传播方法，用于使用未标注的单视图RGB-D序列（包括RGB、深度和点云数据）进行弱监督下的三维人体姿态估计（HPE）。UniPose通过在易于获取的RGB-D序列上进行自监督学习，将大规模RGB数据集（如MS COCO）中的2D HPE注释转移到3D领域，从而消除了劳动密集型3D关键点标注的需求。这种方法弥合了2D和3D领域的差距，并避免了与多视角相机校准或合成到现实的数据偏移相关的问题。 在训练过程中，UniPose利用现成的2D姿态估计作为弱监督信号，用于点云网络，并结合空间-时间约束（如身体对称性和关节运动）。2D至3D反投影损失和跨模态交互进一步增强了这一过程。通过将点云网络的3D HPE结果视为伪真实值，我们的锚点到关节预测方法在RGB和深度网络上执行3D提升操作，与最先进的方法相比，在处理2D HPE结果中的不准确性时更为鲁棒。 在CMU Panoptic和ITOP数据集上的实验表明，UniPose达到了与完全监督方法相当的性能。结合大规模未标注数据（如NTU RGB+D 60）提高了其在挑战性条件下的表现，展示了其实用应用的巨大潜力。我们提出的3D提升方法也取得了最新的成果。
URL
https://arxiv.org/abs/2509.23376
PDF
-
Weakly Supervised Food Image Segmentation using Vision Transformers and Segment Anything Model
2025-09-23 14:01:51Ioannis Sarafis, Alexandros Papadopoulos, Anastasios DelopoulosAbstract
In this paper, we propose a weakly supervised semantic segmentation approach for food images which takes advantage of the zero-shot capabilities and promptability of the Segment Anything Model (SAM) along with the attention mechanisms of Vision Transformers (ViTs). Specifically, we use class activation maps (CAMs) from ViTs to generate prompts for SAM, resulting in masks suitable for food image segmentation. The ViT model, a Swin Transformer, is trained exclusively using image-level annotations, eliminating the need for pixel-level annotations during training. Additionally, to enhance the quality of the SAM-generated masks, we examine the use of image preprocessing techniques in combination with single-mask and multi-mask SAM generation strategies. The methodology is evaluated on the FoodSeg103 dataset, generating an average of 2.4 masks per image (excluding background), and achieving an mIoU of 0.54 for the multi-mask scenario. We envision the proposed approach as a tool to accelerate food image annotation tasks or as an integrated component in food and nutrition tracking applications.
Abstract (translated)
在这篇论文中，我们提出了一种针对食品图像的弱监督语义分割方法，该方法利用了Segment Anything Model (SAM) 的零样本能力和可提示性以及Vision Transformers (ViTs) 的注意力机制。具体来说，我们使用来自ViT的类激活图(CAMs)为SAM生成提示，从而产生适合于食品图像分割的掩码。作为ViT模型的Swin Transformer仅通过图像级别的标注进行训练，无需在训练过程中使用像素级别的注释。 为了提高由SAM生成的掩码的质量，我们还研究了结合单个掩码和多个掩码的SAM生成策略与图像预处理技术相结合的方法。该方法在FoodSeg103数据集上进行了评估，在不包括背景的情况下平均每张图片产生2.4个掩码，并且在多掩码场景下达到了0.54的mIoU。 我们设想提出的这种方法可以作为一种工具来加速食品图像标注任务，或者作为食物和营养追踪应用中的集成组件。
URL
https://arxiv.org/abs/2509.19028
PDF
-
Prompt-DAS: Annotation-Efficient Prompt Learning for Domain Adaptive Semantic Segmentation of Electron Microscopy Images
2025-09-23 13:26:06Jiabao Chen, Shan Xiong, Jialin PengAbstract
Domain adaptive segmentation (DAS) of numerous organelle instances from large-scale electron microscopy (EM) is a promising way to enable annotation-efficient learning. Inspired by SAM, we propose a promptable multitask framework, namely Prompt-DAS, which is flexible enough to utilize any number of point prompts during the adaptation training stage and testing stage. Thus, with varying prompt configurations, Prompt-DAS can perform unsupervised domain adaptation (UDA) and weakly supervised domain adaptation (WDA), as well as interactive segmentation during testing. Unlike the foundation model SAM, which necessitates a prompt for each individual object instance, Prompt-DAS is only trained on a small dataset and can utilize full points on all instances, sparse points on partial instances, or even no points at all, facilitated by the incorporation of an auxiliary center-point detection task. Moreover, a novel prompt-guided contrastive learning is proposed to enhance discriminative feature learning. Comprehensive experiments conducted on challenging benchmarks demonstrate the effectiveness of the proposed approach over existing UDA, WDA, and SAM-based approaches.
Abstract (translated)
领域自适应分割（DAS）从大规模电子显微镜（EM）图像中识别众多细胞器实例是一种有望提高注释效率的学习方式。受到SAM的启发，我们提出了一种可提示的多任务框架——Prompt-DAS，它在适应性训练阶段和测试阶段都能够灵活地利用任意数量的点提示。因此，通过不同的提示配置，Prompt-DAS可以执行无监督领域自适应（UDA）、弱监督领域自适应（WDA）以及交互式分割。与基础模型SAM不同，后者需要为每个单独的对象实例提供一个提示，Prompt-DAS仅在一个小数据集上进行训练，并且能够利用所有实例的全部点、部分实例的稀疏点或根本不使用任何点，这得益于辅助中心点检测任务的加入。此外，还提出了一种新颖的提示引导对比学习方法以增强判别性特征的学习。在具有挑战性的基准上的全面实验表明，所提出的方法优于现有的UDA、WDA和基于SAM的方法。
URL
https://arxiv.org/abs/2509.18973
PDF
-
Depth Edge Alignment Loss: DEALing with Depth in Weakly Supervised Semantic Segmentation
2025-09-22 12:42:10Patrick Schmidt, Vasileios Belagiannis, Lazaros NalpantidisAbstract
Autonomous robotic systems applied to new domains require an abundance of expensive, pixel-level dense labels to train robust semantic segmentation models under full supervision. This study proposes a model-agnostic Depth Edge Alignment Loss to improve Weakly Supervised Semantic Segmentation models across different datasets. The methodology generates pixel-level semantic labels from image-level supervision, avoiding expensive annotation processes. While weak supervision is widely explored in traditional computer vision, our approach adds supervision with pixel-level depth information, a modality commonly available in robotic systems. We demonstrate how our approach improves segmentation performance across datasets and models, but can also be combined with other losses for even better performance, with improvements up to +5.439, +1.274 and +16.416 points in mean Intersection over Union on the PASCAL VOC / MS COCO validation, and the HOPE static onboarding split, respectively. Our code will be made publicly available.
Abstract (translated)
自主机器人系统在新领域的应用需要大量的昂贵的像素级密集标签来训练完全监督下的稳健语义分割模型。本研究提出了一种与模型无关的深度边缘对齐损失（Depth Edge Alignment Loss），以改进跨不同数据集的弱监督语义分割模型。该方法通过图像级别的监督生成像素级语义标签，从而避免了昂贵的标注过程。尽管传统计算机视觉中广泛探索了弱监督问题，但我们的方法增加了像素级深度信息作为监督，这种模态在机器人系统中通常可以获取到。我们展示了如何改进不同数据集和模型上的分割性能，并且该方法还可以与其他损失结合使用以获得更好的效果，在PASCAL VOC / MS COCO验证集和HOPE静态上行分集分别提高了+5.439，+1.274和+16.416的平均交并比（mean Intersection over Union）点数。我们的代码将公开发布。
URL
https://arxiv.org/abs/2509.17702
PDF
