Weakly_Supervised
-
Geographically-Weighted Weakly Supervised Bayesian High-Resolution Transformer for 200m Resolution Pan-Arctic Sea Ice Concentration Mapping and Uncertainty Estimation using Sentinel-1, RCM, and AMSR2 Data
2026-03-03 20:23:11Mabel Heffring, Lincoln Linlin XuAbstract
Although high-resolution mapping of pan-Arctic sea ice with reliable corresponding uncertainty is essential for operational sea ice concentration (SIC) charting, it is a difficult task due to key challenges, such as the subtle nature of ice signature features, inexact SIC labels, model uncertainty, and data heterogeneity. This study presents a novel Bayesian High-Resolution Transformer approach for 200 meter resolution pan-Arctic SIC mapping and uncertainty quantification using Sentinel-1, RADARSAT Constellation Mission (RCM), and Advanced Microwave Scanning Radiometer 2 (AMSR2) data. First, to improve small and subtle sea ice feature (e.g., cracks/leads, ponds, and ice floes) extraction, we design a novel high-resolution Transformer model with both global and local modules that can better discern the subtle differences in sea ice patterns. Second, to address low-resolution and inexact SIC labels, we design a geographically-weighted weakly supervised loss function to supervise the model at region level instead of pixel level, and to prioritize pure open water and ice pack signatures while mitigating the impact of ambiguity in the marginal ice zone (MIZ). Third, to improve uncertainty quantification, we design a Bayesian extension of the proposed Transformer model, treating its parameters as random variables to more effectively capture uncertainties. Fourth, to address data heterogeneity, we fuse three different data types (Sentinel-1, RCM, and AMSR2) at decision-level to improve both SIC mapping and uncertainty quantification. The proposed approach is evaluated under pan-Arctic minimum-extent conditions in 2021 and 2025. Results demonstrate that the proposed model achieves 0.70 overall feature detection accuracy using Sentinel-1 data, while also preserving pan-Arctic SIC patterns (Sentinel-1 R\textsuperscript{2} = 0.90 relative to the ARTIST Sea Ice product).
Abstract (translated)
尽管高分辨率的泛北极海冰制图并可靠地量化相应的不确定性对于操作性海冰浓度（SIC）绘图至关重要，但由于一些关键挑战——如冰特征细微、SIC标签不准确、模型不确定性以及数据异质性的原因，这一任务十分艰难。本研究提出了一种新颖的贝叶斯高分辨率变换器方法，用于利用Sentinel-1、RADARSAT星座使命（RCM）和高级微波扫描辐射计2号（AMSR2）的数据进行泛北极海冰浓度的200米分辨率制图及不确定性量化。具体而言： 第一，为了改进小而微妙的海冰特征（如裂缝/缝隙、池塘、浮冰）的提取，我们设计了一种新颖的高分辨率变换器模型，该模型同时具备全局和局部模块，能够更好地识别海冰模式中的细微差异。 第二，为了解决低分辨率和不准确的SIC标签问题，我们设计了一个地理加权弱监督损失函数，在区域级别而非像素级别上监督模型，并优先考虑纯净开放水域及密集冰区的特征，同时缓解边际冰带（MIZ）中模糊性的不利影响。 第三，为了改进不确定性量化，我们将贝叶斯扩展应用于提出的变换器模型，将其参数视为随机变量，从而更有效地捕捉到不确定因素。 第四，为了解决数据异质性问题，我们融合了三种不同类型的数据（Sentinel-1、RCM和AMSR2），在决策层面提高SIC制图及不确定性量化的准确性。 所提出的方法已在2021年和2025年的泛北极最小范围条件下进行了评估。结果显示，在使用Sentinel-1数据的情况下，该模型实现了0.70的整体特征检测准确率，并且保留了泛北极的海冰浓度模式（相对于ARTIST Sea Ice产品，Sentinel-1 R² = 0.90）。
URL
https://arxiv.org/abs/2603.03503
PDF
-
Benchmarking Compact VLMs for Clip-Level Surveillance Anomaly Detection Under Weak Supervision
2026-03-03 11:42:47Kirill Borodin, Kirill Kondrashov, Nikita Vasiliev, Ksenia Gladkova, Inna Larina, Mikhail Gorodnichev, Grach MkrtchianAbstract
CCTV safety monitoring demands anomaly detectors combine reliable clip-level accuracy with predictable per-clip latency despite weak supervision. This work investigates compact vision-language models (VLMs) as practical detectors for this regime. A unified evaluation protocol standardizes preprocessing, prompting, dataset splits, metrics, and runtime settings to compare parameter-efficiently adapted compact VLMs against training-free VLM pipelines and weakly supervised baselines. Evaluation spans accuracy, precision, recall, F1, ROC-AUC, and average per-clip latency to jointly quantify detection quality and efficiency. With parameter-efficient adaptation, compact VLMs achieve performance on par with, and in several cases exceeding, established approaches while retaining competitive per-clip latency. Adaptation further reduces prompt sensitivity, producing more consistent behavior across prompt regimes under the shared protocol. These results show that parameter-efficient fine-tuning enables compact VLMs to serve as dependable clip-level anomaly detectors, yielding a favorable accuracy-efficiency trade-off within a transparent and consistent experimental setup.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13306
PDF
-
Weakly Supervised Video Anomaly Detection with Anomaly-Connected Components and Intention Reasoning
Abstract
Weakly supervised video anomaly detection (WS-VAD) involves identifying the temporal intervals that contain anomalous events in untrimmed videos, where only video-level annotations are provided as supervisory signals. However, a key limitation persists in WS-VAD, as dense frame-level annotations are absent, which often leaves existing methods struggling to learn anomaly semantics effectively. To address this issue, we propose a novel framework named LAS-VAD, short for Learning Anomaly Semantics for WS-VAD, which integrates anomaly-connected component mechanism and intention awareness mechanism. The former is designed to assign video frames into distinct semantic groups within a video, and frame segments within the same group are deemed to share identical semantic information. The latter leverages an intention-aware strategy to distinguish between similar normal and abnormal behaviors (e.g., taking items and stealing). To further model the semantic information of anomalies, as anomaly occurrence is accompanied by distinct characteristic attributes (i.e., explosions are characterized by flames and thick smoke), we additionally incorporate anomaly attribute information to guide accurate detection. Extensive experiments on two benchmark datasets, XD-Violence and UCF-Crime, demonstrate that our LAS-VAD outperforms current state-of-the-art methods with remarkable gains.
Abstract (translated)
弱监督视频异常检测（WS-VAD）旨在识别未修剪视频中包含异常事件的时间间隔，仅使用视频级别的标注作为监督信号。然而，在这种情况下，由于缺少密集的帧级注释，现有的方法在有效学习异常语义方面面临挑战。为了解决这个问题，我们提出了一种新的框架——用于弱监督视频异常检测（WS-VAD）的学习异常语义框架（LAS-VAD），该框架结合了异常连接组件机制和意图感知机制。 异常连接组件机制旨在将视频帧分配到视频内的不同语义组中，并认为同一组中的帧片段共享相同的语义信息。而意图感知策略则通过识别类似但不同的正常行为与异常行为（例如，取物品和偷窃）之间的区别来提高检测准确性。 此外，为了进一步建模异常的语义信息——因为异常发生时伴随着独特的特征属性（如爆炸事件具有火焰和浓烟等特定属性），我们还整合了异常属性信息以指导准确检测。在两个基准数据集XD-Violence和UCF-Crime上进行的大量实验表明，我们的LAS-VAD方法比当前最先进的方法有显著的性能提升。
URL
https://arxiv.org/abs/2603.00550
PDF
-
FastBUS: A Fast Bayesian Framework for Unified Weakly-Supervised Learning
2026-02-28 07:27:21Ziquan Wang, Haobo Wang, Ke Chen, Lei Feng, Gang ChenAbstract
Machine Learning often involves various imprecise labels, leading to diverse weakly supervised settings. While recent methods aim for universal handling, they usually suffer from complex manual pre-work, ignore the relationships between associated labels, or are unable to batch process due to computational design flaws, resulting in long running times. To address these limitations, we propose a novel general framework that efficiently infers latent true label distributions across various weak supervisions. Our key idea is to express the label brute-force search process as a probabilistic transition of label variables, compressing diverse weakly supervised DFS tree structures into a shared Bayesian network. From this, we derived a latent probability calculation algorithm based on generalized belief propagation and proposed two joint acceleration strategies: 1) introducing a low-rank assumption to approximate the transition matrix, reducing time complexity; 2) designing an end-to-end state evolution module to learn batch-scale transition matrices, facilitating multi-category batch processing. In addition, the equivalence of our method with the EM algorithm in most scenarios is further demonstrated. Extensive experiments show that our method achieves SOTA results under most weakly supervised settings, and achieves up to hundreds of times faster acceleration in running time compared to other general methods.
Abstract (translated)
机器学习经常涉及各种不精确的标签，导致了多种弱监督设置。尽管最近的方法旨在实现通用处理，但它们通常需要复杂的手动前期工作、忽略相关标签之间的关系，或者由于计算设计缺陷而无法批量处理，从而导致运行时间过长。为了解决这些限制，我们提出了一种新的通用框架，该框架能够高效地在各种弱监督下推断潜在的真实标签分布。 我们的核心思想是将标签暴力搜索过程表达为标签变量的概率转换，并将多样的弱监督DFS（深度优先搜索）树结构压缩成一个共享的贝叶斯网络。基于此，我们从广义信念传播中导出了一个潜在概率计算算法，并提出了两种联合加速策略： 1. 引入低秩假设来近似转移矩阵，从而降低时间复杂度； 2. 设计了一个端到端状态演化模块以学习批量规模的转移矩阵，促进多类别批量处理。 此外，我们的方法在大多数情况下与EM（期望最大化）算法等价性也得到了进一步证明。广泛的实验表明，在大多数弱监督设置下，我们的方法实现了最先进的结果，并且相比其他通用方法能够实现运行时间最多快几百倍的加速效果。
URL
https://arxiv.org/abs/2603.00517
PDF
-
TriLite: Efficient Weakly Supervised Object Localization with Universal Visual Features and Tri-Region Disentanglement
2026-02-26 15:33:22Arian Sabaghi, Jos\'e OramasAbstract
Weakly supervised object localization (WSOL) aims to localize target objects in images using only image-level labels. Despite recent progress, many approaches still rely on multi-stage pipelines or full fine-tuning of large backbones, which increases training cost, while the broader WSOL community continues to face the challenge of partial object coverage. We present TriLite, a single-stage WSOL framework that leverages a frozen Vision Transformer with Dinov2 pre-training in a self-supervised manner, and introduces only a minimal number of trainable parameters (fewer than 800K on ImageNet-1K) for both classification and localization. At its core is the proposed TriHead module, which decomposes patch features into foreground, background, and ambiguous regions, thereby improving object coverage while suppressing spurious activations. By disentangling classification and localization objectives, TriLite effectively exploits the universal representations learned by self-supervised ViTs without requiring expensive end-to-end training. Extensive experiments on CUB-200-2011, ImageNet-1K, and OpenImages demonstrate that TriLite sets a new state of the art, while remaining significantly more parameter-efficient and easier to train than prior methods. The code will be released soon.
Abstract (translated)
弱监督目标定位（WSOL）旨在仅使用图像级标签来识别图像中的目标对象。尽管近年来取得了一些进展，但许多方法仍然依赖于多阶段流程或大型骨干网络的完全微调，这增加了训练成本，而更广泛的WSOL社区仍在面对部分目标覆盖范围的问题挑战。 我们提出了一种名为TriLite的一阶段WSOL框架，该框架利用冻结的Vision Transformer（ViT）在自监督模式下进行Dinov2预训练，并且引入了非常少量可训练参数（少于800K个参数用于ImageNet-1K），这些参数同时支持分类和定位任务。TriLite的核心是所提出的TriHead模块，该模块将图像块特征分解为前景、背景及模棱两可区域，从而提高目标覆盖范围并抑制不必要激活。 通过分离分类和定位的目标，TriLite可以有效地利用自监督ViT学习到的通用表示，而无需昂贵的端到端训练。在CUB-200-2011、ImageNet-1K 和 OpenImages 上进行的广泛实验表明，TriLite在保持比先前方法更高效且更容易训练的同时，确立了新的技术水平。代码不久将公开发布。
URL
https://arxiv.org/abs/2602.23120
PDF
-
Cytoarchitecture in Words: Weakly Supervised Vision-Language Modeling for Human Brain Microscopy
2026-02-26 15:10:39Matthew Sutton, Katrin Amunts, Timo Dickscheid, Christian SchifferAbstract
Foundation models increasingly offer potential to support interactive, agentic workflows that assist researchers during analysis and interpretation of image data. Such workflows often require coupling vision to language to provide a natural-language interface. However, paired image-text data needed to learn this coupling are scarce and difficult to obtain in many research and clinical settings. One such setting is microscopic analysis of cell-body-stained histological human brain sections, which enables the study of cytoarchitecture: cell density and morphology and their laminar and areal organization. Here, we propose a label-mediated method that generates meaningful captions from images by linking images and text only through a label, without requiring curated paired image-text data. Given the label, we automatically mine area descriptions from related literature and use them as synthetic captions reflecting canonical cytoarchitectonic attributes. An existing cytoarchitectonic vision foundation model (CytoNet) is then coupled to a large language model via an image-to-text training objective, enabling microscopy regions to be described in natural language. Across 57 brain areas, the resulting method produces plausible area-level descriptions and supports open-set use through explicit rejection of unseen areas. It matches the cytoarchitectonic reference label for in-scope patches with 90.6% accuracy and, with the area label masked, its descriptions remain discriminative enough to recover the area in an 8-way test with 68.6% accuracy. These results suggest that weak, label-mediated pairing can suffice to connect existing biomedical vision foundation models to language, providing a practical recipe for integrating natural-language in domains where fine-grained paired annotations are scarce.
Abstract (translated)
基础模型日益展现出支持互动、代理工作流程的潜力，这些工作流程能在图像数据分析过程中协助研究人员。这类工作流程通常需要将视觉与语言相结合，以提供自然语言界面。然而，在许多研究和临床环境中获取学习这种结合所需的配对图像文本数据却非常稀缺且难以获得。一个典型的例子是人类大脑切片中细胞体着色组织学的显微分析，这有助于研究细胞结构：包括细胞密度、形态以及它们在层状和区域中的排列方式。在此背景下，我们提出了一种标签介导的方法，该方法通过标签将图像与文本链接起来生成有意义的描述，而无需依赖精心策划的配对图像文本数据。给定特定标签后，我们可以自动从相关文献中挖掘出对应区域的描述，并将其用作合成说明，反映规范化的细胞结构属性。然后，现有的细胞结构视觉基础模型（CytoNet）通过一个将图像转换为文本的训练目标与大型语言模型相结合，使得显微镜下观察到的区域能够以自然语言进行描述。在涉及57个脑区的情况下，该方法产生的描述符具备了合理的区域性特征，并支持开放集使用，通过对未见过的区域明确拒绝来实现这一点。对于在范围内的补丁，其匹配细胞结构参考标签准确率为90.6%，并且即使遮蔽了区域标签，在一个八分类测试中仍能以68.6%的准确率恢复出该区域。 这些结果表明，弱标签介导配对足以将现有的生物医学视觉基础模型与语言连接起来，在精细标注稀缺的情况下为自然语言的集成提供了一个实际的方法。
URL
https://arxiv.org/abs/2602.23088
PDF
-
Cross Pseudo Labeling For Weakly Supervised Video Anomaly Detection
2026-02-19 04:42:13Lee Dayeon, Kim Dongheyong, Park Chaewon, Woo Sungmin, Lee SangyounAbstract
Weakly supervised video anomaly detection aims to detect anomalies and identify abnormal categories with only video-level labels. We propose CPL-VAD, a dual-branch framework with cross pseudo labeling. The binary anomaly detection branch focuses on snippet-level anomaly localization, while the category classification branch leverages vision-language alignment to recognize abnormal event categories. By exchanging pseudo labels, the two branches transfer complementary strengths, combining temporal precision with semantic discrimination. Experiments on XD-Violence and UCF-Crime demonstrate that CPL-VAD achieves state-of-the-art performance in both anomaly detection and abnormal category classification.
Abstract (translated)
URL
https://arxiv.org/abs/2602.17077
PDF
-
MB-DSMIL-CL-PL: Scalable Weakly Supervised Ovarian Cancer Subtype Classification and Localisation Using Contrastive and Prototype Learning with Frozen Patch Features
2026-02-16 19:33:33Marcus Jenkins, Jasenka Mazibrada, Bogdan Leahu, Michal MackiewiczAbstract
The study of histopathological subtypes is valuable for the personalisation of effective treatment strategies for ovarian cancer. However, increasing diagnostic workloads present a challenge for UK pathology departments, leading to the rise in AI approaches. While traditional approaches in this field have relied on pre-computed, frozen image features, recent advances have shifted towards end-to-end feature extraction, providing an improvement in accuracy but at the expense of significantly reduced scalability during training and time-consuming experimentation. In this paper, we propose a new approach for subtype classification and localisation in ovarian cancer histopathology images using contrastive and prototype learning with pre-computed, frozen features via feature-space augmentations. Compared to DSMIL, our method achieves an improvement of 70.4\% and 15.3\% in F1 score for instance- and slide-level classification, respectively, along with AUC gains of 16.9\% for instance localisation and 2.3\% for slide classification, while maintaining the use of frozen patch features.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15138
PDF
-
AIWizards at MULTIPRIDE: A Hierarchical Approach to Slur Reclamation Detection
2026-02-13 11:01:19Luca Tedeschini, Matteo FasuloAbstract
Detecting reclaimed slurs represents a fundamental challenge for hate speech detection systems, as the same lexcal items can function either as abusive expressions or as in-group affirmations depending on social identity and context. In this work, we address Subtask B of the MultiPRIDE shared task at EVALITA 2026 by proposing a hierarchical approach to modeling the slur reclamation process. Our core assumption is that members of the LGBTQ+ community are more likely, on average, to employ certain slurs in a eclamatory manner. Based on this hypothesis, we decompose the task into two stages. First, using a weakly supervised LLM-based annotation, we assign fuzzy labels to users indicating the likelihood of belonging to the LGBTQ+ community, inferred from the tweet and the user bio. These soft labels are then used to train a BERT-like model to predict community membership, encouraging the model to learn latent representations associated with LGBTQ+ identity. In the second stage, we integrate this latent space with a newly initialized model for the downstream slur reclamation detection task. The intuition is that the first model encodes user-oriented sociolinguistic signals, which are then fused with representations learned by a model pretrained for hate speech detection. Experimental results on Italian and Spanish show that our approach achieves performance statistically comparable to a strong BERT-based baseline, while providing a modular and extensible framework for incorporating sociolinguistic context into hate speech modeling. We argue that more fine-grained hierarchical modeling of user identity and discourse context may further improve the detection of reclaimed language. We release our code at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2602.12818
PDF
-
Prototype-driven fusion of pathology and spatial transcriptomics for interpretable survival prediction
2026-02-12 21:59:07Lihe Liu, Xiaoxi Pan, Yinyin Yuan, Lulu ShangAbstract
Whole slide images (WSIs) enable weakly supervised prognostic modeling via multiple instance learning (MIL). Spatial transcriptomics (ST) preserves in situ gene expression, providing a spatial molecular context that complements morphology. As paired WSI-ST cohorts scale to population level, leveraging their complementary spatial signals for prognosis becomes crucial; however, principled cross-modal fusion strategies remain limited for this paradigm. To this end, we introduce PathoSpatial, an interpretable end-to-end framework integrating co-registered WSIs and ST to learn spatially informed prognostic representations. PathoSpatial uses task-guided prototype learning within a multi-level experts architecture, adaptively orchestrating unsupervised within-modality discovery with supervised cross-modal aggregation. By design, PathoSpatial substantially strengthens interpretability while maintaining discriminative ability. We evaluate PathoSpatial on a triple-negative breast cancer cohort with paired ST and WSIs. PathoSpatial delivers strong and consistent performance across five survival endpoints, achieving superior or comparable performance to leading unimodal and multimodal methods. PathoSpatial inherently enables post-hoc prototype interpretation and molecular risk decomposition, providing quantitative, biologically grounded explanations, highlighting candidate prognostic factors. We present PathoSpatial as a proof-of-concept for scalable and interpretable multimodal learning for spatial omics-pathology fusion.
Abstract (translated)
URL
https://arxiv.org/abs/2602.12441
PDF
-
PLESS: Pseudo-Label Enhancement with Spreading Scribbles for Weakly Supervised Segmentation
2026-02-12 06:24:05Yeva Gabrielyan (Akian College of Science and Engineering, American University of Armenia, Yerevan, Armenia), Varduhi Yeghiazaryan (Akian College of Science and Engineering, American University of Armenia, Yerevan, Armenia), Irina Voiculescu (Department of Computer Science, University of Oxford, Oxford, UK)Abstract
Weakly supervised learning with scribble annotations uses sparse user-drawn strokes to indicate segmentation labels on a small subset of pixels. This annotation reduces the cost of dense pixel-wise labeling, but suffers inherently from noisy and incomplete supervision. Recent scribble-based approaches in medical image segmentation address this limitation using pseudo-label-based training; however, the quality of the pseudo-labels remains a key performance limit. We propose PLESS, a generic pseudo-label enhancement strategy which improves reliability and spatial consistency. It builds on a hierarchical partitioning of the image into a hierarchy of spatially coherent regions. PLESS propagates scribble information to refine pseudo-labels within semantically coherent regions. The framework is model-agnostic and easily integrates into existing pseudo-label methods. Experiments on two public cardiac MRI datasets (ACDC and MSCMRseg) across four scribble-supervised algorithms show consistent improvements in segmentation accuracy. Code will be made available on GitHub upon acceptance.
Abstract (translated)
带有潦草标注的弱监督学习使用稀疏的手绘笔画来指示少量像素的分割标签，从而减少了密集像素级标记的成本。然而，这种注释本身存在噪声和不完整指导的问题。最近基于潦草标注的方法在医学图像分割中通过伪标签训练解决了这一限制；但是，伪标签的质量仍然是性能的关键瓶颈。 我们提出了PLESS（Pseudo-Label Enhancement Strategy for Scribble Annotations），这是一种通用的伪标签增强策略，旨在提高可靠性和空间一致性。该方法建立在一个将图像划分为一系列空间连贯区域的层次结构上，并利用潦草信息在语义一致区域内细化伪标签。PLESS框架与模型无关且易于整合到现有的伪标签方法中。 实验表明，在两个公开的心脏MRI数据集（ACDC和MSCMRseg）以及四种基于潦草标注的方法上，该策略均显著提高了分割精度。论文被接受后，代码将在GitHub上发布。
URL
https://arxiv.org/abs/2602.11628
PDF
-
ArtContext: Contextualizing Artworks with Open-Access Art History Articles and Wikidata Knowledge through a LoRA-Tuned CLIP Model
Abstract
Many Art History articles discuss artworks in general as well as specific parts of works, such as layout, iconography, or material culture. However, when viewing an artwork, it is not trivial to identify what different articles have said about the piece. Therefore, we propose ArtContext, a pipeline for taking a corpus of Open-Access Art History articles and Wikidata Knowledge and annotating Artworks with this information. We do this using a novel corpus collection pipeline, then learn a bespoke CLIP model adapted using Low-Rank Adaptation (LoRA) to make it domain-specific. We show that the new model, PaintingCLIP, which is weakly supervised by the collected corpus, outperforms CLIP and provides context for a given artwork. The proposed pipeline is generalisable and can be readily applied to numerous humanities areas.
Abstract (translated)
许多艺术史文章会讨论艺术品的总体情况以及作品的具体部分，如布局、图象学或物质文化。然而，在观看一件艺术品时，很难确定不同文章对该件作品的说法。因此，我们提出了ArtContext这一流程，它能够从一个开放获取的艺术史文献语料库和维基数据知识中提取信息，并将这些信息标注到艺术作品上。为此，我们使用了一种新型的语料收集流程，然后通过低秩适应（LoRA）技术对CLIP模型进行调整，使其具有特定领域的特性。我们展示了一个新模型PaintingCLIP，在该模型被收集的语料库弱监督下，其表现优于原始的CLIP模型，并能为给定的艺术作品提供上下文信息。所提出的流程是通用化的，可以轻松应用于多个人文领域。
URL
https://arxiv.org/abs/2602.11349
PDF
-
Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidance
2026-02-11 05:44:30Shengyang Sun, Jiashen Hua, Junyi Feng, Xiaojin GongAbstract
Weakly supervised multimodal video anomaly detection has gained significant attention, yet the potential of the text modality remains under-explored. Text provides explicit semantic information that can enhance anomaly characterization and reduce false alarms. However, extracting effective text features is challenging due to the inability of general-purpose language models to capture anomaly-specific nuances and the scarcity of relevant descriptions. Furthermore, multimodal fusion often suffers from redundancy and imbalance. To address these issues, we propose a novel text-guided framework. First, we introduce an in-context learning-based multi-stage text augmentation mechanism to generate high-quality anomaly text samples for fine-tuning the text feature extractor. Second, we design a multi-scale bottleneck Transformer fusion module that uses compressed bottleneck tokens to progressively integrate information across modalities, mitigating redundancy and imbalance. Experiments on UCF-Crime and XD-Violence demonstrate state-of-the-art performance.
Abstract (translated)
弱监督多模态视频异常检测已经引起了广泛关注，然而文本模态的潜力仍然未被充分探索。文本提供了明确的语义信息，可以增强对异常行为的描述和减少误报率。但是，由于通用语言模型难以捕捉特定于异常的信息以及相关描述稀少的问题，提取有效文本特征颇具挑战性。此外，在多模态融合过程中经常会出现冗余和不平衡的问题。为了解决这些问题，我们提出了一种新颖的基于文本引导框架的方法。 首先，我们引入了一个基于上下文学习的多阶段文本增强机制，用于生成高质量的异常文本样本以对文本特征提取器进行微调。其次，我们设计了一种多尺度瓶颈Transformer融合模块，该模块使用压缩后的瓶颈令牌逐步整合不同模态的信息，从而缓解冗余和不平衡的问题。 在UCF-Crime和XD-Violence数据集上的实验结果表明了我们的方法达到了最先进的性能水平。
URL
https://arxiv.org/abs/2602.10549
PDF
-
Learning to Detect Baked Goods with Limited Supervision
2026-02-10 17:06:36Thomas H. Schmitt, Maximilian Bundscherer, Tobias BockletAbstract
Monitoring leftover products provides valuable insights that can be used to optimize future production. This is especially important for German bakeries because freshly baked goods have a very short shelf life. Automating this process can reduce labor costs, improve accuracy, and streamline operations. We propose automating this process using an object detection model to identify baked goods from images. However, the large diversity of German baked goods makes fully supervised training prohibitively expensive and limits scalability. Although open-vocabulary detectors (e.g., OWLv2, Grounding DINO) offer lexibility, we demonstrate that they are insufficient for our task. While motivated by bakeries, our work addresses the broader challenges of deploying computer vision in industries, where tasks are specialized and annotated datasets are scarce. We compile dataset splits with varying supervision levels, covering 19 classes of baked goods. We propose two training workflows to train an object detection model with limited supervision. First, we combine OWLv2 and Grounding DINO localization with image-level supervision to train the model in a weakly supervised manner. Second, we improve viewpoint robustness by fine-tuning on video frames annotated using Segment Anything 2 as a pseudo-label propagation model. Using these workflows, we train YOLOv11 for our detection task due to its favorable speed accuracy tradeoff. Relying solely on image-level supervision, the model achieves a mean Average Precision (mAP) of 0.91. Finetuning with pseudo-labels raises model performance by 19.3% under non-ideal deployment conditions. Combining these workflows trains a model that surpasses our fully-supervised baseline model under non-ideal deployment conditions, despite relying only on image-level supervision.
Abstract (translated)
监控剩余产品的状态可以提供有价值的见解，这些见解可用于优化未来的生产流程。对于德国的面包店来说，这一点尤为重要，因为新鲜烘焙的产品保质期非常短。自动化这一过程能够减少劳动力成本、提高准确性和简化操作流程。我们建议使用物体检测模型从图像中识别烘焙产品来实现这一目标。 然而，由于德国面包种类繁多，完全监督式的训练方法成本过高且不具备可扩展性。尽管开放词汇的检测器（如OWLv2和Grounding DINO）提供了灵活性，但我们证明它们对于我们的任务来说是不够的。虽然这项工作最初受到烘焙行业的启发，但它更广泛地解决了在行业中部署计算机视觉技术时面临的挑战：这些行业中的任务非常专业化且注释数据集稀缺。 为了应对这一问题，我们编制了不同监督水平的数据集拆分，涵盖了19类烘焙产品。为此，我们提出了两种有限监督训练物体检测模型的训练工作流程： 首先，我们将OWLv2和Grounding DINO定位与图像级监督相结合，以弱监督的方式训练模型。 其次，为了提高视角鲁棒性，我们在使用Segment Anything 2作为伪标签传播模型进行注释的视频帧上进行了微调。 利用这些工作流程，并考虑到YOLOv11在速度精度权衡方面的优势，我们针对检测任务对YOLOv11进行了训练。仅依赖图像级监督的情况下，该模型实现了0.91的平均精度（mAP）。通过使用伪标签进行微调，在非理想部署条件下使模型性能提高了19.3%。 结合这些工作流程，即使在只依靠图像级监督的情况下，也能训练出一个超过我们完全监督基准模型性能的模型。这种方法特别适用于解决那些标注数据稀缺、任务具有特殊性的行业的挑战。
URL
https://arxiv.org/abs/2602.09979
PDF
-
Weakly Supervised Contrastive Learning for Histopathology Patch Embeddings
2026-02-10 07:17:11Bodong Zhang, Xiwen Li, Hamid Manoochehri, Xiaoya Tang, Deepika Sirohi, Beatrice S. Knudsen, Tolga TasdizenAbstract
Digital histopathology whole slide images (WSIs) provide gigapixel-scale high-resolution images that are highly useful for disease diagnosis. However, digital histopathology image analysis faces significant challenges due to the limited training labels, since manually annotating specific regions or small patches cropped from large WSIs requires substantial time and effort. Weakly supervised multiple instance learning (MIL) offers a practical and efficient solution by requiring only bag-level (slide-level) labels, while each bag typically contains multiple instances (patches). Most MIL methods directly use frozen image patch features generated by various image encoders as inputs and primarily focus on feature aggregation. However, feature representation learning for encoder pretraining in MIL settings has largely been neglected. In our work, we propose a novel feature representation learning framework called weakly supervised contrastive learning (WeakSupCon) that incorporates bag-level label information during training. Our method does not rely on instance-level pseudo-labeling, yet it effectively separates patches with different labels in the feature space. Experimental results demonstrate that the image features generated by our WeakSupCon method lead to improved downstream MIL performance compared to self-supervised contrastive learning approaches in three datasets. Our related code is available at this http URL
Abstract (translated)
数字全滑病理图像（WSIs）提供了像素级的高分辨率图像，这对于疾病诊断非常有用。然而，由于标注训练数据量有限，数字病理图像分析面临着重大挑战，因为手动注释特定区域或从大型WSI中裁剪的小块需要大量的时间和精力。弱监督下的多实例学习（MIL）通过只需要袋子级别的标签提供了一种实用且高效的解决方案，而每个袋子通常包含多个实例（斑块）。大多数MIL方法直接使用由各种图像编码器生成的冻结图像斑块特征作为输入，并主要集中在特征聚合上。然而，在MIL环境中用于编码器预训练的特征表示学习被很大程度忽视了。 在我们的工作中，我们提出了一种名为弱监督对比学习（WeakSupCon）的新颖特征表示学习框架，该框架在训练过程中结合使用袋子级别的标签信息。我们的方法不依赖于实例级别伪标记，但在特征空间中有效地将具有不同标签的斑块分开。实验结果表明，在三个数据集中，我们提出的WeakSupCon方法生成的图像特征相比于自监督对比学习方法带来了更好的下游MIL性能改进。 相关代码可在此链接访问：[此URL]（请根据实际发布的地址替换）。
URL
https://arxiv.org/abs/2602.09477
PDF
-
GAFR-Net: A Graph Attention and Fuzzy-Rule Network for Interpretable Breast Cancer Image Classification
2026-02-10 01:25:57Lin-Guo Gao, Suxing LiuAbstract
Accurate classification of breast cancer histopathology images is pivotal for early oncological diagnosis and therapeutic this http URL, conventional deep learning architectures often encounter performance degradation under limited annotations and suffer from a "blackbox" nature, hindering their clinical integration. To mitigate these limitations, we propose GAFRNet, a robust and interpretable Graph Attention and FuzzyRule Network specifically engineered for histopathology image classification with scarce supervision. GAFRNet constructs a similarity-driven graph representation to model intersample relationships and employs a multihead graph attention mechanism to capture complex relational features across heterogeneous tissue this http URL, a differentiable fuzzy-rule module encodes intrinsic topological descriptorsincluding node degree, clustering coefficient, and label consistencyinto explicit, human-understandable diagnostic logic. This design establishes transparent "IF-THEN" mappings that mimic the heuristic deduction process of medical experts, providing clear reasoning behind each prediction without relying on post-hoc attribution methods. Extensive evaluations on three benchmark datasets (BreakHis, Mini-DDSM, and ICIAR2018) demonstrate that GAFR-Net consistently outperforms various state-of-the-art methods across multiple magnifications and classification tasks. These results validate the superior generalization and practical utility of GAFR-Net as a reliable decision-support tool for weakly supervised medical image analysis.
Abstract (translated)
乳腺癌组织病理图像的准确分类对于早期肿瘤诊断和治疗至关重要。然而，传统的深度学习架构在标注数据有限的情况下性能会下降，并且由于其“黑箱”特性而难以被临床应用。为了解决这些问题，我们提出了一种名为GAFRNet（Graph Attention and Fuzzy Rule Network）的强大且可解释的网络，专门用于稀缺监督条件下的组织病理图像分类。GAFRNet通过构建以相似性驱动的图表示来建模样本间的相互关系，并采用多头图注意力机制来捕捉异质组织中的复杂关联特征。此外，一个可微分模糊规则模块将内在拓扑描述符（包括节点度、聚类系数和标签一致性）编码为明确的人类易于理解的诊断逻辑。这种设计建立了透明的“如果-那么”映射关系，模拟了医疗专家基于直觉进行推断的过程，并且无需依赖事后归因方法就能提供清晰的推理依据。 在三个基准数据集（BreakHis、Mini-DDSM和ICIA2018）上的广泛评估表明，GAFRNet在多种放大倍率和分类任务中均超越了各种最新的方法。这些结果验证了GAFR-Net作为弱监督医学影像分析可靠决策支持工具的优越泛化能力和实用价值。 通过这种方式，GAFRNet不仅能够提高诊断准确性，还能增强医生对模型输出的理解与信任度，从而促进其在临床环境中的广泛采用。
URL
https://arxiv.org/abs/2602.09318
PDF
-
Tighnari v2: Mitigating Label Noise and Distribution Shift in Multimodal Plant Distribution Prediction via Mixture of Experts and Weakly Supervised Learning
2026-02-09 05:23:22Haixu Liu, Yufei Wang, Tianxiang Xu, Chuancheng Shi, Hongsheng XingAbstract
Large-scale, cross-species plant distribution prediction plays a crucial role in biodiversity conservation, yet modeling efforts in this area still face significant challenges due to the sparsity and bias of observational data. Presence-Absence (PA) data provide accurate and noise-free labels, but are costly to obtain and limited in quantity; Presence-Only (PO) data, by contrast, offer broad spatial coverage and rich spatiotemporal distribution, but suffer from severe label noise in negative samples. To address these real-world constraints, this paper proposes a multimodal fusion framework that fully leverages the strengths of both PA and PO data. We introduce an innovative pseudo-label aggregation strategy for PO data based on the geographic coverage of satellite imagery, enabling geographic alignment between the label space and remote sensing feature space. In terms of model architecture, we adopt Swin Transformer Base as the backbone for satellite imagery, utilize the TabM network for tabular feature extraction, retain the Temporal Swin Transformer for time-series modeling, and employ a stackable serial tri-modal cross-attention mechanism to optimize the fusion of heterogeneous modalities. Furthermore, empirical analysis reveals significant geographic distribution shifts between PA training and test samples, and models trained by directly mixing PO and PA data tend to experience performance degradation due to label noise in PO data. To address this, we draw on the mixture-of-experts paradigm: test samples are partitioned according to their spatial proximity to PA samples, and different models trained on distinct datasets are used for inference and post-processing within each partition. Experiments on the GeoLifeCLEF 2025 dataset demonstrate that our approach achieves superior predictive performance in scenarios with limited PA coverage and pronounced distribution shifts.
Abstract (translated)
大规模、跨物种的植物分布预测在生物多样性保护中起着关键作用，但在这一领域建模工作仍面临观测数据稀疏和偏差等重大挑战。存在-不存在（PA）数据提供准确且无噪声的标签，但获取成本高昂且数量有限；仅存在（PO）数据则提供了广泛的空间覆盖范围以及丰富的时间空间分布信息，但却因负样本中严重的标签噪声而受到影响。 为了解决这些实际限制，本文提出了一种多模态融合框架，该框架充分利用了PA和PO数据各自的优点。我们引入了一种基于卫星图像地理覆盖的创新伪标签聚合策略，从而实现标签空间与遥感特征空间之间的地理位置对齐。在模型架构方面，我们采用Swin Transformer Base作为卫星图像的基础网络，并使用TabM网络进行表格特征提取，保留时间序列建模中的Temporal Swin Transformer，同时利用可堆叠串行三模式交叉注意机制来优化异构模式的融合。 此外，实证分析显示PA训练和测试样本之间存在显著的地理分布偏移，直接混合PO和PA数据训练模型往往会因为PO数据中标签噪声而导致性能下降。为了应对这一问题，我们借鉴了专家混合范式：根据其与PA样本的空间接近程度对测试样本进行划分，并在每个分区使用不同数据集上训练的不同模型来进行推理和后处理。 通过GeoLifeCLEF 2025数据集的实验表明，我们的方法在PA覆盖范围有限且分布偏移显著的情况下达到了优越的预测性能。
URL
https://arxiv.org/abs/2602.08282
PDF
-
Weak to Strong: VLM-Based Pseudo-Labeling as a Weakly Supervised Training Strategy in Multimodal Video-based Hidden Emotion Understanding Tasks
2026-02-08 17:02:55Yufei Wang, Haixu Liu, Tianxiang Xu, Chuancheng Shi, Hongsheng XingAbstract
To tackle the automatic recognition of "concealed emotions" in videos, this paper proposes a multimodal weak-supervision framework and achieves state-of-the-art results on the iMiGUE tennis-interview dataset. First, YOLO 11x detects and crops human portraits frame-by-frame, and DINOv2-Base extracts visual features from the cropped regions. Next, by integrating Chain-of-Thought and Reflection prompting (CoT + Reflection), Gemini 2.5 Pro automatically generates pseudo-labels and reasoning texts that serve as weak supervision for downstream models. Subsequently, OpenPose produces 137-dimensional key-point sequences, augmented with inter-frame offset features; the usual graph neural network backbone is simplified to an MLP to efficiently model the spatiotemporal relationships of the three key-point streams. An ultra-long-sequence Transformer independently encodes both the image and key-point sequences, and their representations are concatenated with BERT-encoded interview transcripts. Each modality is first pre-trained in isolation, then fine-tuned jointly, with pseudo-labeled samples merged into the training set for further gains. Experiments demonstrate that, despite severe class imbalance, the proposed approach lifts accuracy from under 0.6 in prior work to over 0.69, establishing a new public benchmark. The study also validates that an "MLP-ified" key-point backbone can match - or even surpass - GCN-based counterparts in this task.
Abstract (translated)
为了应对视频中“隐蔽情绪”的自动识别问题，本文提出了一种多模态弱监督框架，并在iMiGUE网球访谈数据集上取得了最先进的成果。首先，YOLO 11x每帧检测并裁剪人物肖像，DINOv2-Base从裁剪区域提取视觉特征。接着，通过整合Chain-of-Thought和反思提示（CoT + Reflection），Gemini 2.5 Pro自动生成伪标签和推理文本，这些作为下游模型的弱监督信号。随后，OpenPose生成137维关键点序列，并加入帧间偏移特征；通常用于建模时空关系的关键点流图神经网络骨干被简化为MLP（多层感知机），以更高效地处理这一任务。一个超长序列Transformer独立编码图像和关键点序列，然后将它们的表示与BERT编码后的访谈记录拼接起来。每种模式首先单独预训练，之后共同微调，并将伪标签样本合并到训练集中以进一步提升性能。实验表明，在严重类不平衡的情况下，所提出的方法使准确性从先前研究中的不足0.6提高到了超过0.69的新水平，确立了一个新的公共基准标准。此外，该研究表明，“MLP化”的关键点骨干网络可以与基于GCN（图卷积网络）的同类方法相匹敌甚至超越。
URL
https://arxiv.org/abs/2602.08057
PDF
-
PANC: Prior-Aware Normalized Cut for Object Segmentation
2026-02-06 18:07:20Juan Guti\'errez, Victor Guti\'errez-Garcia, Jos\'e Luis Blanco-MurilloAbstract
Fully unsupervised segmentation pipelines naively seek the most salient object, should this be present. As a result, most of the methods reported in the literature deliver non-deterministic partitions that are sensitive to initialization, seed order, and threshold heuristics. We propose PANC, a weakly supervised spectral segmentation framework that uses a minimal set of annotated visual tokens to produce stable, controllable, and reproducible object masks. From the TokenCut approach, we augment the token-token affinity graph with a handful of priors coupled to anchor nodes. By manipulating the graph topology, we bias the spectral eigenspace toward partitions that are consistent with the annotations. Our approach preserves the global grouping enforced by dense self-supervised visual features, trading annotated tokens for significant gains in reproducibility, user control, and segmentation quality. Using 5 to 30 annotations per dataset, our training-free method achieves state-of-the-art performance among weakly and unsupervised approaches on standard benchmarks (e.g., DUTS-TE, ECSSD, MS COCO). Contrarily, it excels in domains where dense labels are costly or intra-class differences are subtle. We report strong and reliable results on homogeneous, fine-grained, and texture-limited domains, achieving 96.8% (+14.43% over SotA), 78.0% (+0.2%), and 78.8% (+0.37%) average mean intersection-over-union (mIoU) on CrackForest (CFD), CUB-200-2011, and HAM10000 datasets, respectively. For multi-object benchmarks, the framework showcases explicit, user-controllable semantic segmentation.
Abstract (translated)
完全无监督的分割流程通常会寻找最显著的对象，前提是该对象存在。因此，文献中报告的大多数方法提供了非确定性的分区结果，这些结果对初始化、种子顺序和阈值启发式非常敏感。我们提出了一种弱监督光谱分割框架PANC，它使用一组最小化的注释视觉标记来生成稳定、可控且可重复的对象掩码。从TokenCut方法出发，我们在令牌-令牌亲和图中添加了一些与锚节点耦合的先验知识。通过调整图的拓扑结构，我们偏向于让光谱特征空间倾向于那些与标注一致的分区结果。我们的方法保持了由密集自监督视觉特征强制执行的整体分组效果，并以注释标记为代价换取可重复性、用户控制和分割质量方面的显著提升。 在标准基准测试（如DUTS-TE、ECSSD、MS COCO）上，使用每个数据集5到30个标注的情况下，我们的无训练方法在弱监督与完全无监督的方法中达到了最佳性能。相反，在密集标签成本高或类内差异微妙的领域，该方法表现出色。我们在同质性、细粒度和纹理有限的领域报告了强大且可靠的结果，分别实现了CrackForest (CFD)上96.8%（比最先进方法提高14.43%），CUB-200-2011上78.0%（提高0.2%）以及HAM10000数据集上的78.8%（提高0.37%）的平均交并比（mIoU）。对于多对象基准测试，该框架展示了用户可控的语义分割。
URL
https://arxiv.org/abs/2602.06912
PDF
-
Towards Segmenting the Invisible: An End-to-End Registration and Segmentation Framework for Weakly Supervised Tumour Analysis
2026-02-05 08:55:26Budhaditya Mukhopadhyay, Chirag Mandal, Pavan Tummala, Naghmeh Mahmoodian, Andreas N\"urnberger, Soumick ChatterjeeAbstract
Liver tumour ablation presents a significant clinical challenge: whilst tumours are clearly visible on pre-operative MRI, they are often effectively invisible on intra-operative CT due to minimal contrast between pathological and healthy tissue. This work investigates the feasibility of cross-modality weak supervision for scenarios where pathology is visible in one modality (MRI) but absent in another (CT). We present a hybrid registration-segmentation framework that combines MSCGUNet for inter-modal image registration with a UNet-based segmentation module, enabling registration-assisted pseudo-label generation for CT images. Our evaluation on the CHAOS dataset demonstrates that the pipeline can successfully register and segment healthy liver anatomy, achieving a Dice score of 0.72. However, when applied to clinical data containing tumours, performance degrades substantially (Dice score of 0.16), revealing the fundamental limitations of current registration methods when the target pathology lacks corresponding visual features in the target modality. We analyse the "domain gap" and "feature absence" problems, demonstrating that whilst spatial propagation of labels via registration is feasible for visible structures, segmenting truly invisible pathology remains an open challenge. Our findings highlight that registration-based label transfer cannot compensate for the absence of discriminative features in the target modality, providing important insights for future research in cross-modality medical image analysis. Code an weights are available at: this https URL
Abstract (translated)
肝脏肿瘤消融在临床上是一个重大的挑战：尽管术前MRI图像上可以清晰地看到肿瘤，但在术中CT图像上由于病理组织和健康组织之间的对比度极低，这些肿瘤往往变得几乎不可见。这项研究探讨了跨模式弱监督方法的可行性，在这种方法中，一种成像方式（如MRI）能显示病理性结构，而另一种成像方式（如CT）则无法显示。我们提出了一种结合MSCGUNet进行跨模态图像配准与基于UNet的分割模块的混合框架，该框架能够生成注册辅助伪标签以供CT图像使用。 我们在CHAOS数据集上的评估表明，该流程可以成功地对健康肝脏解剖结构进行配准和分割，达到Dice分数为0.72的成绩。然而，在包含肿瘤的实际临床数据中应用时，性能显著下降（Dice分数为0.16），这揭示了现有注册方法在目标病理性特征缺乏相应视觉特性时的固有限制。 我们分析了“领域差距”和“特征缺失”的问题，表明尽管通过配准可以在可视结构间传播标签，但分割真正不可见的病理仍然是一个开放性的挑战。我们的研究结果强调，基于注册的标签转移不能弥补目标模态中缺乏区分性特征的问题，并为未来跨模式医学图像分析的研究提供了重要的见解。 代码和权重可在以下网址获得：[链接]（此处应插入具体URL）
URL
https://arxiv.org/abs/2602.05453
PDF
