Weakly_Supervised
-
A Domain-Specific Curated Benchmark for Entity and Document-Level Relation Extraction
2026-02-04 08:38:55Marco Martinelli, Stefano Marchesin, Vanessa Bonato, Giorgio Maria Di Nunzio, Nicola Ferro, Ornella Irrera, Laura Menotti, Federica Vezzani, Gianmaria SilvelloAbstract
Information Extraction (IE), encompassing Named Entity Recognition (NER), Named Entity Linking (NEL), and Relation Extraction (RE), is critical for transforming the rapidly growing volume of scientific publications into structured, actionable knowledge. This need is especially evident in fast-evolving biomedical fields such as the gut-brain axis, where research investigates complex interactions between the gut microbiota and brain-related disorders. Existing biomedical IE benchmarks, however, are often narrow in scope and rely heavily on distantly supervised or automatically generated annotations, limiting their utility for advancing robust IE methods. We introduce GutBrainIE, a benchmark based on more than 1,600 PubMed abstracts, manually annotated by biomedical and terminological experts with fine-grained entities, concept-level links, and relations. While grounded in the gut-brain axis, the benchmark's rich schema, multiple tasks, and combination of highly curated and weakly supervised data make it broadly applicable to the development and evaluation of biomedical IE systems across domains.
Abstract (translated)
信息抽取（IE）包括命名实体识别（NER）、命名实体链接（NEL）和关系提取（RE），对于将快速增长的科学出版物转化为结构化、可操作的知识至关重要。特别是在快速发展的生物医学领域，如肠-脑轴，研究复杂地探讨了肠道微生物群与大脑相关疾病之间的相互作用，这种需求尤为明显。然而，现有的生物医学IE基准测试通常范围狭窄，并且严重依赖于远程监督或自动产生的注释，这限制了它们在推进稳健IE方法方面的实用性。 我们引入了GutBrainIE，这是一个基于超过1,600篇PubMed摘要的基准测试，这些摘要由生物医学和术语学专家手动标注，包含了细粒度实体、概念级链接以及关系。虽然该基准测试建立在肠-脑轴的基础上，但其丰富的架构、多项任务设置及高度策划与弱监督数据的结合，使其能够广泛应用于跨领域生物医学IE系统的开发与评估。
URL
https://arxiv.org/abs/2602.04320
PDF
-
SPWOOD: Sparse Partial Weakly-Supervised Oriented Object Detection
2026-02-03 15:21:01Wei Zhang, Xiang Liu, Ningjing Liu, Mingxin Liu, Wei Liao, Chunyan Xu, Xue YangAbstract
A consistent trend throughout the research of oriented object detection has been the pursuit of maintaining comparable performance with fewer and weaker annotations. This is particularly crucial in the remote sensing domain, where the dense object distribution and a wide variety of categories contribute to prohibitively high costs. Based on the supervision level, existing oriented object detection algorithms can be broadly grouped into fully supervised, semi-supervised, and weakly supervised methods. Within the scope of this work, we further categorize them to include sparsely supervised and partially weakly-supervised methods. To address the challenges of large-scale labeling, we introduce the first Sparse Partial Weakly-Supervised Oriented Object Detection framework, designed to efficiently leverage only a few sparse weakly-labeled data and plenty of unlabeled data. Our framework incorporates three key innovations: (1) We design a Sparse-annotation-Orientation-and-Scale-aware Student (SOS-Student) model to separate unlabeled objects from the background in a sparsely-labeled setting, and learn orientation and scale information from orientation-agnostic or scale-agnostic weak annotations. (2) We construct a novel Multi-level Pseudo-label Filtering strategy that leverages the distribution of model predictions, which is informed by the model's multi-layer predictions. (3) We propose a unique sparse partitioning approach, ensuring equal treatment for each category. Extensive experiments on the DOTA and DIOR datasets show that our framework achieves a significant performance gain over traditional oriented object detection methods mentioned above, offering a highly cost-effective solution. Our code is publicly available at this https URL.
Abstract (translated)
在整个面向对象检测的研究中，一个持续的趋势是追求在使用较少和较弱标注的情况下保持相当的性能。这在遥感领域尤为重要，因为密集的对象分布和众多类别导致了高昂的成本。根据监督水平，现有的面向对象检测算法可以大致分为全监督、半监督和弱监督方法。在此工作中，我们进一步将它们分类为稀疏监督和部分弱监督方法。为了应对大规模标注的挑战，我们引入了第一个稀疏部分弱监督面向对象检测框架，旨在仅利用少量稀疏弱标记数据和大量未标记数据来高效运行。 我们的框架包含三个关键创新： 1. 我们设计了一种稀疏注释方向尺度感知的学生模型（SOS-学生模型），在稀疏标注的情况下将未标注的对象从背景中分离出来，并且能够从无方向性或无尺度性的弱标签中学习到方向和尺度信息。 2. 我们构建了一个新颖的多级伪标签过滤策略，该策略利用了模型预测分布的信息，这些信息来源于模型多层次的预测结果。 3. 我们提出了一种独特的稀疏分区方法，确保每个类别都能得到平等对待。 在DOTA和DIOR数据集上的大量实验表明，我们的框架比上述传统的面向对象检测方法获得了显著的性能提升，并提供了高度成本效益的解决方案。我们公开提供代码：[此链接]（请在此处插入实际的URL）。
URL
https://arxiv.org/abs/2602.03634
PDF
-
Finding Optimal Video Moment without Training: Gaussian Boundary Optimization for Weakly Supervised Video Grounding
2026-02-03 04:01:12Sunoh Kim, Kimin Yun, Daeho UmAbstract
Weakly supervised temporal video grounding aims to localize query-relevant segments in untrimmed videos using only video-sentence pairs, without requiring ground-truth segment annotations that specify exact temporal boundaries. Recent approaches tackle this task by utilizing Gaussian-based temporal proposals to represent query-relevant segments. However, their inference strategies rely on heuristic mappings from Gaussian parameters to segment boundaries, resulting in suboptimal localization performance. To address this issue, we propose Gaussian Boundary Optimization (GBO), a novel inference framework that predicts segment boundaries by solving a principled optimization problem that balances proposal coverage and segment compactness. We derive a closed-form solution for this problem and rigorously analyze the optimality conditions under varying penalty regimes. Beyond its theoretical foundations, GBO offers several practical advantages: it is training-free and compatible with both single-Gaussian and mixture-based proposal architectures. Our experiments show that GBO significantly improves localization, achieving state-of-the-art results across standard benchmarks. Extensive experiments demonstrate the efficiency and generalizability of GBO across various proposal schemes. The code is available at \href{this https URL}{this https URL}.
Abstract (translated)
弱监督视频时间定位的目标是在未裁剪的视频中仅使用视频-句子对来定位与查询相关的片段，而不需依赖明确指定具体时间边界的地面真实标注。最近的研究方法通过利用基于高斯分布的时间提案来表示与查询相关的片段。然而，这些方法的推理策略依赖于从高斯参数到段落边界之间的启发式映射，这导致了次优的时间定位性能。为了解决这个问题，我们提出了高斯边界优化（GBO），这是一种新的推断框架，它通过解决一个平衡提案覆盖范围和片段紧实度的原则性优化问题来预测片段边界。我们为该问题导出了闭合形式的解决方案，并严格分析了在不同惩罚制度下的最优条件。 除了其理论基础外，GBO还提供了许多实用优势：它是无须训练且与单个高斯和混合型提案架构兼容的。我们的实验表明，GBO显著提高了定位性能，在标准基准测试中取得了最先进的结果。广泛的实验证明了GBO在各种提案方案中的效率和泛化能力。代码可在[此处](this https URL)获取。 原文的意思是介绍了一种新的弱监督视频时间定位方法——高斯边界优化（Gaussian Boundary Optimization, GBO），这种方法通过解决一个平衡提案覆盖范围与片段紧实度的原则性优化问题，提高了定位的准确性，并且无须额外训练。这项技术在标准基准测试中取得了领先的成绩，并展示了其在多种提案方案中的高效性和泛化能力。
URL
https://arxiv.org/abs/2602.03071
PDF
-
CIEC: Coupling Implicit and Explicit Cues for Multimodal Weakly Supervised Manipulation Localization
2026-02-02 14:46:38Xinquan Yu, Wei Lu, Xiangyang LuoAbstract
To mitigate the threat of misinformation, multimodal manipulation localization has garnered growing attention. Consider that current methods rely on costly and time-consuming fine-grained annotations, such as patch/token-level annotations. This paper proposes a novel framework named Coupling Implicit and Explicit Cues (CIEC), which aims to achieve multimodal weakly-supervised manipulation localization for image-text pairs utilizing only coarse-grained image/sentence-level annotations. It comprises two branches, image-based and text-based weakly-supervised localization. For the former, we devise the Textual-guidance Refine Patch Selection (TRPS) module. It integrates forgery cues from both visual and textual perspectives to lock onto suspicious regions aided by spatial priors. Followed by the background silencing and spatial contrast constraints to suppress interference from irrelevant areas. For the latter, we devise the Visual-deviation Calibrated Token Grounding (VCTG) module. It focuses on meaningful content words and leverages relative visual bias to assist token localization. Followed by the asymmetric sparse and semantic consistency constraints to mitigate label noise and ensure reliability. Extensive experiments demonstrate the effectiveness of our CIEC, yielding results comparable to fully supervised methods on several evaluation metrics.
Abstract (translated)
URL
https://arxiv.org/abs/2602.02175
PDF
-
WS-IMUBench: Can Weakly Supervised Methods from Audio, Image, and Video Be Adapted for IMU-based Temporal Action Localization?
2026-02-02 09:22:35Pei Li, Jiaxi Yin, Lei Ouyang, Shihan Pan, Ge Wang, Han Ding, Fei WangAbstract
IMU-based Human Activity Recognition (HAR) has enabled a wide range of ubiquitous computing applications, yet its dominant clip classification paradigm cannot capture the rich temporal structure of real-world behaviors. This motivates a shift toward IMU Temporal Action Localization (IMU-TAL), which predicts both action categories and their start/end times in continuous streams. However, current progress is strongly bottlenecked by the need for dense, frame-level boundary annotations, which are costly and difficult to scale. To address this bottleneck, we introduce WS-IMUBench, a systematic benchmark study of weakly supervised IMU-TAL (WS-IMU-TAL) under only sequence-level labels. Rather than proposing a new localization algorithm, we evaluate how well established weakly supervised localization paradigms from audio, image, and video transfer to IMU-TAL under only sequence-level labels. We benchmark seven representative weakly supervised methods on seven public IMU datasets, resulting in over 3,540 model training runs and 7,080 inference evaluations. Guided by three research questions on transferability, effectiveness, and insights, our findings show that (i) transfer is modality-dependent, with temporal-domain methods generally more stable than image-derived proposal-based approaches; (ii) weak supervision can be competitive on favorable datasets (e.g., with longer actions and higher-dimensional sensing); and (iii) dominant failure modes arise from short actions, temporal ambiguity, and proposal quality. Finally, we outline concrete directions for advancing WS-IMU-TAL (e.g., IMU-specific proposal generation, boundary-aware objectives, and stronger temporal reasoning). Beyond individual results, WS-IMUBench establishes a reproducible benchmarking template, datasets, protocols, and analyses, to accelerate community-wide progress toward scalable WS-IMU-TAL.
Abstract (translated)
URL
https://arxiv.org/abs/2602.01850
PDF
-
Offline Discovery of Interpretable Skills from Multi-Task Trajectories
2026-02-01 05:03:58Chongyu Zhu, Mithun Vanniasinghe, Jiayu Chen, Chi-Guhn LeeAbstract
Hierarchical Imitation Learning is a powerful paradigm for acquiring complex robot behaviors from demonstrations. A central challenge, however, lies in discovering reusable skills from long-horizon, multi-task offline data, especially when the data lacks explicit rewards or subtask annotations. In this work, we introduce LOKI, a three-stage end-to-end learning framework designed for offline skill discovery and hierarchical imitation. The framework commences with a two-stage, weakly supervised skill discovery process: Stage one performs coarse, task-aware macro-segmentation by employing an alignment-enforced Vector Quantized VAE guided by weak task labels. Stage two then refines these segments at a micro-level using a self-supervised sequential model, followed by an iterative clustering process to consolidate skill boundaries. The third stage then leverages these precise boundaries to construct a hierarchical policy within an option-based framework-complete with a learned termination condition beta for explicit skill switching. LOKI achieves high success rates on the challenging D4RL Kitchen benchmark and outperforms standard HIL baselines. Furthermore, we demonstrate that the discovered skills are semantically meaningful, aligning with human intuition, and exhibit compositionality by successfully sequencing them to solve a novel, unseen task.
Abstract (translated)
URL
https://arxiv.org/abs/2602.01018
PDF
-
Fire on Motion: Optimizing Video Pass-bands for Efficient Spiking Action Recognition
2026-01-30 07:47:23Shuhan Ye, Yuanbin Qian, Yi Yu, Chong Wang, Yuqi Xie, Jiazhen Xu, Kun Wang, Xudong JiangAbstract
Spiking neural networks (SNNs) have gained traction in vision due to their energy efficiency, bio-plausibility, and inherent temporal processing. Yet, despite this temporal capacity, most progress concentrates on static image benchmarks, and SNNs still underperform on dynamic video tasks compared to artificial neural networks (ANNs). In this work, we diagnose a fundamental pass-band mismatch: Standard spiking dynamics behave as a temporal low pass that emphasizes static content while attenuating motion bearing bands, where task relevant information concentrates in dynamic tasks. This phenomenon explains why SNNs can approach ANNs on static tasks yet fall behind on tasks that demand richer temporal this http URL remedy this, we propose the Pass-Bands Optimizer (PBO), a plug-and-play module that optimizes the temporal pass-band toward task-relevant motion bands. PBO introduces only two learnable parameters, and a lightweight consistency constraint that preserves semantics and boundaries, incurring negligible computational overhead and requires no architectural changes. PBO deliberately suppresses static components that contribute little to discrimination, effectively high passing the stream so that spiking activity concentrates on motion bearing content. On UCF101, PBO yields over ten percentage points improvement. On more complex multi-modal action recognition and weakly supervised video anomaly detection, PBO delivers consistent and significant gains, offering a new perspective for SNN based video processing and understanding.
Abstract (translated)
URL
https://arxiv.org/abs/2601.22675
PDF
-
Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization
2026-01-29 09:35:27Midou Guo, Qilin Yin, Wei Lu, Xiangyang Luo, Rui YangAbstract
Modern deepfakes have evolved into localized and intermittent manipulations that require fine-grained temporal localization. The prohibitive cost of frame-level annotation makes weakly supervised methods a practical necessity, which rely only on video-level labels. To this end, we propose Reconstruction-based Temporal Deepfake Localization (RT-DeepLoc), a weakly supervised temporal forgery localization framework that identifies forgeries via reconstruction errors. Our framework uses a Masked Autoencoder (MAE) trained exclusively on authentic data to learn its intrinsic spatiotemporal patterns; this allows the model to produce significant reconstruction discrepancies for forged segments, effectively providing the missing fine-grained cues for localization. To robustly leverage these indicators, we introduce a novel Asymmetric Intra-video Contrastive Loss (AICL). By focusing on the compactness of authentic features guided by these reconstruction cues, AICL establishes a stable decision boundary that enhances local discrimination while preserving generalization to unseen forgeries. Extensive experiments on large-scale datasets, including LAV-DF, demonstrate that RT-DeepLoc achieves state-of-the-art performance in weakly-supervised temporal forgery localization.
Abstract (translated)
URL
https://arxiv.org/abs/2601.21458
PDF
-
Learning From a Steady Hand: A Weakly Supervised Agent for Robot Assistance under Microscopy
2026-01-28 17:03:43Huanyu Tian, Martin Huber, Lingyun Zeng, Zhe Han, Wayne Bennett, Giuseppe Silvestri, Gerardo Mendizabal-Ruiz, Tom Vercauteren, Alejandro Chavez-Badiola, Christos BergelesAbstract
This paper rethinks steady-hand robotic manipulation by using a weakly supervised framework that fuses calibration-aware perception with admittance control. Unlike conventional automation that relies on labor-intensive 2D labeling, our framework leverages reusable warm-up trajectories to extract implicit spatial information, thereby achieving calibration-aware, depth-resolved perception without the need for external fiducials or manual depth annotation. By explicitly characterizing residuals from observation and calibration models, the system establishes a task-space error budget from recorded warm-ups. The uncertainty budget yields a lateral closed-loop accuracy of approx. 49 micrometers at 95% confidence (worst-case testing subset) and a depth accuracy of <= 291 micrometers at 95% confidence bound during large in-plane moves. In a within-subject user study (N=8), the learned agent reduces overall NASA-TLX workload by 77.1% relative to the simple steady-hand assistance baseline. These results demonstrate that the weakly supervised agent improves the reliability of microscope-guided biomedical micromanipulation without introducing complex setup requirements, offering a practical framework for microscope-guided intervention.
Abstract (translated)
URL
https://arxiv.org/abs/2601.20776
PDF
-
Weakly supervised framework for wildlife detection and counting in challenging Arctic environments: a case study on caribou
2026-01-26 19:02:18Ghazaleh Serati, Samuel Foucher, Jerome TheauAbstract
Caribou across the Arctic has declined in recent decades, motivating scalable and accurate monitoring approaches to guide evidence-based conservation actions and policy decisions. Manual interpretation from this imagery is labor-intensive and error-prone, underscoring the need for automatic and reliable detection across varying scenes. Yet, such automatic detection is challenging due to severe background heterogeneity, dominant empty terrain (class imbalance), small or occluded targets, and wide variation in density and scale. To make the detection model (HerdNet) more robust to these challenges, a weakly supervised patch-level pretraining based on a detection network's architecture is proposed. The detection dataset includes five caribou herds distributed across Alaska. By learning from empty vs. non-empty labels in this dataset, the approach produces early weakly supervised knowledge for enhanced detection compared to HerdNet, which is initialized from generic weights. Accordingly, the patch-based pretrain network attained high accuracy on multi-herd imagery (2017) and on an independent year's (2019) test sets (F1: 93.7%/92.6%, respectively), enabling reliable mapping of regions containing animals to facilitate manual counting on large aerial imagery. Transferred to detection, initialization from weakly supervised pretraining yielded consistent gains over ImageNet weights on both positive patches (F1: 92.6%/93.5% vs. 89.3%/88.6%), and full-image counting (F1: 95.5%/93.3% vs. 91.5%/90.4%). Remaining limitations are false positives from animal-like background clutter and false negatives related to low animal density occlusions. Overall, pretraining on coarse labels prior to detection makes it possible to rely on weakly-supervised pretrained weights even when labeled data are limited, achieving results comparable to generic-weight initialization.
Abstract (translated)
URL
https://arxiv.org/abs/2601.18891
PDF
-
Context Patch Fusion With Class Token Enhancement for Weakly Supervised Semantic Segmentation
2026-01-21 07:12:23Yiyang Fu, Hui Li, Wangyu WuAbstract
Weakly Supervised Semantic Segmentation (WSSS), which relies only on image-level labels, has attracted significant attention for its cost-effectiveness and scalability. Existing methods mainly enhance inter-class distinctions and employ data augmentation to mitigate semantic ambiguity and reduce spurious activations. However, they often neglect the complex contextual dependencies among image patches, resulting in incomplete local representations and limited segmentation accuracy. To address these issues, we propose the Context Patch Fusion with Class Token Enhancement (CPF-CTE) framework, which exploits contextual relations among patches to enrich feature representations and improve segmentation. At its core, the Contextual-Fusion Bidirectional Long Short-Term Memory (CF-BiLSTM) module captures spatial dependencies between patches and enables bidirectional information flow, yielding a more comprehensive understanding of spatial correlations. This strengthens feature learning and segmentation robustness. Moreover, we introduce learnable class tokens that dynamically encode and refine class-specific semantics, enhancing discriminative capability. By effectively integrating spatial and semantic cues, CPF-CTE produces richer and more accurate representations of image content. Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 validate that CPF-CTE consistently surpasses prior WSSS methods.
Abstract (translated)
弱监督语义分割（WSSS）依赖于图像级别的标签，因其成本效益和可扩展性而引起了广泛关注。现有的方法主要通过增强类间区分度并采用数据增强技术来减轻语义模糊性和减少虚假激活。然而，它们往往忽视了图块之间的复杂上下文依赖关系，导致局部表示不完整且分割精度受限。为解决这些问题，我们提出了带有类别标记增强（CPF-CTE）的上下文图块融合框架，该框架利用图块之间的上下文关系来丰富特征表示并提高分割性能。在核心部分，上下文融合双向长短时记忆模块（Contextual-Fusion BiLSTM, CF-BiLSTM）捕获图块间的空间依赖性，并实现双向信息流动，从而对空间相关性提供更全面的理解。这增强了特征学习和分割的鲁棒性。此外，我们引入了可学习的类别标记，这些标记能够动态编码并细化特定类别的语义信息，提高了判别能力。通过有效整合空间和语义线索，CPF-CTE生成了更加丰富且准确的图像内容表示。 在PASCAL VOC 2012及MS COCO 2014数据集上的大量实验验证表明，CPF-CTE方法始终优于先前的WSSS方法。
URL
https://arxiv.org/abs/2601.14718
PDF
-
Route, Retrieve, Reflect, Repair: Self-Improving Agentic Framework for Visual Detection and Linguistic Reasoning in Medical Imaging
2026-01-13 03:44:06Md. Faiyaz Abdullah Sayeedi, Rashedur Rahman, Siam Tahsin Bhuiyan, Sefatul Wasi, Ashraful Islam, Saadia Binte Alam, AKM Mahbubur RahmanAbstract
Medical image analysis increasingly relies on large vision-language models (VLMs), yet most systems remain single-pass black boxes that offer limited control over reasoning, safety, and spatial grounding. We propose R^4, an agentic framework that decomposes medical imaging workflows into four coordinated agents: a Router that configures task- and specialization-aware prompts from the image, patient history, and metadata; a Retriever that uses exemplar memory and pass@k sampling to jointly generate free-text reports and bounding boxes; a Reflector that critiques each draft-box pair for key clinical error modes (negation, laterality, unsupported claims, contradictions, missing findings, and localization errors); and a Repairer that iteratively revises both narrative and spatial outputs under targeted constraints while curating high-quality exemplars for future cases. Instantiated on chest X-ray analysis with multiple modern VLM backbones and evaluated on report generation and weakly supervised detection, R^4 consistently boosts LLM-as-a-Judge scores by roughly +1.7-+2.5 points and mAP50 by +2.5-+3.5 absolute points over strong single-VLM baselines, without any gradient-based fine-tuning. These results show that agentic routing, reflection, and repair can turn strong but brittle VLMs into more reliable and better grounded tools for clinical image interpretation. Our code can be found at: this https URL
Abstract (translated)
医学图像分析越来越依赖于大型视觉语言模型（VLM），然而，大多数系统仍然是一次性黑盒操作，对推理、安全和空间定位的控制有限。我们提出了R^4框架，这是一个代理框架，将医学成像工作流程分解为四个协调的代理：一个路由器，根据从图像、患者历史记录和元数据中提取的任务和专业信息来配置提示；一个检索器，使用范例记忆和pass@k抽样同时生成自由文本报告和边界框；一个反思者，批评每对草案-边框的关键临床错误模式（否定、侧向性、缺乏支持的主张、矛盾、遗漏发现以及定位错误）；以及一个修复者，在有针对性的约束下迭代修订叙述性和空间输出的同时为未来案例策划高质量范例。在胸部X光片分析中使用多个现代VLM后端，并通过报告生成和弱监督检测进行评估，R^4 在不进行任何基于梯度的微调的情况下，相对于强大的单一VLM基线，显著提高了LLM作为裁判（LLM-as-a-Judge）评分约1.7至2.5分，并将mAP50绝对提高了2.5至3.5个点。这些结果表明，代理路由、反思和修复可以将强大但脆弱的VLM转变为更可靠且更好地定位的临床图像解释工具。我们的代码可在以下链接找到：[此URL]
URL
https://arxiv.org/abs/2601.08192
PDF
-
Kidney Cancer Detection Using 3D-Based Latent Diffusion Models
2026-01-09 15:30:00Jen Dusseljee, Sarah de Boer, Alessa HeringAbstract
In this work, we present a novel latent diffusion-based pipeline for 3D kidney anomaly detection on contrast-enhanced abdominal CT. The method combines Denoising Diffusion Probabilistic Models (DDPMs), Denoising Diffusion Implicit Models (DDIMs), and Vector-Quantized Generative Adversarial Networks (VQ-GANs). Unlike prior slice-wise approaches, our method operates directly on an image volume and leverages weak supervision with only case-level pseudo-labels. We benchmark our approach against state-of-the-art supervised segmentation and detection models. This study demonstrates the feasibility and promise of 3D latent diffusion for weakly supervised anomaly detection. While the current results do not yet match supervised baselines, they reveal key directions for improving reconstruction fidelity and lesion localization. Our findings provide an important step toward annotation-efficient, generative modeling of complex abdominal anatomy.
Abstract (translated)
在这项工作中，我们提出了一种新颖的基于潜在扩散的管道，用于在增强腹部CT图像上进行3D肾脏异常检测。该方法结合了去噪扩散概率模型（DDPMs）、去噪扩散隐式模型（DDIMs）和向量量化生成对抗网络（VQ-GANs）。与之前的逐片处理方法不同，我们的方法直接针对整个图像体积进行操作，并利用仅基于病例级别的伪标签的弱监督。我们将该方法与最先进的有监督分割和检测模型进行了基准测试。这项研究表明了在弱监督下使用3D潜在扩散技术进行异常检测的可能性和前景。尽管目前的结果尚未达到有监督基线的标准，但它们揭示了提高重建准确性和病灶定位的关键方向。我们的研究为高效注释、复杂腹部解剖结构的生成建模提供了一个重要的步骤。
URL
https://arxiv.org/abs/2601.05852
PDF
-
SAPL: Semantic-Agnostic Prompt Learning in CLIP for Weakly Supervised Image Manipulation Localization
2026-01-09 07:25:55Xinghao Wang, Changtao Miao, Dianmo Sheng, Tao Gong, Qi Chu, Nenghai Yu, Quanchen Zou, Deyue Zhang, Xiangzheng ZhangAbstract
Malicious image manipulation threatens public safety and requires efficient localization methods. Existing approaches depend on costly pixel-level annotations which make training expensive. Existing weakly supervised methods rely only on image-level binary labels and focus on global classification, often overlooking local edge cues that are critical for precise localization. We observe that feature variations at manipulated boundaries are substantially larger than in interior regions. To address this gap, we propose Semantic-Agnostic Prompt Learning (SAPL) in CLIP, which learns text prompts that intentionally encode non-semantic, boundary-centric cues so that CLIPs multimodal similarity highlights manipulation edges rather than high-level object semantics. SAPL combines two complementary modules Edge-aware Contextual Prompt Learning (ECPL) and Hierarchical Edge Contrastive Learning (HECL) to exploit edge information in both textual and visual spaces. The proposed ECPL leverages edge-enhanced image features to generate learnable textual prompts via an attention mechanism, embedding semantic-irrelevant information into text features, to guide CLIP focusing on manipulation edges. The proposed HECL extract genuine and manipulated edge patches, and utilize contrastive learning to boost the discrimination between genuine edge patches and manipulated edge patches. Finally, we predict the manipulated regions from the similarity map after processing. Extensive experiments on multiple public benchmarks demonstrate that SAPL significantly outperforms existing approaches, achieving state-of-the-art localization performance.
Abstract (translated)
恶意图像篡改威胁到公共安全，需要高效的定位方法。现有方法依赖于昂贵的像素级标注，使得训练成本高昂。现有的弱监督方法仅依靠图级别二进制标签，并专注于全局分类，往往忽视了局部边缘线索对于精确定位至关重要的作用。我们观察到，在被篡改边界上的特征变化远大于内部区域的变化。为解决这一问题，我们在CLIP模型中提出了语义不可知提示学习（SAPL），该方法通过有意编码非语义的边界中心线索来学习文本提示，使CLIP的多模态相似性更能突出篡改边缘而不是高层次的对象语义。SAPL结合了两个互补模块：边缘感知上下文提示学习（ECPL）和分层边缘对比学习（HECL），用于在文本空间和视觉空间中利用边缘信息。 所提出的ECPL通过增强图像特征的边缘，生成可学习的文字提示，并借助注意力机制将语义无关的信息嵌入到文字特征中，引导CLIP关注篡改边缘。而所提出的HECL则提取真实边界片和被篡改边界片，并利用对比学习来提高对真实边界片与被篡改边界片之间的区分能力。 最后，在处理完相似性地图后，我们预测出被篡改的区域。在多个公开基准上的广泛实验表明，SAPL显著优于现有方法，实现了最先进的定位性能。
URL
https://arxiv.org/abs/2601.06222
PDF
-
PrismVAU: Prompt-Refined Inference System for Multimodal Video Anomaly Understanding
2026-01-06 11:11:06I\~naki Erregue, Kamal Nasrollahi, Sergio EscaleraAbstract
Video Anomaly Understanding (VAU) extends traditional Video Anomaly Detection (VAD) by not only localizing anomalies but also describing and reasoning about their context. Existing VAU approaches often rely on fine-tuned multimodal large language models (MLLMs) or external modules such as video captioners, which introduce costly annotations, complex training pipelines, and high inference overhead. In this work, we introduce PrismVAU, a lightweight yet effective system for real-time VAU that leverages a single off-the-shelf MLLM for anomaly scoring, explanation, and prompt optimization. PrismVAU operates in two complementary stages: (1) a coarse anomaly scoring module that computes frame-level anomaly scores via similarity to textual anchors, and (2) an MLLM-based refinement module that contextualizes anomalies through system and user prompts. Both textual anchors and prompts are optimized with a weakly supervised Automatic Prompt Engineering (APE) framework. Extensive experiments on standard VAD benchmarks demonstrate that PrismVAU delivers competitive detection performance and interpretable anomaly explanations -- without relying on instruction tuning, frame-level annotations, and external modules or dense processing -- making it an efficient and practical solution for real-world applications.
Abstract (translated)
视频异常理解（Video Anomaly Understanding，简称VAU）在传统的视频异常检测（Video Anomaly Detection，简称VAD）基础上更进一步，不仅定位了异常事件，还描述并解释了这些异常的上下文。现有的VAU方法往往依赖于微调过的多模态大规模语言模型（Multimodal Large Language Models，简称MLLMs）或外部模块如视频字幕生成器，这引入了昂贵的数据标注、复杂的训练流程和高推理成本。 在此项工作中，我们提出了PrismVAU，这是一个轻量级且高效的系统，适用于实时的VAU任务。该系统利用单一现成的多模态大规模语言模型（MLLM）来完成异常评分、解释以及提示优化。PrismVAU通过两个互补阶段实现其功能：(1) 一个粗略的异常评分模块，它计算基于与文本锚点相似度的帧级异常分数；(2) 一个基于多模态大规模语言模型（MLLM）的细化模块，该模块通过系统和用户提示来上下文化异常。这些文本锚点和提示都经过弱监督自动提示工程（Automatic Prompt Engineering, APE）框架进行优化。 在标准VAD基准上的广泛实验表明，PrismVAU能够提供竞争力强的检测性能以及可解释性的异常解释——并且无需依赖指令调优、帧级标注或外部模块及密集处理流程。这使得PrismVAU成为实际应用中一个高效且实用的解决方案。
URL
https://arxiv.org/abs/2601.02927
PDF
-
HeadLighter: Disentangling Illumination in Generative 3D Gaussian Heads via Lightstage Captures
2026-01-05 13:32:37Yating Wang, Yuan Sun, Xuan Wang, Ran Yi, Boyao Zhou, Yipengjing Sun, Hongyu Liu, Yinuo Wang, Lizhuang MaAbstract
Recent 3D-aware head generative models based on 3D Gaussian Splatting achieve real-time, photorealistic and view-consistent head synthesis. However, a fundamental limitation persists: the deep entanglement of illumination and intrinsic appearance prevents controllable relighting. Existing disentanglement methods rely on strong assumptions to enable weakly supervised learning, which restricts their capacity for complex illumination. To address this challenge, we introduce HeadLighter, a novel supervised framework that learns a physically plausible decomposition of appearance and illumination in head generative models. Specifically, we design a dual-branch architecture that separately models lighting-invariant head attributes and physically grounded rendering components. A progressive disentanglement training is employed to gradually inject head appearance priors into the generative architecture, supervised by multi-view images captured under controlled light conditions with a light stage setup. We further introduce a distillation strategy to generate high-quality normals for realistic rendering. Experiments demonstrate that our method preserves high-quality generation and real-time rendering, while simultaneously supporting explicit lighting and viewpoint editing. We will publicly release our code and dataset.
Abstract (translated)
近期基于3D高斯点阵的三维感知头部生成模型实现了实时、逼真且视角一致的头部合成。然而，这些模型仍存在一个基本限制：光照与内在外观之间的深度纠缠阻碍了可控重光照的能力。现有的脱耦方法依赖于强烈的假设来实现弱监督学习，这限制了它们处理复杂照明场景的能力。 为了应对这一挑战，我们引入了一种名为HeadLighter的新框架，该框架在头部生成模型中学习了一个物理上合理的外观和光照分解。具体来说，我们设计了一种双分支架构，分别对光照不变的头部属性和基于物理原理的渲染组件进行建模。通过使用多视角图像（这些图像是在具有灯光舞台设置的受控照明条件下捕获的）监督下逐步注入头部外观先验，采用渐进式脱耦训练方法。 此外，我们还引入了一种蒸馏策略来生成高质量的法线图以支持逼真的渲染。实验结果表明，我们的方法在保持高质量生成和实时渲染的同时，也能够支持明确的光照编辑和视角调整。我们将公开发布代码和数据集。
URL
https://arxiv.org/abs/2601.02103
PDF
-
LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum
2026-01-04 22:42:20Zhichao Xu, Shengyao Zhuang, Crystina Zhang, Xueguang Ma, Yijun Tian, Maitrey Mehta, Jimmy Lin, Vivek SrikumarAbstract
While dense retrieval models have become the standard for state-of-the-art information retrieval, their deployment is often constrained by high memory requirements and reliance on GPU accelerators for vector similarity search. Learned sparse retrieval offers a compelling alternative by enabling efficient search via inverted indices, yet it has historically received less attention than dense approaches. In this report, we introduce LACONIC, a family of learned sparse retrievers based on the Llama-3 architecture (1B, 3B, and 8B). We propose a streamlined two-phase training curriculum consisting of (1) weakly supervised pre-finetuning to adapt causal LLMs for bidirectional contextualization and (2) high-signal finetuning using curated hard negatives. Our results demonstrate that LACONIC effectively bridges the performance gap with dense models: the 8B variant achieves a state-of-the-art 60.2 nDCG on the MTEB Retrieval benchmark, ranking 15th on the leaderboard as of January 1, 2026, while utilizing 71\% less index memory than an equivalent dense model. By delivering high retrieval effectiveness on commodity CPU hardware with a fraction of the compute budget required by competing models, LACONIC provides a scalable and efficient solution for real-world search applications.
Abstract (translated)
虽然密集检索模型已成为最先进的信息检索标准，但它们的部署通常受到高内存需求和对GPU加速器进行向量相似度搜索依赖性的限制。相比之下，通过逆向索引实现高效搜索的学习稀疏检索提供了一种有吸引力的替代方案，尽管在过去它比密集方法得到了较少的关注。 在本报告中，我们介绍了LACONIC，这是一个基于Llama-3架构（1B、3B和8B）的系列学习稀疏检索器。我们提出了一项简化的两阶段训练课程：首先通过弱监督预微调将因果LLM适应为双向上下文化；其次使用精心挑选的硬负样本进行高信号微调。我们的结果显示，LACONIC有效地缩小了与密集模型之间的性能差距：其8B变体在MTEB检索基准上实现了60.2 nDCG的最佳状态，截至2026年1月1日，在排行榜上排名第15位，并且相比等效的密集模型减少了71%的索引内存使用量。通过仅使用竞争模型所需计算预算的一小部分，LACONIC就能在普通CPU硬件上实现高检索效率，从而为现实世界的搜索应用提供了可扩展和高效的解决方案。
URL
https://arxiv.org/abs/2601.01684
PDF
-
A Dual-Branch Local-Global Framework for Cross-Resolution Land Cover Mapping
2025-12-23 02:32:02Peng Gao, Ke Li, Di Wang, Yongshan Zhu, Yiming Zhang, Xuemei Luo, Yifeng WangAbstract
Cross-resolution land cover mapping aims to produce high-resolution semantic predictions from coarse or low-resolution supervision, yet the severe resolution mismatch makes effective learning highly challenging. Existing weakly supervised approaches often struggle to align fine-grained spatial structures with coarse labels, leading to noisy supervision and degraded mapping accuracy. To tackle this problem, we propose DDTM, a dual-branch weakly supervised framework that explicitly decouples local semantic refinement from global contextual reasoning. Specifically, DDTM introduces a diffusion-based branch to progressively refine fine-scale local semantics under coarse supervision, while a transformer-based branch enforces long-range contextual consistency across large spatial extents. In addition, we design a pseudo-label confidence evaluation module to mitigate noise induced by cross-resolution inconsistencies and to selectively exploit reliable supervisory signals. Extensive experiments demonstrate that DDTM establishes a new state-of-the-art on the Chesapeake Bay benchmark, achieving 66.52\% mIoU and substantially outperforming prior weakly supervised methods. The code is available at this https URL.
Abstract (translated)
跨分辨率地表覆盖制图旨在从粗略或低分辨率监督中生成高分辨率的语义预测，但由于严重的分辨率不匹配，有效的学习变得极具挑战性。现有的弱监督方法往往难以将细粒度的空间结构与粗糙标签对齐，导致嘈杂的监督信息和映射精度下降。为了解决这个问题，我们提出了DDTM，这是一种双分支弱监督框架，明确地解耦了局部语义细化和全局上下文推理。 具体而言，DDTM引入了一个基于扩散的方法分支，在粗略监督下逐步精炼细尺度的局部语义，而一个基于变压器的方法分支则强制在整个大空间范围内保持长期的上下文一致性。此外，我们设计了一种伪标签置信度评估模块来减轻由跨分辨率不一致引起的噪声，并选择性地利用可靠的监督信号。 广泛的实验表明，DDTM在切萨皮克湾基准测试上建立了新的最先进的性能，达到了66.52%的mIoU（平均交并比），远远优于以前的弱监督方法。代码可在提供的链接获取。
URL
https://arxiv.org/abs/2512.19990
PDF
-
HookMIL: Revisiting Context Modeling in Multiple Instance Learning for Computational Pathology
2025-12-20 09:14:10Xitong Ling, Minxi Ouyang, Xiaoxiao Li, Jiawen Li, Ying Chen, Yuxuan Sun, Xinrui Chen, Tian Guan, Xiaoping Liu, Yonghong HeAbstract
Multiple Instance Learning (MIL) has enabled weakly supervised analysis of whole-slide images (WSIs) in computational pathology. However, traditional MIL approaches often lose crucial contextual information, while transformer-based variants, though more expressive, suffer from quadratic complexity and redundant computations. To address these limitations, we propose HookMIL, a context-aware and computationally efficient MIL framework that leverages compact, learnable hook tokens for structured contextual aggregation. These tokens can be initialized from (i) key-patch visual features, (ii) text embeddings from vision-language pathology models, and (iii) spatially grounded features from spatial transcriptomics-vision models. This multimodal initialization enables Hook Tokens to incorporate rich textual and spatial priors, accelerating convergence and enhancing representation quality. During training, Hook tokens interact with instances through bidirectional attention with linear complexity. To further promote specialization, we introduce a Hook Diversity Loss that encourages each token to focus on distinct histopathological patterns. Additionally, a hook-to-hook communication mechanism refines contextual interactions while minimizing redundancy. Extensive experiments on four public pathology datasets demonstrate that HookMIL achieves state-of-the-art performance, with improved computational efficiency and interpretability. Codes are available at this https URL.
Abstract (translated)
多实例学习（Multiple Instance Learning，MIL）已经在计算病理学中的全滑片图像（Whole-Slide Images，WSIs）的弱监督分析中发挥了重要作用。然而，传统的MIL方法常常会丢失关键的上下文信息，而基于Transformer的方法虽然更加表达丰富，但面临着二次方复杂度和冗余计算的问题。为了克服这些限制，我们提出了一种名为HookMIL的框架，这是一种具有上下文感知能力和计算效率的多实例学习框架。该框架利用紧凑且可学习的hook令牌进行结构化上下文聚合。 这些钩子令牌可以从以下三种方式之一初始化：（i）关键补丁视觉特征；（ii）来自视觉-语言病理学模型的文本嵌入；以及（iii）空间转录组学-视觉模型的空间定位特征。这种多模态初始化使Hook Tokens能够整合丰富的文本和空间先验信息，从而加速收敛并提高表示质量。 在训练过程中，Hook令牌通过线性复杂度的双向注意力与实例进行互动。为进一步促进专业化，我们引入了一种钩子多样性损失（Hook Diversity Loss），鼓励每个钩子令牌专注于不同的组织病理学模式。此外，一个hook-to-hook通信机制细化了上下文交互，并减少了冗余。 在四个公开的病理数据集上进行了广泛的实验，结果表明HookMIL达到了最先进的性能，在计算效率和可解释性方面都有所提升。代码可在上述链接获得。
URL
https://arxiv.org/abs/2512.22188
PDF
-
SynthSeg-Agents: Multi-Agent Synthetic Data Generation for Zero-Shot Weakly Supervised Semantic Segmentation
2025-12-17 10:58:38Wangyu Wu, Zhenhong Chen, Xiaowei Huang, Fei Ma, Jimin XiaoAbstract
Weakly Supervised Semantic Segmentation (WSSS) with image level labels aims to produce pixel level predictions without requiring dense annotations. While recent approaches have leveraged generative models to augment existing data, they remain dependent on real world training samples. In this paper, we introduce a novel direction, Zero Shot Weakly Supervised Semantic Segmentation (ZSWSSS), and propose SynthSeg Agents, a multi agent framework driven by Large Language Models (LLMs) to generate synthetic training data entirely without real images. SynthSeg Agents comprises two key modules, a Self Refine Prompt Agent and an Image Generation Agent. The Self Refine Prompt Agent autonomously crafts diverse and semantically rich image prompts via iterative refinement, memory mechanisms, and prompt space exploration, guided by CLIP based similarity and nearest neighbor diversity filtering. These prompts are then passed to the Image Generation Agent, which leverages Vision Language Models (VLMs) to synthesize candidate images. A frozen CLIP scoring model is employed to select high quality samples, and a ViT based classifier is further trained to relabel the entire synthetic dataset with improved semantic precision. Our framework produces high quality training data without any real image supervision. Experiments on PASCAL VOC 2012 and COCO 2014 show that SynthSeg Agents achieves competitive performance without using real training images. This highlights the potential of LLM driven agents in enabling cost efficient and scalable semantic segmentation.
Abstract (translated)
弱监督语义分割（WSSS）使用图像级标签的目标是无需密集注释就能生成像素级别的预测。尽管最近的研究已经利用了生成模型来增强现有数据，但它们仍然依赖于真实世界的训练样本。在本文中，我们提出了一种新颖的方向——零样本弱监督语义分割（ZSWSSS），并提出了SynthSeg Agents，这是一个由大型语言模型（LLMs）驱动的多代理框架，用于完全不使用真实图像生成合成训练数据。SynthSeg Agents 包括两个关键模块：自我优化提示代理和图像生成代理。 **自我优化提示代理**自主地通过迭代细化、记忆机制以及基于CLIP相似性和最近邻多样性过滤的提示空间探索来创建多样且语义丰富的图像提示。 这些提示随后传递给**图像生成代理**，该代理利用视觉语言模型（VLMs）合成候选图像。一个冻结的CLIP评分模型被用来选择高质量样本，并通过训练一个基于ViT的分类器进一步为整个合成数据集重新标记，以提高语义精度。 我们的框架能够在没有任何真实图像监督的情况下产生高质量的训练数据。在PASCAL VOC 2012和COCO 2014上的实验表明，SynthSeg Agents 在不使用任何实际训练图像的情况下也达到了竞争性的性能水平。这突显了LLM驱动代理在实现低成本且可扩展语义分割中的潜力。
URL
https://arxiv.org/abs/2512.15310
PDF
