Abstract
Whole Slide Images (WSIs) are giga-pixel in scale and are typically partitioned into small instances in WSI classification pipelines for computational feasibility. However, obtaining extensive instance level annotations is costly, making few-shot weakly supervised WSI classification (FSWC) crucial for learning from limited slide-level labels. Recently, pre-trained vision-language models (VLMs) have been adopted in FSWC, yet they exhibit several limitations. Existing prompt tuning methods in FSWC substantially increase both the number of trainable parameters and inference overhead. Moreover, current methods discard instances with low alignment to text embeddings from VLMs, potentially leading to information loss. To address these challenges, we propose two key contributions. First, we introduce a new parameter efficient prompt tuning method by scaling and shifting features in text encoder, which significantly reduces the computational cost. Second, to leverage not only the pre-trained knowledge of VLMs, but also the inherent hierarchical structure of WSIs, we introduce a WSI representation learning approach with a soft hierarchical textual guidance strategy without utilizing hard instance filtering. Comprehensive evaluations on pathology datasets covering breast, lung, and ovarian cancer types demonstrate consistent improvements up-to 10.9%, 7.8%, and 13.8% respectively, over the state-of-the-art methods in FSWC. Our method reduces the number of trainable parameters by 18.1% on both breast and lung cancer datasets, and 5.8% on the ovarian cancer dataset, while also excelling at weakly-supervised tumor localization. Code at this https URL.
Abstract (translated)
全切片图像（WSIs）具有吉像素级规模，通常需在WSI分类流程中被分割为小实例以保障计算可行性。然而，获取大量实例级标注成本高昂，使得基于有限切片级标签学习的少样本弱监督WSI分类（FSWC）至关重要。近期，预训练视觉语言模型（VLMs）被应用于FSWC，但仍存在若干局限：现有FSWC中的提示调优方法显著增加了可训练参数量和推理开销；同时，当前方法会丢弃与VLM文本嵌入对齐度低的实例，可能导致信息损失。为应对这些挑战，我们提出两项关键贡献：第一，通过缩放和偏移文本编码器中的特征，提出一种新的参数高效提示调优方法，大幅降低计算成本；第二，为同时利用VLMs的预训练知识与WSIs的内在层次结构，我们引入一种不依赖硬实例过滤的软层次文本引导WSI表征学习方法。在涵盖乳腺癌、肺癌和卵巢癌类型的病理数据集上的综合评估表明，该方法在FSWC任务中相较于最新方法分别实现高达10.9%、7.8%和13.8%的持续提升。我们的方法在乳腺癌和肺癌数据集上减少18.1%的可训练参数，在卵巢癌数据集上减少5.8%，同时在弱监督肿瘤定位任务中表现卓越。代码见此链接。
URL
https://arxiv.org/abs/2603.21504
