Abstract
With the rapid advancement of pathology foundation models (FMs), the representation learning of whole slide images (WSIs) attracts increasing attention. Existing studies develop high-quality patch feature extractors and employ carefully designed aggregation schemes to derive slide-level representations. However, mainstream weakly supervised slide representation learning methods, primarily based on multiple instance learning (MIL), are tailored to specific downstream tasks, which limits their generalizability. To address this issue, some studies explore unsupervised slide representation learning. However, these approaches focus solely on the visual modality of patches, neglecting the rich semantic information embedded in textual data. In this work, we propose ProAlign, a cross-modal unsupervised slide representation learning framework. Specifically, we leverage a large language model (LLM) to generate descriptive text for the prototype types present in a WSI, introducing patch-text contrast to construct initial prototype embeddings. Furthermore, we propose a parameter-free attention aggregation strategy that utilizes the similarity between patches and these prototypes to form unsupervised slide embeddings applicable to a wide range of downstream tasks. Extensive experiments on four public datasets show that ProAlign outperforms existing unsupervised frameworks and achieves performance comparable to some weakly supervised models.
Abstract (translated)
随着病理基础模型（FMs）的迅速发展，全滑片图像（WSIs）的表示学习引起了越来越多的关注。现有研究开发了高质量的补丁特征提取器，并采用了精心设计的聚合方案以推导出样本级别的表示。然而，主流的弱监督下的滑片表示学习方法主要基于多实例学习（MIL），这些方法是为特定下游任务量身定制的，这限制了它们的泛化能力。为了解决这个问题，一些研究探索了无监督的滑片表示学习方法。但是，这些方法仅关注补丁的视觉模式，忽略了嵌入在文本数据中的丰富语义信息。 在此项工作中，我们提出了一种跨模态无监督滑片表示学习框架ProAlign。具体而言，我们利用大型语言模型（LLM）为WSI中存在的原型类型生成描述性文本，并引入了补丁-文本对比以构建初始的原型嵌入。此外，我们还提出了一个参数自由的注意力聚合策略，该策略利用补丁与这些原型之间的相似度来形成适用于广泛下游任务的无监督滑片嵌入。 在四个公开数据集上进行的大量实验表明，ProAlign优于现有的无监督框架，并且其性能可媲美一些弱监督模型。
URL
https://arxiv.org/abs/2503.20190
