Abstract
Multiple Instance Learning (MIL) has enabled weakly supervised analysis of whole-slide images (WSIs) in computational pathology. However, traditional MIL approaches often lose crucial contextual information, while transformer-based variants, though more expressive, suffer from quadratic complexity and redundant computations. To address these limitations, we propose HookMIL, a context-aware and computationally efficient MIL framework that leverages compact, learnable hook tokens for structured contextual aggregation. These tokens can be initialized from (i) key-patch visual features, (ii) text embeddings from vision-language pathology models, and (iii) spatially grounded features from spatial transcriptomics-vision models. This multimodal initialization enables Hook Tokens to incorporate rich textual and spatial priors, accelerating convergence and enhancing representation quality. During training, Hook tokens interact with instances through bidirectional attention with linear complexity. To further promote specialization, we introduce a Hook Diversity Loss that encourages each token to focus on distinct histopathological patterns. Additionally, a hook-to-hook communication mechanism refines contextual interactions while minimizing redundancy. Extensive experiments on four public pathology datasets demonstrate that HookMIL achieves state-of-the-art performance, with improved computational efficiency and interpretability. Codes are available at this https URL.
Abstract (translated)
多实例学习（Multiple Instance Learning，MIL）已经在计算病理学中的全滑片图像（Whole-Slide Images，WSIs）的弱监督分析中发挥了重要作用。然而，传统的MIL方法常常会丢失关键的上下文信息，而基于Transformer的方法虽然更加表达丰富，但面临着二次方复杂度和冗余计算的问题。为了克服这些限制，我们提出了一种名为HookMIL的框架，这是一种具有上下文感知能力和计算效率的多实例学习框架。该框架利用紧凑且可学习的hook令牌进行结构化上下文聚合。 这些钩子令牌可以从以下三种方式之一初始化：（i）关键补丁视觉特征；（ii）来自视觉-语言病理学模型的文本嵌入；以及（iii）空间转录组学-视觉模型的空间定位特征。这种多模态初始化使Hook Tokens能够整合丰富的文本和空间先验信息，从而加速收敛并提高表示质量。 在训练过程中，Hook令牌通过线性复杂度的双向注意力与实例进行互动。为进一步促进专业化，我们引入了一种钩子多样性损失（Hook Diversity Loss），鼓励每个钩子令牌专注于不同的组织病理学模式。此外，一个hook-to-hook通信机制细化了上下文交互，并减少了冗余。 在四个公开的病理数据集上进行了广泛的实验，结果表明HookMIL达到了最先进的性能，在计算效率和可解释性方面都有所提升。代码可在上述链接获得。
URL
https://arxiv.org/abs/2512.22188
