Abstract
In recent years, foundation models such as CLIP, DINO,and CONCH have demonstrated remarkable domain generalization and unsupervised feature extraction capabilities across diverse imaging tasks. However, systematic and independent evaluations of these models for pixel-level semantic segmentation in histopathology remain scarce. In this study, we propose a robust benchmarking approach to asses 10 foundational models on four histopathological datasets covering both morphological tissue-region and cellular/nuclear segmentation tasks. Our method leverages attention maps of foundation models as pixel-wise features, which are then classified using a machine learning algorithm, XGBoost, enabling fast, interpretable, and model-agnostic evaluation without finetuning. We show that the vision language foundation model, CONCH performed the best across datasets when compared to vision-only foundation models, with PathDino as close second. Further analysis shows that models trained on distinct histopathology cohorts capture complementary morphological representations, and concatenating their features yields superior segmentation performance. Concatenating features from CONCH, PathDino and CellViT outperformed individual models across all the datasets by 7.95% (averaged across the datasets), suggesting that ensembles of foundation models can better generalize to diverse histopathological segmentation tasks.
Abstract (translated)
近年来，基础模型（如CLIP、DINO和CONCH）在各种成像任务中展现了出色的领域泛化能力和无监督特征提取能力。然而，在像素级语义分割的组织病理学应用方面，这些模型系统的独立评估仍然很少见。在这项研究中，我们提出了一种稳健的基准测试方法，用于评估10个基础模型在四个涵盖形态组织区域和细胞/核分割任务的组织病理学数据集上的性能。我们的方法利用了基础模型的注意力图作为像素级特征，并使用机器学习算法XGBoost进行分类，这使得无需微调即可快速、可解释且与模型无关地评估这些模型。我们发现，在所有数据集中，视觉语言基础模型CONCH的表现优于仅基于视觉的基础模型，而PathDino紧随其后。进一步的分析表明，针对不同的组织病理学队列训练出的模型捕捉到了互补的形态特征表示，并将它们的特征合并可以显著提高分割性能。结合CONCH、PathDino和CellViT的特征在所有数据集上均超越了单一模型的表现（平均高出7.95%），表明基础模型的集成能够更好地泛化到各种组织病理学分割任务中。
URL
https://arxiv.org/abs/2602.18747
