Abstract
Medical image segmentation is fundamental to clinical decision-making, yet existing models remain fragmented. They are usually trained on single knowledge sources and specific to individual tasks, modalities, or organs. This fragmentation contrasts sharply with clinical practice, where experts seamlessly integrate diverse knowledge: anatomical priors from training, exemplar-based reasoning from reference cases, and iterative refinement through real-time interaction. We present $\textbf{K-Prism}$, a unified segmentation framework that mirrors this clinical flexibility by systematically integrating three knowledge paradigms: (i) $\textit{semantic priors}$ learned from annotated datasets, (ii) $\textit{in-context knowledge}$ from few-shot reference examples, and (iii) $\textit{interactive feedback}$ from user inputs like clicks or scribbles. Our key insight is that these heterogeneous knowledge sources can be encoded into a dual-prompt representation: 1-D sparse prompts defining $\textit{what}$ to segment and 2-D dense prompts indicating $\textit{where}$ to attend, which are then dynamically routed through a Mixture-of-Experts (MoE) decoder. This design enables flexible switching between paradigms and joint training across diverse tasks without architectural modifications. Comprehensive experiments on 18 public datasets spanning diverse modalities (CT, MRI, X-ray, pathology, ultrasound, etc.) demonstrate that K-Prism achieves state-of-the-art performance across semantic, in-context, and interactive segmentation settings. Code will be released upon publication.
Abstract (translated)
医学图像分割对于临床决策至关重要，但现有的模型仍然存在碎片化的问题。这些模型通常仅基于单一的知识来源进行训练，并且针对特定任务、模态或器官。这种碎片化的状况与临床实践中专家们能够无缝整合多样知识形成了鲜明对比：从训练中获得的解剖学先验知识、参考案例中的类比推理以及通过实时互动实现的迭代改进。我们提出了 $\textbf{K-Prism}$，这是一个统一的分割框架，它模仿了这种临床灵活性，系统地集成了三种知识范式：（i）从标注数据集中学习到的语义先验；（ii）来自少量参考示例的上下文知识；以及（iii）来自于用户输入（如点击或涂鸦）的交互反馈。我们的关键洞察是这些异质的知识来源可以被编码成一种双重提示表示形式：1-D稀疏提示定义“要分割什么”，2-D密集提示指示“在哪里注意”，然后通过专家混合模型（MoE）解码器动态路由。这种设计使得在不同的范式之间灵活切换并跨多种任务进行联合训练成为可能，而无需对架构做出任何修改。在涵盖不同模态的18个公共数据集上进行了全面实验（CT、MRI、X光片、病理学、超声等），结果表明K-Prism 在语义分割、上下文分割和交互式分割设置中均达到了最先进的性能水平。发布代码将在论文发表后提供。
URL
https://arxiv.org/abs/2509.25594
