Abstract
Semantic segmentation of electron microscopy (EM) images of biological samples remains a challenge in the life sciences. EM data captures details of biological structures, sometimes with such complexity that even human observers can find it overwhelming. We introduce {\epsilon}-Seg, a method based on hierarchical variational autoencoders (HVAEs), employing center-region masking, sparse label contrastive learning (CL), a Gaussian mixture model (GMM) prior, and clustering-free label prediction. Center-region masking and the inpainting loss encourage the model to learn robust and representative embeddings to distinguish the desired classes, even if training labels are sparse (0.05% of the total image data or less). For optimal performance, we employ CL and a GMM prior to shape the latent space of the HVAE such that encoded input patches tend to cluster wrt. the semantic classes we wish to distinguish. Finally, instead of clustering latent embeddings for semantic segmentation, we propose a MLP semantic segmentation head to directly predict class labels from latent embeddings. We show empirical results of {\epsilon}-Seg and baseline methods on 2 dense EM datasets of biological tissues and demonstrate the applicability of our method also on fluorescence microscopy data. Our results show that {\epsilon}-Seg is capable of achieving competitive sparsely-supervised segmentation results on complex biological image data, even if only limited amounts of training labels are available.
Abstract (translated)
生物样本的电子显微镜（EM）图像语义分割在生命科学领域仍是一个挑战。EM数据捕捉了生物结构的详细信息，有时复杂到即使是人类观察者也难以应对。我们提出了{\epsilon}-Seg方法，该方法基于分层变分自编码器（HVAE），采用中心区域掩码、稀疏标签对比学习（CL）、高斯混合模型（GMM）先验以及无聚类的标签预测技术。 - 中心区域掩码和修补损失促使模型学会稳健且具有代表性的嵌入，即使训练标签非常稀疏（占总图像数据的0.05%或更少），也能区分所需的类别。 - 为了达到最佳性能，我们采用CL和GMM先验来塑造HVAE的潜在空间，使编码输入补丁倾向于根据我们希望区别的语义类进行聚类。 - 最后，在不使用潜在嵌入集群的情况下进行语义分割，{\epsilon}-Seg方法提出了一种MLP（多层感知器）语义分割头直接从潜在嵌入预测类别标签。 我们在两种密集EM生物组织数据集上展示了{\epsilon}-Seg及其基线方法的实证结果，并表明我们的方法在荧光显微镜图像数据上也具有适用性。研究表明，即使只有少量训练标签可用时，{\epsilon}-Seg仍能对复杂生物图像数据实现竞争性的稀疏监督分割效果。
URL
https://arxiv.org/abs/2510.18637
