Abstract
Contemporary approaches to instance segmentation in cell science use 2D or 3D convolutional networks depending on the experiment and data structures. However, limitations in microscopy systems or efforts to prevent phototoxicity commonly require recording sub-optimally sampled data regimes that greatly reduces the utility of such 3D data, especially in crowded environments with significant axial overlap between objects. In such regimes, 2D segmentations are both more reliable for cell morphology and easier to annotate. In this work, we propose the Projection Enhancement Network (PEN), a novel convolutional module which processes the sub-sampled 3D data and produces a 2D RGB semantic compression, and is trained in conjunction with an instance segmentation network of choice to produce 2D segmentations. Our approach combines augmentation to increase cell density using a low-density cell image dataset to train PEN, and curated datasets to evaluate PEN. We show that with PEN, the learned semantic representation in CellPose encodes depth and greatly improves segmentation performance in comparison to maximum intensity projection images as input, but does not similarly aid segmentation in region-based networks like Mask-RCNN. Finally, we dissect the segmentation strength against cell density of PEN with CellPose on disseminated cells from side-by-side spheroids. We present PEN as a data-driven solution to form compressed representations of 3D data that improve 2D segmentations from instance segmentation networks.
Abstract (translated)
当代在细胞科学中实例分割的方法取决于实验和数据结构,通常使用2D或3D卷积网络。然而,显微镜系统的局限性或防止光毒性的努力通常会要求记录不太理想的采样数据体制,这极大地减少了这种3D数据的有用性,特别是在拥挤环境中,物体之间的 axial 重叠很大。在这种体制下,2D分割比细胞形态更加可靠,并且更容易标注。在这项工作中,我们提出了投影增强网络(PEN),这是一种新的卷积模块,处理未采样的3D数据并产生2D RGB语义压缩,并与选择实例分割网络进行训练以产生2D分割。我们的方法结合了增强来增加细胞密度,使用低密度细胞图像数据集训练PEN,并使用 curated 数据集评估PEN。我们表明,与PEN一起, CellPose 学习的语义表示编码深度,相对于输入的最大光强度投影图像,显著提高了分割性能,但不像Mask-RCNN等区域网络那样,对分割没有类似地帮助。最后,我们分解了分割强度与细胞密度的PEN与相邻的微囊细胞中的分散细胞进行 CellPose。我们提出了 PEN 作为数据驱动的解决方案,以形成压缩的3D数据表示,从实例分割网络改善2D分割。
URL
https://arxiv.org/abs/2301.10877
