Abstract
DepthCropSeg++: a foundation model for crop segmentation, capable of segmenting different crop species under open in-field environment. Crop segmentation is a fundamental task for modern agriculture, which closely relates to many downstream tasks such as plant phenotyping, density estimation, and weed control. In the era of foundation models, a number of generic large language and vision models have been developed. These models have demonstrated remarkable real world generalization due to significant model capacity and largescale datasets. However, current crop segmentation models mostly learn from limited data due to expensive pixel-level labelling cost, often performing well only under specific crop types or controlled environment. In this work, we follow the vein of our previous work DepthCropSeg, an almost unsupervised approach to crop segmentation, to scale up a cross-species and crossscene crop segmentation dataset, with 28,406 images across 30+ species and 15 environmental conditions. We also build upon a state-of-the-art semantic segmentation architecture ViT-Adapter architecture, enhance it with dynamic upsampling for improved detail awareness, and train the model with a two-stage selftraining pipeline. To systematically validate model performance, we conduct comprehensive experiments to justify the effectiveness and generalization capabilities across multiple crop datasets. Results demonstrate that DepthCropSeg++ achieves 93.11% mIoU on a comprehensive testing set, outperforming both supervised baselines and general-purpose vision foundation models like Segmentation Anything Model (SAM) by significant margins (+0.36% and +48.57% respectively). The model particularly excels in challenging scenarios including night-time environment (86.90% mIoU), high-density canopies (90.09% mIoU), and unseen crop varieties (90.09% mIoU), indicating a new state of the art for crop segmentation.
Abstract (translated)
DepthCropSeg++：一种作物分割的基础模型，在开放的田间环境中能够对不同种类的作物进行精确分割。作物分割是现代农业中的基本任务，与植物表型分析、密度估算和杂草控制等下游任务密切相关。在基础模型的时代，已经开发出了多种通用的大规模语言和视觉模型，这些模型由于拥有强大的模型容量和大规模的数据集，在现实世界中展示了出色的泛化能力。然而，目前的作物分割模型大多因为昂贵的像素级标注成本而依赖于有限的数据训练，通常只能针对特定类型的作物或受控环境表现出色。 在此项工作中，我们遵循了先前工作的DepthCropSeg路线——一种近乎无监督的方法进行作物分割，并通过构建一个涵盖30多种作物种类和15种不同环境条件的跨物种、跨场景的大规模数据集（共计28,406张图像）来扩展这种技术。此外，我们基于最先进的语义分割架构ViT-Adapter架构进行了改进，增强了动态上采样以提高细节识别能力，并通过两阶段自训练流水线进行模型训练。为了系统地验证模型性能，我们在多个作物数据集上开展了全面的实验，证明了其有效性和泛化能力。 结果显示，DepthCropSeg++在综合测试集上的mIoU（平均交并比）达到了93.11%，显著优于监督基准模型和通用视觉基础模型如Segmentation Anything Model (SAM)。特别是在夜间环境、高密度树冠以及未见过的作物品种等具有挑战性的场景中，DepthCropSeg++的表现尤为出色，分别达到86.90% mIoU、90.09% mIoU 和 90.09% mIoU，这标志着在作物分割领域达到了新的技术水平。
URL
https://arxiv.org/abs/2601.12366
