Abstract
Estimating three-dimensional morphological traits from two-dimensional RGB images presents inherent challenges due to the loss of depth information, projection distortions, and occlusions under field conditions. In this work, we explore multiple approaches for non-destructive volume estimation of wheat spikes, using RGB image sequences and structured-light 3D scans as ground truth references. Due to the complex geometry of the spikes, we propose a neural network approach for volume estimation in 2D images, employing a transfer learning pipeline that combines DINOv2, a self-supervised Vision Transformer, with a unidirectional Long Short-Term Memory (LSTM) network. By using deep supervision, the model is able to learn more robust intermediate representations, which enhances its generalisation ability across varying evaluation sequences. We benchmark our model against two conventional baselines: a 2D area-based projection and a geometric reconstruction using axis-aligned cross-sections. Our deep supervised model achieves a mean absolute percentage error (MAPE) of 6.46% on six-view indoor images, outperforming the area (9.36%) and geometric (13.98%) baselines. Fine-tuning the model on field-based single-image data enables domain adaptation, yielding a MAPE of 10.82%. We demonstrate that object shape significantly impacts volume prediction accuracy, with irregular geometries such as wheat spikes posing greater challenges for geometric methods compared to our deep learning approach.
Abstract (translated)
从二维RGB图像估算三维形态特征由于深度信息丢失、投影失真以及现场条件下的遮挡问题，存在固有挑战。在这项工作中，我们探索了多种方法用于小麦穗的无损体积估计，使用RGB图像序列和结构光3D扫描作为基准参考数据。鉴于穗子复杂的几何形状，我们提出了一种基于二维图像的卷积神经网络体积估算方法，采用一个结合自监督视觉变换器DINOv2与单向长短期记忆（LSTM）网络的迁移学习管道。通过深度监督训练模型，它能够学习更加稳健的中间表示形式，从而提高其在不同评估序列中的泛化能力。 我们使用两种传统基准进行对比测试：基于二维面积的投影方法和轴对齐横截面的几何重建法。我们的深度监督模型在六视图室内图像数据集上取得了6.46%的平均绝对百分比误差（MAPE），优于面积（9.36%）和几何（13.98%）基准方法。通过针对基于单张现场图片的数据进行微调，该模型实现了领域适应性，在这种情况下其预测结果的MAPE值为10.82%。 实验表明物体形状对体积预测准确性具有重大影响，对于像小麦穗这样几何不规则的对象来说，传统的几何方法面临着更大的挑战，而我们的深度学习方法表现更佳。
URL
https://arxiv.org/abs/2506.18060
