Abstract
Semantic segmentation metrics for 3D point clouds, such as mean Intersection over Union (mIoU) and Overall Accuracy (OA), present two key limitations in the context of aerial LiDAR data. First, they treat all misclassifications equally regardless of their spatial context, overlooking cases where the geometric severity of errors directly impacts the quality of derived geospatial products such as Digital Terrain Models. Second, they are often dominated by the large proportion of easily classified points, which can mask meaningful differences between models and under-represent performance in challenging regions. To address these limitations, we propose a novel evaluation framework for comparing semantic segmentation models through two complementary approaches. First, we introduce distance-based metrics that account for the spatial deviation between each misclassified point and the nearest ground-truth point of the predicted class, capturing the geometric severity of errors. Second, we propose a focused evaluation on a common subset of hard points, defined as the points misclassified by at least one of the evaluated models, thereby reducing the bias introduced by easily classified points and better revealing differences in model performance in challenging regions. We validate our framework by comparing three state-of-the-art deep learning models on three aerial LiDAR datasets. Results demonstrate that the proposed metrics provide complementary information to traditional measures, revealing spatial error patterns that are critical for Earth Observation applications but invisible to conventional evaluation approaches. The proposed framework enables more informed model selection for scenarios where spatial consistency is critical.
Abstract (translated)
针对航空LiDAR数据的3D点云语义分割评估指标（如平均交并比mIoU和总体准确率OA）存在两个关键局限性。首先，这些指标对所有误分类情况一视同仁， irrespective of their spatial context，忽略了误差的几何严重性直接影响数字地形模型等衍生地理空间产品质量的情况。其次，由于大量易分类点占据主导，可能掩盖模型间的实质性差异，并低估困难区域的性能表现。 为应对这些局限，我们提出一种新型评估框架，通过两种互补方法比较语义分割模型：其一，引入基于距离的指标，通过计算每个误分类点与预测类别最近真实点之间的空间偏差，量化误差的几何严重性；其二，聚焦于困难点的共性子集（定义为至少被一个评估模型误分类的点），从而减少易分类点带来的偏差，更准确揭示模型在困难区域的性能差异。 我们在三个航空LiDAR数据集上对比三种先进深度学习模型验证该框架。结果表明，所提指标能为传统度量提供补充信息，揭示对地球观测应用至关重要但传统评估方法无法捕捉的空间误差模式。该框架可在空间一致性要求严苛的场景中支持更明智的模型选择。
URL
https://arxiv.org/abs/2603.22420
