Abstract
Disjoint sampling is critical for rigorous and unbiased evaluation of state-of-the-art (SOTA) models. When training, validation, and test sets overlap or share data, it introduces a bias that inflates performance metrics and prevents accurate assessment of a model's true ability to generalize to new examples. This paper presents an innovative disjoint sampling approach for training SOTA models on Hyperspectral image classification (HSIC) tasks. By separating training, validation, and test data without overlap, the proposed method facilitates a fairer evaluation of how well a model can classify pixels it was not exposed to during training or validation. Experiments demonstrate the approach significantly improves a model's generalization compared to alternatives that include training and validation data in test data. By eliminating data leakage between sets, disjoint sampling provides reliable metrics for benchmarking progress in HSIC. Researchers can have confidence that reported performance truly reflects a model's capabilities for classifying new scenes, not just memorized pixels. This rigorous methodology is critical for advancing SOTA models and their real-world application to large-scale land mapping with Hyperspectral sensors. The source code is available at this https URL.
Abstract (translated)
离散采样对于准确和无偏见地评估最先进的（SOTA）模型至关重要。当训练集、验证集和测试集不重叠或共享数据时，它引入了偏差，导致性能指标膨胀，并阻止了对模型在为新实例上进行准确评估。本文提出了一种创新性的离散采样方法，用于在 Hyperspectral image classification (HSIC) 任务上训练 SOTA 模型。通过分离训练集、验证集和测试集，所提出的方法有助于更公平地评估模型在训练集或验证集上从未暴露过的像素的分类能力。实验证明，与包括训练和验证数据在测试集中的替代方法相比，该方法显著提高了模型的泛化能力。通过消除数据集之间的泄漏，离散采样为基于HSIC 的基准测试提供了可靠的度量。研究人员可以放心地相信，所报告的性能反映了模型对分类新场景的能力，而不仅仅是记忆中的像素。这种严谨的方法对于推动 SOTA 模型及其在大型地图应用中的实际应用至关重要。源代码可在此处访问：https://www.osgeo.org/。
URL
https://arxiv.org/abs/2404.14944
