Abstract
Tweets provides valuable semantic context for earth observation tasks and serves as a complementary modality to remote sensing imagery. In building function classification (BFC), tweets are often collected using geographic heuristics and labeled via external databases, an inherently weakly supervised process that introduces both label noise and sentence level feature noise (e.g., irrelevant or uninformative tweets). While label noise has been widely studied, the impact of sentence level feature noise remains underexplored, largely due to the lack of clean benchmark datasets for controlled analysis. In this work, we propose a method for generating a synthetic oracle dataset using LLM, designed to contain only tweets that are both correctly labeled and semantically relevant to their associated buildings. This oracle dataset enables systematic investigation of noise impacts that are otherwise difficult to isolate in real-world data. To assess its utility, we compare model performance using Naive Bayes and mBERT classifiers under three configurations: real vs. synthetic training data, and cross-domain generalization. Results show that noise in real tweets significantly degrades the contextual learning capacity of mBERT, reducing its performance to that of a simple keyword-based model. In contrast, the clean synthetic dataset allows mBERT to learn effectively, outperforming Naive Bayes Bayes by a large margin. These findings highlight that addressing feature noise is more critical than model complexity in this task. Our synthetic dataset offers a novel experimental environment for future noise injection studies and is publicly available on GitHub.
Abstract (translated)
推文为地球观测任务提供了有价值的语义背景，并作为遥感图像的补充模态。在建筑物功能分类（BFC）中，通常通过地理启发式方法收集推文，并使用外部数据库进行标注，这一过程本质上是弱监督的，会引入标签噪声和句子级别的特征噪声（例如，无关或无信息的推文）。虽然标签噪声已得到广泛研究，但句子级别特征噪声的影响却鲜有探讨，这主要是因为缺乏用于受控分析的干净基准数据集。在本文中，我们提出了一种使用大型语言模型（LLM）生成合成oracle数据集的方法，该方法仅包含正确标注且语义上与相关建筑物相关的推文。这一合成数据集使得系统性地研究噪声影响成为可能，这些影响在现实世界的数据中难以隔离。为了评估其效用，我们比较了使用朴素贝叶斯和mBERT分类器的模型性能，在三种配置下进行对比：真实数据与合成训练数据，以及跨域泛化能力。结果显示，真实推文中的噪声显著降低了mBERT的情境学习能力，使其表现下降到简单关键词模型的水平。相比之下，干净的合成数据集使mBERT能够有效学习，并大幅超越朴素贝叶斯分类器的表现。这些发现表明，在此任务中解决特征噪声比增加模型复杂性更为关键。我们的合成数据集为未来的噪声注入研究提供了新颖的实验环境，并已公开发布在GitHub上。
URL
https://arxiv.org/abs/2503.22856
