Abstract
Environmental sound understanding in computational auditory scene analysis (CASA) is often formulated as an audio-only recognition problem. This formulation leaves a persistent drawback in multi-label audio tagging (AT): acoustic similarity can make certain events difficult to separate from waveforms alone. In such cases, disambiguating cues often lie outside the waveform. Geospatial semantic context (GSC), derived from geographic information system data, e.g., points of interest (POI), provides location-tied environmental priors that can help reduce this ambiguity. A systematic study of this direction is enabled through the proposed geospatial audio tagging (Geo-AT) task, which conditions multi-label sound event tagging on GSC alongside audio. To benchmark Geo-AT, Geo-ATBench is introduced as a polyphonic audio benchmark with geographical annotations, containing 10.71 hours of audio across 28 event categories; each clip is paired with a GSC representation from 11 semantic context categories. GeoFusion-AT is proposed as a unified geo-audio fusion framework that evaluates feature-, representation-, and decision-level fusion on representative audio backbones, with audio- and GSC-only baselines. Results show that incorporating GSC improves AT performance, especially on acoustically confounded labels, indicating geospatial semantics provide effective priors beyond audio alone. A crowdsourced listening study with 10 participants on 579 samples shows that there is no significant difference in performance between models on Geo-ATBench labels and aggregated human labels, supporting Geo-ATBench as a human-aligned benchmark. The Geo-AT task, benchmark Geo-ATBench, and reproducible geo-audio fusion framework GeoFusion-AT provide a foundation for studying AT with geospatial semantic context within the CASA community. Dataset, code, models are on homepage (this https URL).
Abstract (translated)
在计算听觉场景分析（CASA）中的环境声音理解通常被表述为一个仅基于音频的识别问题。这种表述方式导致了多标签音频标记（AT）的一个持久缺陷：声学相似性使得某些事件难以仅通过波形来区分。在这种情况下，明确这些事件所需的信息往往存在于波形之外。从地理信息系统数据中提取的地空间语义上下文（GSC），例如兴趣点（POI），提供了与地理位置相关的环境先验信息，有助于减少这种不确定性。 为了系统地研究这一方向，提出了一个基于地空间音频标记（Geo-AT）的任务，该任务在多标签声音事件标注时结合了地理空间语境和音频。为评估Geo-AT，提出了一种名为Geo-ATBench的多声部音频基准测试集，并带有地理位置注释，其中包括跨越28个类别、总计10.71小时的音频；每个音频片段都配有一个从11个语义上下文类别中选取的地空间表示。提出了GeoFusion-AT作为统一的地空间音频融合框架，在代表性音频骨干网络上评估特征级、表征级和决策级融合，并带有仅基于音频和GSC的基线模型。 结果表明，结合地空间信息可以提升音频标记性能，特别是在声学混淆标签上的表现尤为显著。这说明了地空间语义提供了有效的先验信息，超越了单独的音频数据所能提供的范畴。一项由10名参与者参与、涉及579个样本的人群外包聆听研究表明，Geo-ATBench模型与汇总人类标注之间的性能没有显著差异，证明了Geo-ATBench是一个人机一致性的基准测试。 该地空间音频标记任务（Geo-AT）、基准Geo-ATBench以及可重复的地空间音频融合框架GeoFusion-AT为CASA社区研究带有地空间语义上下文的音频标签提供了一个基础。数据集、代码和模型可在其主页上找到（此链接：this https URL）。
URL
https://arxiv.org/abs/2603.10623
