Abstract
Previous methods for image geo-localization have typically treated the task as either classification or retrieval, often relying on black-box decisions that lack interpretability. The rise of large vision-language models (LVLMs) has enabled a rethinking of geo-localization as a reasoning-driven task grounded in visual cues. However, two major challenges persist. On the data side, existing reasoning-focused datasets are primarily based on street-view imagery, offering limited scene diversity and constrained viewpoints. On the modeling side, current approaches predominantly rely on supervised fine-tuning, which yields only marginal improvements in reasoning capabilities. To address these challenges, we propose a novel pipeline that constructs a reasoning-oriented geo-localization dataset, MP16-Reason, using diverse social media images. We introduce GLOBE, Group-relative policy optimization for Locatability assessment and Optimized visual-clue reasoning, yielding Bi-objective geo-Enhancement for the VLM in recognition and reasoning. GLOBE incorporates task-specific rewards that jointly enhance locatability assessment, visual clue reasoning, and geolocation accuracy. Both qualitative and quantitative results demonstrate that GLOBE outperforms state-of-the-art open-source LVLMs on geo-localization tasks, particularly in diverse visual scenes, while also generating more insightful and interpretable reasoning trajectories.
Abstract (translated)
之前的图像地理定位方法通常将其视为分类或检索任务，常常依赖于缺乏可解释性的黑盒决策。随着大型视觉语言模型（LVLM）的兴起，人们开始重新思考将地理定位作为基于视觉线索的推理驱动型任务。然而，仍然存在两个主要挑战。在数据方面，现有的以推理为中心的数据集主要基于街景图像，这提供了有限的场景多样性以及受限的视角选择。在建模方面，当前的方法主要依赖于监督微调，这仅能带来有限的推理能力提升。 为了解决这些挑战，我们提出了一种新的工作流程，构建了一个侧重于推理的地理定位数据集——MP16-Reason，该数据集使用了多样化的社交媒体图像。同时，我们引入了GLOBE（基于组相对策略优化的可定位性评估和优化视觉线索推理），旨在为LVLM在识别与推理任务中的表现带来双目标增强。GLOBE整合了特定于任务的奖励机制，以共同提升可定位性评估、视觉线索推理以及地理坐标的准确性。 无论是定性的还是定量的结果都表明，GLOBE超越了现有开源LVLM模型，在多样化视觉场景下的地理定位任务中表现出色，并且生成更具有洞察力和解释性的推理路径。
URL
https://arxiv.org/abs/2506.14674
