Abstract
Integrating aerial imagery-based scene generation into applications like autonomous driving and gaming enhances realism in 3D environments, but challenges remain in creating detailed content for occluded areas and ensuring real-time, consistent rendering. In this paper, we introduce Skyeyes, a novel framework that can generate photorealistic sequences of ground view images using only aerial view inputs, thereby creating a ground roaming experience. More specifically, we combine a 3D representation with a view consistent generation model, which ensures coherence between generated images. This method allows for the creation of geometrically consistent ground view images, even with large view gaps. The images maintain improved spatial-temporal coherence and realism, enhancing scene comprehension and visualization from aerial perspectives. To the best of our knowledge, there are no publicly available datasets that contain pairwise geo-aligned aerial and ground view imagery. Therefore, we build a large, synthetic, and geo-aligned dataset using Unreal Engine. Both qualitative and quantitative analyses on this synthetic dataset display superior results compared to other leading synthesis approaches. See the project page for more results: this https URL.
Abstract (translated)
将基于高空影像的场景生成集成到诸如自动驾驶和游戏等应用中，可以增强三维环境中现实感，但仍然需要创建详细的遮蔽区域内容，并确保实时和一致的渲染。在本文中，我们介绍了Skyeyes，一种新方法，可以使用仅包括高空影像输入来生成逼真的地面视图序列，从而实现地面漫游体验。具体来说，我们结合了3D表示和一致生成视图模型，确保生成的图像之间保持连贯性。这种方法允许在视图缺口较大时创建几何一致的地面视图图像，增强场景理解和可视化。据我们所知，目前没有公开可用的数据集包含成对的高空和地面影像的地理对齐。因此，我们使用Unreal Engine构建了一个大型、合成和地理对齐的数据集。这个合成数据集的质量和数量分析都显示出了比其他顶级合成方法更好的结果。更多结果请查看项目页面：https://this URL。
URL
https://arxiv.org/abs/2409.16685
