Abstract
What did it feel like to walk through a city from the past? In this work, we describe Nostalgin (Nostalgia Engine), a method that can faithfully reconstruct cities from historical images. Unlike existing work in city reconstruction, we focus on the task of reconstructing 3D cities from historical images. Working with historical image data is substantially more difficult, as there are significantly fewer buildings available and the details of the camera parameters which captured the images are unknown. Nostalgin can generate a city model even if there is only a single image per facade, regardless of viewpoint or occlusions. To achieve this, our novel architecture combines image segmentation, rectification, and inpainting. We motivate our design decisions with experimental analysis of individual components of our pipeline, and show that we can improve on baselines in both speed and visual realism. We demonstrate the efficacy of our pipeline by recreating two 1940s Manhattan city blocks. We aim to deploy Nostalgin as an open source platform where users can generate immersive historical experiences from their own photos.
Abstract (translated)
从过去走过一座城市感觉如何？在这部作品中，我们描述了怀旧引擎，一种可以从历史图像忠实地重建城市的方法。与现有的城市重建工作不同，我们的重点是从历史图像重建三维城市。使用历史图像数据要困难得多，因为可用建筑明显减少，拍摄图像的相机参数细节未知。怀旧可以生成一个城市模型，即使每个立面只有一个图像，不管是视角还是闭塞。为了实现这一点，我们的新架构结合了图像分割、校正和修复。我们通过对管道各个组件的实验分析来激励我们的设计决策，并表明我们可以在速度和视觉真实性方面改进基线。我们通过再现20世纪40年代曼哈顿的两个街区来展示我们的管道的功效。我们的目标是将怀旧作为一个开放源代码平台，用户可以从自己的照片中产生沉浸式的历史体验。
URL
https://arxiv.org/abs/1905.01772
