Abstract
We introduce HouseTour, a method for spatially-aware 3D camera trajectory and natural language summary generation from a collection of images depicting an existing 3D space. Unlike existing vision-language models (VLMs), which struggle with geometric reasoning, our approach generates smooth video trajectories via a diffusion process constrained by known camera poses and integrates this information into the VLM for 3D-grounded descriptions. We synthesize the final video using 3D Gaussian splatting to render novel views along the trajectory. To support this task, we present the HouseTour dataset, which includes over 1,200 house-tour videos with camera poses, 3D reconstructions, and real estate descriptions. Experiments demonstrate that incorporating 3D camera trajectories into the text generation process improves performance over methods handling each task independently. We evaluate both individual and end-to-end performance, introducing a new joint metric. Our work enables automated, professional-quality video creation for real estate and touristic applications without requiring specialized expertise or equipment.
Abstract (translated)
我们介绍了HouseTour方法，这是一种从一系列描绘现有三维空间的图像中生成具有空间意识的3D相机轨迹和自然语言摘要的方法。与现有的视觉-语言模型（VLMs）不同，这些模型在几何推理方面存在困难，我们的方法通过一个受已知相机姿态约束的扩散过程来生成平滑的视频轨迹，并将此信息整合到VLM中以产生基于三维空间的描述。我们使用3D高斯点喷射技术合成最终视频，在整个轨迹上渲染新的视角视图。为了支持这项任务，我们提出了HouseTour数据集，该数据集包括超过1,200个房屋导览视频、相机姿态、三维重建和房地产描述。实验表明，将3D摄像机轨迹纳入文本生成过程中可以改善独立处理每个任务的方法的性能。我们评估了单独的任务性能以及端到端的整体性能，并引入了一个新的联合评价指标。我们的工作使得无需专门的技术知识或设备便能够自动创建出高质量的专业级视频，适用于房地产和旅游应用领域。
URL
https://arxiv.org/abs/2510.18054
