Abstract
Advances in 3D reconstruction have enabled high-quality 3D capture, but require a user to collect hundreds to thousands of images to create a 3D scene. We present CAT3D, a method for creating anything in 3D by simulating this real-world capture process with a multi-view diffusion model. Given any number of input images and a set of target novel viewpoints, our model generates highly consistent novel views of a scene. These generated views can be used as input to robust 3D reconstruction techniques to produce 3D representations that can be rendered from any viewpoint in real-time. CAT3D can create entire 3D scenes in as little as one minute, and outperforms existing methods for single image and few-view 3D scene creation. See our project page for results and interactive demos at this https URL .
Abstract (translated)
3D重建技术的进步使得高质量的3D捕捉成为可能，但需要用户收集数百到数千张图像来创建3D场景。我们提出了一种名为CAT3D的方法，通过使用多视角扩散模型模拟这种现实世界的捕捉过程，来创建任何3D物体。给定任意数量的输入图像和一组目标新视角，我们的模型生成场景中高度一致的新视角。这些生成的视图可以作为输入，用于具有实时渲染能力的稳健3D重建技术，产生可以从任何视角渲染的3D表示。CAT3D可以在不到一分钟的时间内创建整个3D场景，并超越了现有方法在单张图像和少数视角3D场景创建方面的表现。请查看我们的项目页面，以查看结果和交互式演示。https://url.com/cat3d 。
URL
https://arxiv.org/abs/2405.10314
