Abstract
We present SeeingThroughClutter, a method for reconstructing structured 3D representations from single images by segmenting and modeling objects individually. Prior approaches rely on intermediate tasks such as semantic segmentation and depth estimation, which often underperform in complex scenes, particularly in the presence of occlusion and clutter. We address this by introducing an iterative object removal and reconstruction pipeline that decomposes complex scenes into a sequence of simpler subtasks. Using VLMs as orchestrators, foreground objects are removed one at a time via detection, segmentation, object removal, and 3D fitting. We show that removing objects allows for cleaner segmentations of subsequent objects, even in highly occluded scenes. Our method requires no task-specific training and benefits directly from ongoing advances in foundation models. We demonstrate stateof-the-art robustness on 3D-Front and ADE20K datasets. Project Page: this https URL
Abstract (translated)
我们介绍了SeeingThroughClutter方法，这是一种从单张图像中通过单独分割和建模物体来重构结构化3D表示的方法。先前的方法依赖于诸如语义分割和深度估计等中间任务，在复杂场景（特别是在遮挡和杂乱情况下）中这些任务通常表现不佳。为了解决这个问题，我们引入了一种迭代的对象移除与重建流水线，该方法将复杂的场景分解成一系列更简单的子任务。通过使用视觉语言模型作为协调者，前景物体被逐个检测、分割、移除，并进行3D拟合。我们展示了这种方法在即使是非常遮挡的场景中也能为后续物体提供更加干净的分割效果。我们的方法无需特定于任务的训练，可直接从基础模型的进步中受益。我们在3D-Front和ADE20K数据集上展示了最先进的鲁棒性。 项目页面：[此处应填写具体的URL链接]
URL
https://arxiv.org/abs/2602.04053
