Paper Reading AI Learner

ObjFiller-3D: Consistent Multi-view 3D Inpainting via Video Diffusion Models

2025-08-25 17:59:40
Haitang Feng, Jie Liu, Jie Tang, Gangshan Wu, Beiqi Chen, Jianhuang Lai, Guangcong Wang

Abstract

3D inpainting often relies on multi-view 2D image inpainting, where the inherent inconsistencies across different inpainted views can result in blurred textures, spatial discontinuities, and distracting visual artifacts. These inconsistencies pose significant challenges when striving for accurate and realistic 3D object completion, particularly in applications that demand high fidelity and structural coherence. To overcome these limitations, we propose ObjFiller-3D, a novel method designed for the completion and editing of high-quality and consistent 3D objects. Instead of employing a conventional 2D image inpainting model, our approach leverages a curated selection of state-of-the-art video editing model to fill in the masked regions of 3D objects. We analyze the representation gap between 3D and videos, and propose an adaptation of a video inpainting model for 3D scene inpainting. In addition, we introduce a reference-based 3D inpainting method to further enhance the quality of reconstruction. Experiments across diverse datasets show that compared to previous methods, ObjFiller-3D produces more faithful and fine-grained reconstructions (PSNR of 26.6 vs. NeRFiller (15.9) and LPIPS of 0.19 vs. Instant3dit (0.25)). Moreover, it demonstrates strong potential for practical deployment in real-world 3D editing applications. Project page: this https URL Code: this https URL .

Abstract (translated)

3D修复(即3D inpainting)通常依赖于多视角的二维图像修补技术,但在不同的修补视图之间可能存在固有的不一致性问题。这些问题会导致模糊纹理、空间不连续性以及视觉干扰等现象。当致力于实现准确且逼真的3D物体完整重建时,特别是在需要高保真度和结构一致性的应用场景中,这种不一致性带来的挑战尤为显著。 为了解决这些局限性,我们提出了一种名为ObjFiller-3D的新方法,旨在完成并编辑高质量且一致的3D对象。与传统2D图像修补模型不同,我们的方法采用了一系列先进的视频编辑模型来填充3D物体中的遮罩区域。通过分析3D和视频之间的表示差距,我们将一种视频修补模型改编用于3D场景修补,并引入了一种基于参考的3D修补方法以进一步提升重建质量。 实验结果显示,在多个数据集上与先前的方法相比,ObjFiller-3D能够生成更为忠实且细节丰富的重构结果(PSNR为26.6对比NeRFiller的15.9和LPIPS为0.19对比Instant3dit的0.25)。此外,它还展示了在现实世界的3D编辑应用中进行实用部署的巨大潜力。 项目页面:[此处插入URL] 代码链接:[此处插入URL]

URL

https://arxiv.org/abs/2508.18271

PDF

https://arxiv.org/pdf/2508.18271.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot