Abstract
We present a lightweight two-stage framework for joint geometry and color inpainting of damaged 3D objects, motivated by the digital restoration of cultural heritage artifacts. The pipeline separates damage localization from reconstruction. In the first stage, a 2D convolutional network predicts damage masks on RGB slices extracted from a voxelized object, and these predictions are aggregated into a volumetric mask. In the second stage, a diffusion-based 3D U-Net performs mask-conditioned inpainting directly on voxel grids, reconstructing geometry and color while preserving observed regions. The model jointly predicts occupancy and color using a composite objective that combines occupancy reconstruction with masked color reconstruction and perceptual regularization. We evaluate the approach on a curated set of textured artifacts with synthetically generated damage using standard geometric and color metrics. Compared to symmetry-based baselines, our method produces more complete geometry and more coherent color reconstructions at a fixed 32^3 resolution. Overall, the results indicate that explicit mask conditioning is a practical way to guide volumetric diffusion models for joint 3D geometry and color inpainting.
Abstract (translated)
我们提出了一种轻量级的两阶段框架，用于受损三维物体的几何和颜色修补，该框架受到了文化遗迹数字修复工作的启发。此流程将损伤定位与重建过程分开。在第一阶段中，一个2D卷积网络会预测从体素化对象提取的颜色图层中的损坏掩码，并将这些预测汇总成体积掩码。在第二阶段中，基于扩散的3D U-Net直接在体素网格上进行条件修复（即利用掩码信息），同时重建几何形状和颜色并保持可见区域不变。模型通过结合占用度重构、屏蔽颜色重构以及感知正则化的目标函数来联合预测占据情况和颜色。 我们在一组精心挑选的具有合成损坏的文化艺术品上使用标准几何和颜色指标评估了该方法的效果。与基于对称性的基线方法相比，我们的方法在固定的32^3分辨率下生成了更完整的几何重建及更加一致的颜色恢复结果。总体而言，这些结果显示明确地进行掩码条件设置是一种实用的方法，能够引导体积扩散模型实现联合的三维几何和颜色修补任务。
URL
https://arxiv.org/abs/2601.00368
