Abstract
Amodal completion, which is the process of inferring the full appearance of objects despite partial occlusions, is crucial for understanding complex human-object interactions (HOI) in computer vision and robotics. Existing methods, such as those that use pre-trained diffusion models, often struggle to generate plausible completions in dynamic scenarios because they have a limited understanding of HOI. To solve this problem, we've developed a new approach that uses physical prior knowledge along with a specialized multi-regional inpainting technique designed for HOI. By incorporating physical constraints from human topology and contact information, we define two distinct regions: the primary region, where occluded object parts are most likely to be, and the secondary region, where occlusions are less probable. Our multi-regional inpainting method uses customized denoising strategies across these regions within a diffusion model. This improves the accuracy and realism of the generated completions in both their shape and visual detail. Our experimental results show that our approach significantly outperforms existing methods in HOI scenarios, moving machine perception closer to a more human-like understanding of dynamic environments. We also show that our pipeline is robust even without ground-truth contact annotations, which broadens its applicability to tasks like 3D reconstruction and novel view/pose synthesis.
Abstract (translated)
模态完成（amodal completion）是指即使面对物体的部分遮挡，也能推断出其完整外观的过程。这一过程对于理解计算机视觉和机器人领域中复杂的“人与物互动”（human-object interaction, HOI）至关重要。现有的方法，例如使用预训练扩散模型的方法，在处理动态场景时往往难以生成可信的完成图，因为它们对HOI的理解有限。为了解决这个问题，我们开发了一种新的方法，该方法结合了物理先验知识，并设计了一种专门用于HOI的多区域修复技术。 通过引入人体拓扑和接触信息的物理约束条件，我们将需要修复的区域划分为两个不同的部分：主要区域，在这里被遮挡的部分最有可能出现；次要区域，这里的遮挡可能性较小。我们的多区域修复方法在扩散模型中针对这两个区域内采用定制化的降噪策略，从而提升生成完成图的准确性和真实感，无论是形状还是视觉细节。 实验结果显示，相比现有的方法，我们在处理HOI场景时的方法有显著优势，使机器感知向更加接近人类对动态环境的理解迈进了一步。此外，我们还证明了即使在没有地面实况接触标注的情况下，我们的管道依然具有稳健性，这扩展了其应用于三维重建和新视图/姿态合成等任务的可能性。
URL
https://arxiv.org/abs/2508.00427
