Abstract
Synthesizing physically plausible articulated human-object interactions (HOI) without 3D/4D supervision remains a fundamental challenge. While recent zero-shot approaches leverage video diffusion models to synthesize human-object interactions, they are largely confined to rigid-object manipulation and lack explicit 4D geometric reasoning. To bridge this gap, we formulate articulated HOI synthesis as a 4D reconstruction problem from monocular video priors: given only a video generated by a diffusion model, we reconstruct a full 4D articulated scene without any 3D supervision. This reconstruction-based approach treats the generated 2D video as supervision for an inverse rendering problem, recovering geometrically consistent and physically plausible 4D scenes that naturally respect contact, articulation, and temporal coherence. We introduce ArtHOI, the first zero-shot framework for articulated human-object interaction synthesis via 4D reconstruction from video priors. Our key designs are: 1) Flow-based part segmentation: leveraging optical flow as a geometric cue to disentangle dynamic from static regions in monocular video; 2) Decoupled reconstruction pipeline: joint optimization of human motion and object articulation is unstable under monocular ambiguity, so we first recover object articulation, then synthesize human motion conditioned on the reconstructed object states. ArtHOI bridges video-based generation and geometry-aware reconstruction, producing interactions that are both semantically aligned and physically grounded. Across diverse articulated scenes (e.g., opening fridges, cabinets, microwaves), ArtHOI significantly outperforms prior methods in contact accuracy, penetration reduction, and articulation fidelity, extending zero-shot interaction synthesis beyond rigid manipulation through reconstruction-informed synthesis.
Abstract (translated)
合成物理上合理的人体与物体交互（HOI）场景，而无需3D或4D的监督数据，仍然是一个基本挑战。尽管最近的一些零样本方法利用视频扩散模型来合成人体与物体之间的互动，但这些方法主要局限于对刚性对象的操作，并缺乏显式的四维几何推理能力。为了弥合这一差距，我们把关节式HOI生成定义为一项单目视频先验下的四维重建任务：仅使用由扩散模型生成的视频作为输入，无需任何3D监督数据，即可重构出完整的四维关节场景。基于这种重建的方法将生成的2D视频视为反向渲染问题中的监督信号，在保证几何一致性和物理合理性的前提下尊重接触、关节和时间连续性。 我们引入了ArtHOI——第一个通过从视频先验中进行4D重建来实现零样本关节式人体与物体交互合成的框架。我们的关键设计包括： 1. 基于流的人体部位分割：利用光流作为几何线索，将动态区域从单目视频中的静态区域中分离出来； 2. 解耦重构流水线：在单目模糊性下同时优化人类运动和对象关节是不稳定的，因此我们首先恢复物体的关节状态，然后基于重建的对象状态条件生成人体动作。 ArtHOI通过视频生成与几何感知重建之间的桥梁产生了语义对齐且物理实证的互动场景。在各种复杂的关节式场景（例如打开冰箱、橱柜、微波炉）中，ArtHOI在接触准确性、穿透减少和关节真实度上明显优于先前的方法，在没有监督数据的情况下扩展了零样本交互合成能力，超越了刚性操作范围，并通过重建指导合成来实现这一目标。
URL
https://arxiv.org/abs/2603.04338
