Abstract
Hand manipulating objects is an important interaction motion in our daily activities. We faithfully reconstruct this motion with a single RGBD camera by a novel deep reinforcement learning method to leverage physics. Firstly, we propose object compensation control which establishes direct object control to make the network training more stable. Meanwhile, by leveraging the compensation force and torque, we seamlessly upgrade the simple point contact model to a more physical-plausible surface contact model, further improving the reconstruction accuracy and physical correctness. Experiments indicate that without involving any heuristic physical rules, this work still successfully involves physics in the reconstruction of hand-object interactions which are complex motions hard to imitate with deep reinforcement learning. Our code and data are available at this https URL.
Abstract (translated)
手操作物体是我们日常生活中的重要交互动作。我们通过一种新颖的深度强化学习方法，利用物理原理，重构了单个RGBD相机来捕捉这个动作。首先，我们提出了对象补偿控制，建立了直接物体控制，使得网络训练更加稳定。同时，通过利用补偿力和扭矩，我们将简单的点接触模型升级为更物理上合理的表面接触模型，进一步提高了重构精度和物理正确性。实验结果表明，在没有使用任何启发式物理规则的情况下，这项工作仍然成功地涉及了物理在重构手-物体交互过程中的应用，这些复杂动作很难通过深度强化学习来模仿。我们的代码和数据可在此处访问：https://www. this URL。
URL
https://arxiv.org/abs/2405.02676
