Abstract
In this work, we recover the underlying 3D structure of non-geometrically consistent scenes. We focus our analysis on hand-drawn images from cartoons and anime. Many cartoons are created by artists without a 3D rendering engine, which means that any new image of a scene is hand-drawn. The hand-drawn images are usually faithful representations of the world, but only in a qualitative sense, since it is difficult for humans to draw multiple perspectives of an object or scene 3D consistently. Nevertheless, people can easily perceive 3D scenes from inconsistent inputs! In this work, we correct for 2D drawing inconsistencies to recover a plausible 3D structure such that the newly warped drawings are consistent with each other. Our pipeline consists of a user-friendly annotation tool, camera pose estimation, and image deformation to recover a dense structure. Our method warps images to obey a perspective camera model, enabling our aligned results to be plugged into novel-view synthesis reconstruction methods to experience cartoons from viewpoints never drawn before. Our project page is https://toon3d.studio/.
Abstract (translated)
在这项工作中，我们恢复了非几何一致性场景的潜在3D结构。我们的分析重点在于手绘的动漫图像。许多动漫是由没有3D渲染引擎的艺术家创作的，这意味着任何场景的新图像都是手绘的。手绘图像通常忠实于现实世界，但只有从定性意义上说，因为人类很难 consistently绘制物体或场景的多个视角。然而，人们可以很容易地从不一致的输入中感知3D场景！在这项工作中，我们纠正了2D绘图不一致性，以恢复一个可信的3D结构，使得新扭曲的图像相互一致。我们的流程包括一个用户友好的注释工具、目标姿态估计和图像变形以恢复密集结构。我们的方法扭曲图像以遵守透视相机模型，使我们得到的结果可以插入到从未见过的观点合成重建方法中，从从未体验过的角度合成卡通。我们的项目页面是https://toon3d.studio/。
URL
https://arxiv.org/abs/2405.10320
