Paper Reading AI Learner

3Doodle: Compact Abstraction of Objects with 3D Strokes

2024-02-06 04:25:07
Changwoon Choi, Jaeah Lee, Jaesik Park, Young Min Kim

Abstract

While free-hand sketching has long served as an efficient representation to convey characteristics of an object, they are often subjective, deviating significantly from realistic representations. Moreover, sketches are not consistent for arbitrary viewpoints, making it hard to catch 3D shapes. We propose 3Dooole, generating descriptive and view-consistent sketch images given multi-view images of the target object. Our method is based on the idea that a set of 3D strokes can efficiently represent 3D structural information and render view-consistent 2D sketches. We express 2D sketches as a union of view-independent and view-dependent components. 3D cubic B ezier curves indicate view-independent 3D feature lines, while contours of superquadrics express a smooth outline of the volume of varying viewpoints. Our pipeline directly optimizes the parameters of 3D stroke primitives to minimize perceptual losses in a fully differentiable manner. The resulting sparse set of 3D strokes can be rendered as abstract sketches containing essential 3D characteristic shapes of various objects. We demonstrate that 3Doodle can faithfully express concepts of the original images compared with recent sketch generation approaches.

Abstract (translated)

自由手绘早已成为一种有效的表示物体特征的方法,但它们通常具有主观性,与现实主义表现有很大的偏差。此外,手绘图对于任意视角并不一致,使得很难捕捉到3D形状。我们提出3Dooole,它可以根据目标对象的多元视角生成描述性和视图一致的手绘图像。我们的方法基于一个理念,即一系列3D画笔可以有效地表示3D结构信息并渲染视图一致的2D手绘图。我们将2D手绘图表示为独立于视点和相关于视点的组件的并集。3D立方贝塞尔曲线表示独立于视点的3D特征线,而超四元体的轮廓表示不同视点下体积的平滑轮廓。我们的管道直接优化3D画笔原型的参数,以以完全可导的方式最小化感知损失。通过这种方式生成的稀疏集3D画笔可以渲染成包含各种物体关键3D特征形状的抽象手绘图。我们证明了3Doodle能够与最近的手绘生成方法相比,准确表达原始图像的概念。

URL

https://arxiv.org/abs/2402.03690

PDF

https://arxiv.org/pdf/2402.03690.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot