Paper Reading AI Learner

Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor

2024-05-12 10:00:10
Jianhua Shan, Yuhao Sun, Shixin Zhang, Fuchun Sun, Zixi Chen, Zirong Shen, Cesare Stefanini, Yiyong Yang, Shan Luo, Bin Fang

Abstract

Deformable object manipulation is a classical and challenging research area in robotics. Compared with rigid object manipulation, this problem is more complex due to the deformation properties including elastic, plastic, and elastoplastic deformation. In this paper, we describe a new deformable object manipulation method including soft contact simulation, manipulation learning, and sim-to-real transfer. We propose a novel approach utilizing Vision-Based Tactile Sensors (VBTSs) as the end-effector in simulation to produce observations like relative position, squeezed area, and object contour, which are transferable to real robots. For a more realistic contact simulation, a new simulation environment including elastic, plastic, and elastoplastic deformations is created. We utilize RL strategies to train agents in the simulation, and expert demonstrations are applied for challenging tasks. Finally, we build a real experimental platform to complete the sim-to-real transfer and achieve a 90% success rate on difficult tasks such as cylinder and sphere. To test the robustness of our method, we use plasticine of different hardness and sizes to repeat the tasks including cylinder and sphere. The experimental results show superior performances of deformable object manipulation with the proposed method.

Abstract (translated)

变形对象操作是一个经典的机器人研究领域。与刚性对象操作相比,由于变形特性包括弹性、塑性和弹性塑性变形,这个问题更加复杂。在本文中,我们描述了一种新的变形对象操作方法,包括软接触仿真、操作学习和仿真到实物的转移。我们提出了一个利用基于视觉的触觉传感器(VBTSs)作为末端执行器在仿真中产生相对位置、挤压区域和物体轮廓等观察值的全新方法。为了实现更加真实的接触仿真,我们创建了一个包括弹性、塑性和弹性塑性变形的新仿真环境。我们使用强化学习策略对仿真中的代理进行训练,并应用专家演示来解决具有挑战性的任务。最后,我们构建了一个真实实验平台,以实现仿真到实物的转移,并在困难任务(如圆柱体和球体)上实现90%的成功率。为了测试我们方法的稳健性,我们使用不同硬度和大小的塑料来重复包括圆柱体和球体的任务。实验结果表明,与所提出的方法相比,变形对象操作具有卓越的性能。

URL

https://arxiv.org/abs/2405.07237

PDF

https://arxiv.org/pdf/2405.07237.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot