Abstract
We present an open-source robotic framework that integrates computer vision and machine learning based inverse kinematics to enable low-cost laboratory automation tasks such as colony picking and liquid handling. The system uses a custom trained U-net model for semantic segmentation of microbial cultures, combined with Mixture Density Network for predicating joint angles of a simple 5-DOF robot arm. We evaluated the framework using a modified robot arm, upgraded with a custom liquid handling end-effector. Experimental results demonstrate the framework's feasibility for precise, repeatable operations, with mean positional error below 1 mm and joint angle prediction errors below 4 degrees and colony detection capabilities with IoU score of 0.537 and Dice coefficient of 0.596.
Abstract (translated)
我们提出一个开源机器人框架，该框架整合了计算机视觉与基于机器学习的逆运动学方法，以实现菌落挑取和液体处理等低成本实验室自动化任务。该系统采用自定义训练的U-Net模型对微生物培养物进行语义分割，并结合混合密度网络预测简单5自由度机械臂的关节角度。我们使用改装后的机械臂（升级了定制液体处理末端执行器）对该框架进行了评估。实验结果表明，该框架能够实现精确、可重复的操作，平均位置误差低于1毫米，关节角度预测误差低于4度，菌落检测的IoU分数为0.537，Dice系数为0.596。
URL
https://arxiv.org/abs/2603.20465
