Abstract
Optical microrobots, manipulated via optical tweezers (OT), have broad applications in biomedicine. However, reliable pose and depth perception remain fundamental challenges due to the transparent or low-contrast nature of the microrobots, as well as the noisy and dynamic conditions of the microscale environments in which they operate. An open dataset is crucial for enabling reproducible research, facilitating benchmarking, and accelerating the development of perception models tailored to microscale challenges. Standardised evaluation enables consistent comparison across algorithms, ensuring objective benchmarking and facilitating reproducible research. Here, we introduce the OpTical MicroRobot dataset (OTMR), the first publicly available dataset designed to support microrobot perception under the optical microscope. OTMR contains 232,881 images spanning 18 microrobot types and 176 distinct poses. We benchmarked the performance of eight deep learning models, including architectures derived via neural architecture search (NAS), on two key tasks: pose classification and depth regression. Results indicated that Vision Transformer (ViT) achieve the highest accuracy in pose classification, while depth regression benefits from deeper architectures. Additionally, increasing the size of the training dataset leads to substantial improvements across both tasks, highlighting OTMR's potential as a foundational resource for robust and generalisable microrobot perception in complex microscale environments.
Abstract (translated)
光学微机器人通过光镊（OT）操作，在生物医学领域具有广泛的应用。然而，由于微机器人的透明性或低对比度特性以及其运行的显微环境中噪声和动态条件的存在，可靠的姿态和深度感知依然是基本挑战。一个开放的数据集对于促进可重复研究、基准测试及加速适应微尺度挑战的感知模型开发至关重要。标准化评估能够使不同算法之间的比较具有一致性和客观性，并推动可复制的研究进展。 在这里，我们推出了 OpTical MicroRobot 数据集（OTMR），这是第一个公开可用的支持光学显微镜下微机器人感知的数据集。OTMR 包含 232,881 张图像，涵盖了 18 种不同的微机器人类型和 176 种独特的姿态。我们对八个深度学习模型的性能进行了基准测试，包括通过神经架构搜索（NAS）获得的架构，并针对两个关键任务：姿态分类和深度回归。结果表明，在姿态分类任务中，视觉变换器（ViT）取得了最高的准确率；而在深度回归任务中，则从更深的架构中受益更多。此外，随着训练数据集规模的增长，两项任务中的表现都有显著提升，突显了 OTMR 作为复杂微尺度环境中稳健且具有通用性的微机器人感知基础资源的巨大潜力。
URL
https://arxiv.org/abs/2505.18303
