Abstract
Visually impaired individuals face significant challenges navigating and interacting with unknown situations, particularly in tasks requiring spatial awareness and semantic scene understanding. To accelerate the development and evaluate the state of technologies that enable visually impaired people to solve these tasks, the Vision Assistance Race (VIS) at the Cybathlon 2024 competition was organized. In this work, we present Sight Guide, a wearable assistive system designed for the VIS. The system processes data from multiple RGB and depth cameras on an embedded computer that guides the user through complex, real-world-inspired tasks using vibration signals and audio commands. Our software architecture integrates classical robotics algorithms with learning-based approaches to enable capabilities such as obstacle avoidance, object detection, optical character recognition, and touchscreen interaction. In a testing environment, Sight Guide achieved a 95.7% task success rate, and further demonstrated its effectiveness during the Cybathlon competition. This work provides detailed insights into the system design, evaluation results, and lessons learned, and outlines directions towards a broader real-world applicability.
Abstract (translated)
视力障碍者在面对未知环境时面临着重大挑战，尤其是在需要空间意识和语义场景理解的任务中。为了加速开发并评估帮助视障人士解决此类任务的技术状态，Cybathlon 2024 竞赛组织了视觉辅助竞赛（VIS）。在此项工作中，我们介绍了 Sight Guide，这是一个为 VIS 设计的可穿戴助行系统。该系统利用嵌入式计算机处理来自多个 RGB 和深度相机的数据，并通过振动信号和音频指令引导用户完成复杂的、基于现实世界的任务。我们的软件架构结合了经典机器人算法与基于学习的方法，以实现障碍物规避、物体检测、光学字符识别以及触摸屏交互等能力。在测试环境中，Sight Guide 实现了 95.7% 的任务成功率，并且在 Cybathlon 竞赛中进一步证明了其实用性。这项工作详细介绍了系统设计、评估结果和经验教训，并指出了迈向更广泛现实世界应用的方向。
URL
https://arxiv.org/abs/2506.02676
