Abstract
Recent research has begun to examine the potential of automatically finding and fixing accessibility issues that manifest in software. However, while recent work makes important progress, it has generally been skewed toward identifying issues that affect users with certain disabilities, such as those with visual or hearing impairments. However, there are other groups of users with different types of disabilities that also need software tooling support to improve their experience. As such, this paper aims to automatically identify accessibility issues that affect users with motor-impairments. To move toward this goal, this paper introduces a novel approach, called MotorEase, capable of identifying accessibility issues in mobile app UIs that impact motor-impaired users. Motor-impaired users often have limited ability to interact with touch-based devices, and instead may make use of a switch or other assistive mechanism -- hence UIs must be designed to support both limited touch gestures and the use of assistive devices. MotorEase adapts computer vision and text processing techniques to enable a semantic understanding of app UI screens, enabling the detection of violations related to four popular, previously unexplored UI design guidelines that support motor-impaired users, including: (i) visual touch target size, (ii) expanding sections, (iii) persisting elements, and (iv) adjacent icon visual distance. We evaluate MotorEase on a newly derived benchmark, called MotorCheck, that contains 555 manually annotated examples of violations to the above accessibility guidelines, across 1599 screens collected from 70 applications via a mobile app testing tool. Our experiments illustrate that MotorEase is able to identify violations with an average accuracy of ~90%, and a false positive rate of less than 9%, outperforming baseline techniques.
Abstract (translated)
最近的研究开始探讨自动发现和修复软件中表现出来的可访问性问题的潜力。然而，虽然最近的工作取得了重要进展，但通常偏重于识别影响某些残疾用户（如视力或听力障碍）的可访问性问题。然而，还有其他类型的用户需要软件工具支持来改善他们的体验。因此，本文旨在自动识别影响使用机械障碍物的用户的可访问性问题。为了实现这一目标，本文引入了一种名为MotorEase的新方法，该方法能够识别移动应用程序UI中影响残疾用户的可访问性问题。残疾用户通常很难使用触摸式设备，而是可能使用开关或其他辅助机制，因此UI必须支持有限的触摸手势和使用辅助设备。MotorEase利用计算机视觉和文本处理技术来实现对应用程序UI屏幕的语义理解，从而能够检测出与支持残疾用户的四种流行UI设计指南相关的违反行为，包括：（i）视觉触摸目标大小，(ii) 扩展部分，(iii) 持久元素和（iv）相邻图标视觉距离。我们在名为MotorCheck的新生基准上评估了MotorEase，该基准包含555个手动标记的上述可访问性指南的违反实例，从70个应用程序收集的1599个屏幕上进行测试。我们的实验结果表明，MotorEase能够以平均准确度~90%识别出违规行为，假阳性率小于9%，超越了基线技术。
URL
https://arxiv.org/abs/2403.13690
