Paper Reading AI Learner

MotorEase: Automated Detection of Motor Impairment Accessibility Issues in Mobile App UIs

2024-03-20 15:53:07
Arun Krishnavajjala, SM Hasan Mansur, Justin Jose, Kevin Moran


Recent research has begun to examine the potential of automatically finding and fixing accessibility issues that manifest in software. However, while recent work makes important progress, it has generally been skewed toward identifying issues that affect users with certain disabilities, such as those with visual or hearing impairments. However, there are other groups of users with different types of disabilities that also need software tooling support to improve their experience. As such, this paper aims to automatically identify accessibility issues that affect users with motor-impairments. To move toward this goal, this paper introduces a novel approach, called MotorEase, capable of identifying accessibility issues in mobile app UIs that impact motor-impaired users. Motor-impaired users often have limited ability to interact with touch-based devices, and instead may make use of a switch or other assistive mechanism -- hence UIs must be designed to support both limited touch gestures and the use of assistive devices. MotorEase adapts computer vision and text processing techniques to enable a semantic understanding of app UI screens, enabling the detection of violations related to four popular, previously unexplored UI design guidelines that support motor-impaired users, including: (i) visual touch target size, (ii) expanding sections, (iii) persisting elements, and (iv) adjacent icon visual distance. We evaluate MotorEase on a newly derived benchmark, called MotorCheck, that contains 555 manually annotated examples of violations to the above accessibility guidelines, across 1599 screens collected from 70 applications via a mobile app testing tool. Our experiments illustrate that MotorEase is able to identify violations with an average accuracy of ~90%, and a false positive rate of less than 9%, outperforming baseline techniques.

Abstract (translated)

最近的研究开始探讨自动发现和修复软件中表现出来的可访问性问题的潜力。然而,虽然最近的工作取得了重要进展,但通常偏重于识别影响某些残疾用户(如视力或听力障碍)的可访问性问题。然而,还有其他类型的用户需要软件工具支持来改善他们的体验。因此,本文旨在自动识别影响使用机械障碍物的用户的可访问性问题。为了实现这一目标,本文引入了一种名为MotorEase的新方法,该方法能够识别移动应用程序UI中影响残疾用户的可访问性问题。残疾用户通常很难使用触摸式设备,而是可能使用开关或其他辅助机制,因此UI必须支持有限的触摸手势和使用辅助设备。MotorEase利用计算机视觉和文本处理技术来实现对应用程序UI屏幕的语义理解,从而能够检测出与支持残疾用户的四种流行UI设计指南相关的违反行为,包括:(i)视觉触摸目标大小,(ii) 扩展部分,(iii) 持久元素和(iv)相邻图标视觉距离。我们在名为MotorCheck的新生基准上评估了MotorEase,该基准包含555个手动标记的上述可访问性指南的违反实例,从70个应用程序收集的1599个屏幕上进行测试。我们的实验结果表明,MotorEase能够以平均准确度~90%识别出违规行为,假阳性率小于9%,超越了基线技术。



3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot