Abstract
Embodied AI has developed rapidly in recent years, but it is still mainly deployed in laboratories, with various distortions in the Real-world limiting its application. Traditionally, Image Quality Assessment (IQA) methods are applied to predict human preferences for distorted images; however, there is no IQA method to assess the usability of an image in embodied tasks, namely, the perceptual quality for robots. To provide accurate and reliable quality indicators for future embodied scenarios, we first propose the topic: IQA for Embodied AI. Specifically, we (1) based on the Mertonian system and meta-cognitive theory, constructed a perception-cognition-decision-execution pipeline and defined a comprehensive subjective score collection process; (2) established the Embodied-IQA database, containing over 36k reference/distorted image pairs, with more than 5m fine-grained annotations provided by Vision Language Models/Vision Language Action-models/Real-world robots; (3) trained and validated the performance of mainstream IQA methods on Embodied-IQA, demonstrating the need to develop more accurate quality indicators for Embodied AI. We sincerely hope that through evaluation, we can promote the application of Embodied AI under complex distortions in the Real-world. Project page: this https URL
Abstract (translated)
近年来，具身人工智能（Embodied AI）得到了快速发展，但主要仍局限于实验室环境之中。现实世界中的种种限制因素制约了其应用范围。传统上，图像质量评估（IQA）方法被用来预测人类对失真图像的偏好；然而，并不存在能够评估图像在具身任务中可用性的IQA方法，即机器人感知质量的评估手段。为了为未来具身场景提供准确可靠的指标，我们首先提出了“面向具身AI的图像质量评估”这一主题。 具体而言： 1. 基于默顿体系和元认知理论，构建了感知-认知-决策-执行流程，并定义了一套全面的主观评分收集方法。 2. 创建了包含超过36,000对参考/失真图像、由视觉语言模型/视觉语言行动模型/现实世界机器人提供的超过5百万条精细注释的具身-IQA数据库。 3. 在具身-IQA上训练并验证主流IQA方法的表现，展示了需要开发更准确的质量指标来适应具身AI的需求。 我们真诚地希望通过评估工作能够促进在复杂失真环境下的现实世界中应用具身人工智能。项目页面：[此链接](this https URL)
URL
https://arxiv.org/abs/2505.16815
