Abstract
Vision Language Models (VLMs) are increasingly adopted as central reasoning modules for embodied agents. Existing benchmarks evaluate their capabilities under ideal, well-lit conditions, yet robust 24/7 operation demands performance under a wide range of visual degradations, including low-light conditions at night or in dark environments--a core necessity that has been largely overlooked. To address this underexplored challenge, we present DarkEQA, an open-source benchmark for evaluating EQA-relevant perceptual primitives under multi-level low-light conditions. DarkEQA isolates the perception bottleneck by evaluating question answering from egocentric observations under controlled degradations, enabling attributable robustness analysis. A key design feature of DarkEQA is its physical fidelity: visual degradations are modeled in linear RAW space, simulating physics-based illumination drop and sensor noise followed by an ISP-inspired rendering pipeline. We demonstrate the utility of DarkEQA by evaluating a wide range of state-of-the-art VLMs and Low-Light Image Enhancement (LLIE) models. Our analysis systematically reveals VLMs' limitations when operating under these challenging visual conditions. Our code and benchmark dataset will be released upon acceptance.
Abstract (translated)
视觉语言模型（VLM）作为具身代理的核心推理模块被越来越多地采用。现有的基准测试主要评估它们在理想条件下的表现，如光线充足的环境，然而，为了实现全天候可靠运行，在包括夜间或黑暗环境中低光照等广泛视觉退化情况下的性能是必不可少的，这一点却被很大程度上忽视了。为了解决这个未充分探索的挑战，我们提出了DarkEQA，这是一个开源基准测试平台，用于在多级低光条件下评估与问答相关的感觉基本单元的表现。 DarkEQA通过在受控退化的第一人称视角观察中进行问题回答来隔离感知瓶颈，从而允许归因于稳健性的分析。DarkEQA的关键设计特性是物理保真度：视觉退化是在线性RAW空间中建模的，模拟了基于物理学的照明下降和传感器噪声，并通过一个启发自ISP（图像信号处理）的渲染管线进行。 我们通过评估一系列最先进的VLMs和低光图像增强（LLIE）模型来展示了DarkEQA的实用性。我们的分析系统地揭示了视觉语言模型在这些具有挑战性的视觉条件下的局限性。我们的代码和基准测试数据集将在被接受后发布。
URL
https://arxiv.org/abs/2512.24985
