Abstract
Autoregressive (AR) vision-language models (VLMs) have long dominated multimodal understanding, reasoning, and graphical user interface (GUI) grounding. Recently, discrete diffusion vision-language models (DVLMs) have shown strong performance in multimodal reasoning, offering bidirectional attention, parallel token generation, and iterative refinement. However, their potential for GUI grounding remains unexplored. In this work, we evaluate whether discrete DVLMs can serve as a viable alternative to AR models for GUI grounding. We adapt LLaDA-V for single-turn action and bounding-box prediction, framing the task as text generation from multimodal input. To better capture the hierarchical structure of bounding-box geometry, we propose a hybrid masking schedule that combines linear and deterministic masking, improving grounding accuracy by up to 6.1 points in Step Success Rate (SSR) over the GUI-adapted LLaDA-V trained with linear masking. Evaluations on four datasets spanning web, desktop, and mobile interfaces show that the adapted diffusion model with hybrid masking consistently outperforms the linear-masked variant and performs competitively with autoregressive counterparts despite limited pretraining. Systematic ablations reveal that increasing diffusion steps, generation length, and block length improves accuracy but also increases latency, with accuracy plateauing beyond a certain number of diffusion steps. Expanding the training data with diverse GUI domains further reduces latency by about 1.3 seconds and improves grounding accuracy by an average of 20 points across benchmarks. These results demonstrate that discrete DVLMs are a promising modeling framework for GUI grounding and represent an important step toward diffusion-based GUI agents.
Abstract (translated)
自回归（AR）视觉语言模型（VLMs）长期占据多模态理解、推理及图形用户界面（GUI）定位的主导地位。近期，离散扩散视觉语言模型（DVLMs）在多模态推理中展现出强劲性能，具备双向注意力、并行令牌生成与迭代优化等优势。然而，其在GUI定位领域的潜力尚未得到探索。本研究评估离散DVLMs能否作为AR模型在GUI定位任务中的可行替代方案。我们针对单轮动作与边界框预测任务适配LLaDA-V模型，将任务框架化为多模态输入的文本生成。为更好捕捉边界框几何的层次化结构，我们提出一种结合线性与确定性掩码的混合掩码调度策略，相较于采用线性掩码的GUI适配LLaDA-V，其定位精度在步骤成功率（SSR）指标上最高提升6.1个百分点。在涵盖网页、桌面及移动界面的四个数据集上的评估显示，采用混合掩码的扩散模型持续优于线性掩码变体，并在有限预训练条件下达到与自回归模型相当的性能。系统化消融实验表明，增加扩散步数、生成长度与块长度可提升精度，但也会增加延迟，且精度在扩散步数超过一定阈值后趋于饱和。通过扩展涵盖多样化GUI领域的训练数据，可将延迟进一步降低约1.3秒，并在各基准测试中平均提升20个定位精度点。这些结果证明离散DVLMs是GUI定位任务中极具潜力的建模框架，并为基于扩散模型的GUI智能体迈出了重要一步。
URL
https://arxiv.org/abs/2603.26211
