Abstract
Despite recent advancements in Multi-modal Large Language Models (MLLMs) on diverse understanding tasks, these models struggle to solve problems which require extensive multi-step reasoning. This is primarily due to the progressive dilution of visual information during long-context generation, which hinders their ability to fully exploit test-time scaling. To address this issue, we introduce Vision-aligned Latent Reasoning (VaLR), a simple, yet effective reasoning framework that dynamically generates vision-aligned latent tokens before each Chain of Thought reasoning step, guiding the model to reason based on perceptual cues in the latent space. Specifically, VaLR is trained to preserve visual knowledge during reasoning by aligning intermediate embeddings of MLLM with those from vision encoders. Empirical results demonstrate that VaLR consistently outperforms existing approaches across a wide range of benchmarks requiring long-context understanding or precise visual perception, while exhibiting test-time scaling behavior not observed in prior MLLMs. In particular, VaLR improves the performance significantly from 33.0% to 52.9% on VSI-Bench, achieving a 19.9%p gain over Qwen2.5-VL.
Abstract (translated)
尽管在多模态大型语言模型（MLLM）处理多样化理解任务方面取得了近期进展，但这些模型仍然难以解决需要复杂多步推理的问题。这主要是因为在长时间上下文生成过程中视觉信息逐渐稀释，从而限制了它们充分利用测试时放大的能力。为了解决这个问题，我们引入了一种简单而有效的推理框架——视觉对齐潜在推理（VaLR）。该框架在每个Chain of Thought推理步骤之前动态生成与视觉对齐的潜在令牌，引导模型根据感知线索在潜在空间中进行推理。具体来说，VaLR通过将MLLM中间嵌入与视觉编码器中的嵌入对齐来训练以在推理过程中保留视觉知识。 实证结果显示，VaLR在一系列需要长时间上下文理解和精确视觉感知的任务上始终优于现有方法，并且在测试时放大行为方面表现出不同于先前MLLM的特性。特别是在VSI-Bench上，VaLR将性能显著提高至从33.0%到52.9%，比Qwen2.5-VL高出19.9个百分点。 通过这些改进，VaLR证明了它不仅在广泛的基准测试中具有强大的通用性，而且还能有效地解决当前MLLM面临的关键挑战。
URL
https://arxiv.org/abs/2602.04476
