Abstract
We characterize the phenomenon of context-dependent affordance computation in vision-language models (VLMs). Through a large-scale computational study (n=3,213 scene-context pairs from COCO-2017) using Qwen-VL 30B and LLaVA-1.5-13B subject to systematic context priming across 7 agentic personas, we demonstrate massive affordance drift: mean Jaccard similarity between context conditions is 0.095 (95% CI: [0.093, 0.096], p < 0.0001), indicating that >90% of lexical scene description is context-dependent. Sentence-level cosine similarity confirms substantial drift at the semantic level (mean = 0.415, 58.5% context-dependent). Stochastic baseline experiments (2,384 inference runs across 4 temperatures and 5 seeds) confirm this drift reflects genuine context effects rather than generation noise: within-prime variance is substantially lower than cross-prime variance across all conditions. Tucker decomposition with bootstrap stability analysis (n=1,000 resamples) reveals stable orthogonal latent factors: a "Culinary Manifold" isolated to chef contexts and an "Access Axis" spanning child-mobility contrasts. These findings establish that VLMs compute affordances in a substantially context-dependent manner -- with the difference between lexical (90%) and semantic (58.5%) measures reflecting that surface vocabulary changes more than underlying meaning under context shifts -- and suggest a direction for robotics research: dynamic, query-dependent ontological projection (JIT Ontology) rather than static world modeling. We do not claim to establish processing order or architectural primacy; such claims require internal representational analysis beyond output behavior.
Abstract (translated)
我们对视觉语言模型（VLMs）中上下文依赖的可及性计算现象进行了描述。通过使用Qwen-VL 30B和LLaVA-1.5-13B在COCO-2017数据集中进行的大规模计算研究（n=3,213个场景-上下文对），并针对七个代理角色系统性地进行上下文启动，我们展示了巨大的可及性漂移：不同条件下的平均Jaccard相似度为0.095（95%置信区间：[0.093, 0.096]，p < 0.0001），表明超过90%的词汇场景描述依赖于上下文。句子级别的余弦相似性证实了语义层面存在显著漂移（平均值为0.415，58.5%为上下文相关）。通过随机基线实验（2,384次推理运行，在四种温度和五种子集下进行），确认这种漂移反映了真正的上下文效应而非生成噪声：在同一启动条件下的变异性显著低于跨启动条件的变异性。Tucker分解结合自助稳定性分析（n=1,000重采样）揭示了稳定的正交潜在因素：“烹饪流形”仅在厨师情境下出现，“访问轴”贯穿儿童与成人移动性的对比变化。这些发现表明VLMs以显著上下文依赖的方式计算可及性——词汇层面的变化比语义层面更加明显，这反映出表面词汇比深层含义更容易随上下文变化而改变。此外，这些发现为机器人研究提供了一个方向：动态、查询相关的本体投影（即时本体论）而非静态世界模型。我们不主张确定处理顺序或架构优先性；这类断言需要超越输出行为的内部表征分析来支持。
URL
https://arxiv.org/abs/2603.04419
