看见、说出,却未能使用:从可报告的空间事实到多模态大语言模型中的可用状态
英文标题:Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models
研究指出,多模态大语言模型即使能够正确报告空间事实,也不一定能在后续推理中实际利用这些事实。论文提出包含 23,196 个输入的 \textsc{SpaceConflict} 基准,用于评估空间状态的构建与使用,覆盖局部事实绑定、关系组合、跨观察一致性及变换下的状态判断四个层级。实验发现,模型能够从视觉证据中恢复初始状态,却常在需要利用该状态完成变换推理时失败;例如 Qwen3.5-9B 在 100 个初始状态恢复正确的序列中,有 50 个未能完成完整变换,而显式提供状态后这 50 个案例全部得到修复。研究提出 Operational State Supervision(OSS),通过监督任务相关空间状态、变换轨迹及跨上下文共享事实对齐,尤其提升依赖状态组织与使用的 L3、L4 层级表现。
640 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 10 月 2 日]
标题:看见、说出,却未能使用:从可报告的空间事实到多模态大语言模型中的可用状态
查看 Jinchang Zhang 及另外 1 位作者撰写的论文《看见、说出,却未能使用:从可报告的空间事实到多模态大语言模型中的可用状态》的 PDF。
摘要:能够正确报告某项空间事实的多模态大语言模型,并不一定会在后续推理中使用该事实。为研究这一差异,我们提出 \textsc{SpaceConflict},这是一个包含 23,196 个输入、用于空间状态构建与使用的基准。在统一的“支持(Supported)/矛盾(Contradictory)/未知(Unknown)”判断接口下,该基准覆盖局部事实绑定(L1)、关系组合(L2)、跨观察一致性(L3),以及变换下的状态判断(L4)。在同一个世界中提出直接状态查询、完整变换查询和显式初始状态查询,揭示出“可获得性—可利用性鸿沟”:模型能够从视觉证据中恢复初始状态,却会在必须由该状态驱动变换时失败。对于 Qwen3.5-9B,在初始状态被正确恢复的 100 个序列中,有 50 个未能完成完整变换;而显式提供状态后,这 50 个案例全部得到修复。该鸿沟会随模型规模增大而缩小,但并未消失。因此,我们提出操作状态监督(Operational State Supervision,OSS),对任务相关空间状态及其变换轨迹进行监督,并对齐跨上下文的共享事实。OSS 在配对判断的准确率上带来提升,其中对依赖状态组织和使用的 L3 与 L4 层级提升最为显著。因此,评估多模态空间推理时,不仅需要问模型能否看见并说出某项空间事实,还应考察该事实是否会在后续计算中成为可用状态。
提交历史
来自:Jinchang Zhang [
]
[v1]
2026 年 10 月 2 日,周五,06:12:56 UTC(3,961 KB)
English / 原文
[Submitted on 2 Oct 2026]
Title:Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models
View a PDF of the paper titled Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models, by Jinchang Zhang and 1 other authors
Abstract:A multimodal large language model that correctly reports a spatial fact does not necessarily use that fact in subsequent reasoning. To study this distinction, we introduce \textsc{SpaceConflict}, a benchmark of 23{,}196 inputs for the construction and use of spatial state. Under a unified Supported/Contradictory/Unknown judgment interface, it covers local fact binding (L1), relational composition (L2), cross-observation consistency (L3), and state judgment under transformation (L4). Posing a direct-state query, a full-transformation query, and an explicit-initial-state query on the same world reveals an availability--utilization gap: models recover the initial state from visual evidence yet fail when that state must drive a transformation. For Qwen3.5-9B, 50 of 100 sequences with a correctly recovered initial state fail the full transformation, and supplying the state explicitly repairs all 50; the gap narrows with scale but does not close. We therefore propose Operational State Supervision (OSS), which supervises task-relevant spatial states and their transformation trajectories and aligns shared facts across contexts. OSS improves paired accuracy on matched judgments most on L3 and L4, the levels that depend on organizing and using state. Evaluating multimodal spatial reasoning thus requires asking not only whether a model can see and state a spatial fact, but whether that fact becomes a usable state in subsequent computation.
Submission history
From: Jinchang Zhang [
]
[v1]
Fri, 2 Oct 2026 06:12:56 UTC (3,961 KB)