论文研究 arXiv cs.CV Thu, 15 Oct 2026 正文 ok 231 words

迈向以数据为中心理解多模态大语言模型的空间推理

英文标题:Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models

这篇综述聚焦多模态大语言模型(MLLM)在空间推理与空间感知方面的局限,以及通过后训练提升相关能力的数据驱动方法。论文系统考察了从监督微调到强化学习的空间推理训练方案,重点分析所使用的数据集、数据模态、标注方式和评估方法。作者将现有研究归纳为数据集设计、数据集构建与模型评估三类,并总结了构建固定空间标注的四种主要方法。论文还指出,空间推理领域亟需推进统一动态标注、高效推理、评估标准化、更广的空间范围及以人为中心的空间交互。
打开原文相关 6.4 · 机会 1
media2.1 KB media612 B media380 B media9.1 KB media32.7 KB media22.8 KB media24.2 KB media34.7 KB

中文

[提交于 2026 年 10 月 13 日]

标题:迈向以数据为中心理解多模态大语言模型的空间推理

查看 Mahdieh Mallahnezhad 及另外 4 位作者撰写的论文《迈向以数据为中心理解多模态大语言模型的空间推理》PDF。

查看 PDF HTML(实验版)

摘要:多模态大语言模型(MLLM)已被广泛应用于具身智能体和机器人领域的视觉任务。然而,MLLM 在空间推理和感知方面仍存在局限。后训练技术已展现出提升这些能力的潜力,但针对这类方法的数据驱动型综述仍较为有限。本综述考察了针对空间推理任务对预训练 MLLM 进行微调的方法,涵盖从监督学习到强化学习的不同路径,并重点关注所使用的数据集。我们研究了数据集构建、数据模态与标注,以及最终模型评估等主题。基于这些主题,我们将相关研究划分为三类:数据集设计、数据集构建和评估。我们发现,构建固定空间标注主要有四种方法:规则与模板、LLM 提示、程序化生成,以及人类参与闭环(human-in-the-loop)。我们还观察到评估方法存在显著差异,包括不同的评分方法、容差指标和基准类型。最后,我们识别出未来研究的机会方向,包括统一动态标注、高效空间推理、评估标准化、扩展空间范围,以及以人为中心的空间交互。

提交历史

来自:Mahdieh Mallahnezhad [

查看电子邮件

]

[v1]

2026 年 10 月 13 日,星期二,19:42:30 UTC(2,739 KB)

English / 原文

[Submitted on 13 Oct 2026]

Title:Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models

View a PDF of the paper titled Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models, by Mahdieh Mallahnezhad and 4 other authors

View PDF HTML (experimental)

Abstract:Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitations in spatial reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such approaches. In this survey, we consider approaches to fine-tuning pretrained MLLMs for spatial reasoning tasks, ranging from supervised to reinforcement learning, with an emphasis on the datasets used. We investigate themes of dataset construction, data modality and annotation, and evaluation of the resulting models. Based on these themes, we categorize research into three groups: dataset design, dataset construction and evaluation. We find four main methods for constructing fixed spatial annotations: rules and templates, LLM prompting, procedural generation, and human-in-the-loop. We also observe substantial variation in evaluation approaches, including different scoring methods, tolerance metrics, and benchmark types. Finally, we identify opportunities for future research in unified dynamic annotation, efficient spatial reasoning, standardization of evaluation, expanded spatial range, and human-centered spatial interactions.

Submission history

From: Mahdieh Mallahnezhad [

view email

]

[v1]

Tue, 13 Oct 2026 19:42:30 UTC (2,739 KB)