摘要：视觉语言模型（VLM）在需要针对视觉量进行推理的图表问题上仍不可靠，这一弱点通常被归因于推理能力缺陷，并通过增加推理监督来解决。我们探讨，困难究竟在于推理本身，还是在于推理所依赖的更基础技能：读取图中绘制元素（\emph{感知}）、定位这些元素并将其与标签绑定（\emph{定位}），以及执行排序、求总和与求差等单步计算（\emph{简单推理}）。我们提出 \textbf{ChartProbe}，这是一个诊断框架，其测试探针直接由渲染每张图表的代码生成，因此每个标准答案在构造上都是精确的，无需人工标注，并且能够将每次失败归因于单一技能。ChartProbe 实现了一项既有工作尚未尝试的干预：我们不合成复杂推理数据，而是完全不使用复杂问题和推理轨迹，分别针对单一简单技能进行微调，并测量其向留出的复杂推理问题的迁移能力。在三种开放权重 VLM 中，仅监督这些简单技能，便可使模型在从未训练过的复杂推理问题上取得大幅提升：当这些技能较弱且模型能够学会读取图像时，对其进行训练能够在不付出任何推理数据成本的情况下恢复相当一部分复杂推理能力。该提升在三种分布外场景中均成立：未见过的图表类型（饼图）、与我们的图像和模板相互独立的人工编写基准（ChartQA），以及非图表视觉领域（CLEVR）。因此，复杂视觉推理可以在没有复杂推理监督的情况下得到改善。

Abstract:Vision-language models (VLMs) remain unreliable on chart questions that require reasoning over visual quantities, and this weakness is usually attributed to a reasoning deficit and addressed with more reasoning supervision. We ask whether the difficulty lies in reasoning itself, or in the simpler skills that reasoning operates on: reading the plotted elements (\emph{perception}), locating them and binding them to their labels (\emph{grounding}), and performing single-step computations such as ranking, totals, and differences (\emph{simple reasoning}). We introduce \textbf{ChartProbe}, a diagnostic framework whose probes are generated directly from the code that renders each chart, so every gold answer is exact by construction, needs no human annotation, and attributes each failure to a single skill. ChartProbe enables an intervention prior work does not attempt: instead of synthesizing complex-reasoning data, we withhold complex questions and reasoning traces entirely, fine-tune on one simple skill at a time, and measure transfer to held-out complex-reasoning questions. Across three open-weight VLMs, supervising the simpler skills alone produces large gains on complex-reasoning questions the model never trained on: where these skills are weak and the model can be taught to read the image, training them recovers much of complex reasoning at no reasoning-data cost. The gains hold across three out-of-distribution settings: an unseen chart type (pie charts), a human-written benchmark disjoint from our images and templates (ChartQA), and a non-chart visual domain (CLEVR). Complex visual reasoning can therefore improve without complex-reasoning supervision.