[提交于 2026年7月9日]

我们能信任大语言模型的逻辑吗？基于图框架量化不确定性、连贯性与鲁棒性

查看论文《我们能信任大语言模型的逻辑吗？基于图框架量化不确定性、连贯性与鲁棒性》的PDF，作者：Riccardo Revalor 及另外两位作者

查看 PDF HTML（实验性）

摘要：大型语言模型（LLM）容易产生有缺陷且不忠实的推理，而自我一致性（Self-Consistency，SC）等解码策略仅评估最终答案的一致性，忽略中间步骤的逻辑有效性，因此无法检测出这些问题。这引出三个根本性问题：如何可靠地量化LLM推理中的不确定性？相较于朴素的多数投票，语义、结构和因果感知能否筛选出更忠实的推理？以及在对抗条件下推理拓扑的鲁棒性如何？为解决这些问题，我们提出GRAPHEVAL——一种基于图的推理框架，将不确定性量化（UQ）重新构建为整体推理可信度问题。我们提出了一种新颖的UQ指标——图推理连贯性评分（Graph Reasoning Coherence Score，GRCS），用于量化推理空间的语义-结构共识，并捕捉病态的模式坍缩和自信幻觉。我们发现，GRCS是唯一在更强大和较小模型上均与推理忠实度呈一致负相关的指标。此外，我们引入图自我一致性（Graph Self-Consistency，GSC）——一种基于medoid（中心点）的解码策略，以名义准确率换取推理忠实度，揭示了SC在较小模型中因不忠实的幸运猜测而虚高的程度，同时在更强大的模型中保持或提升准确率。最后，通过对抗性medoid消融实验，我们证明GSC所选路径充当「承重路径」，迫使模型偏离该路径会降低推理忠实度，并在特定场景下导致准确率下降。

提交历史

发件人：Debjit Pal [ 查看邮箱 ]

[v1]

2026年7月9日 星期四 00:45:26 UTC (1,688 KB)

[Submitted on 9 Jul 2026]

Title:Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework

View a PDF of the paper titled Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework, by Riccardo Revalor and 2 other authors

View PDF HTML (experimental)

Abstract:Large-Language Models (LLMs) can be prone to flawed and unfaithful reasoning that decoding strategies like Self-Consistency (SC) fail to detect as they evaluate only final-answer agreement while ignoring the logical validity of intermediate steps. This raises three fundamental questions: How can we reliably quantify uncertainty in LLM reasoning? Can semantic, structural, and causal awareness select more faithful reasoning compared to naïve majority voting? and How robust is reasoning topology under adversarial conditions? To address these questions, we introduce GRAPHEVAL, a graph-based reasoning framework that re-frames uncertainty quantification (UQ) as a holistic reasoning fidelity problem. We propose a novel UQ metric, Graph Reasoning Coherence Score (GRCS), that quantifies semantic-structural consensus of the reasoning space and captures pathological mode collapse and confident hallucinations. We find that GRCS is the only metric that is consistently negatively correlated with reasoning faithfulness across both more capable and smaller models. Additionally, we introduce Graph Self-Consistency (GSC), a medoid-based decoding strategy that trades nominal accuracy for reasoning fidelity, exposing the degree to which SC is inflated by unfaithful lucky guesses in smaller models, while preserving or improving accuracy in more capable ones. Finally, through adversarial medoid ablation, we demonstrate that the GSC-selected path acts as a "load-bearing path" and forcing models away from it degrades reasoning faithfulness and, in targeted cases, causes drops in accuracy.

Submission history

From: Debjit Pal [

view email

]

[v1]

Thu, 9 Jul 2026 00:45:26 UTC (1,688 KB)