摘要：工程标准是大型语言模型（LLM）极具挑战性的知识来源，因为它们具有层级化、多模态特性，并且通过图示、表格、交叉引用和规范性规则高度紧密关联。现有的检索增强生成（RAG）方法通常依赖非结构化文本检索，难以应对需要对文本条款、视觉规范和文档结构进行联合推理的复杂问题。为此，我们提出了 GEMS-RAG（Graph-grounded Engineering Multimodal Standards Retrieval-Augmented Generation，基于图接地的工程多模态标准检索增强生成），一个用于工程标准接地推理的专用框架。GEMS-RAG 融合了层级解析、细粒度视觉实体提取、多模态知识图谱构建以及基于图接地的混合检索技术。它避免将图表视作单一的整幅图像，而是将其拆解为有意义的视觉实体，并与文本条款、工程概念和法规规则建立关联。在推理阶段，该框架将稠密检索、稀疏检索与视觉检索与图扩展算法相结合，跨文本、图示及文档结构准确定位相关证据。我们在包含法规条款、技术图纸和标准化交通控制设施的大型工程标准——《统一交通控制设施手册》（MUTCD）上对 GEMS-RAG 进行了评估。结果表明，相比于纯文本、多模态以及基于图谱的 RAG 基线模型，基于图接地的多模态检索显著提升了检索有效性与问答性能。除 MUTCD 外，GEMS-RAG 还为安全关键应用中的工程规范、技术手册、监管文档和科学标准推理提供了一套通用框架。

Abstract:Engineering standards are challenging knowledge sources for large language models (LLMs) because they are hierarchical, multimodal, and densely connected through figures, tables, cross-references, and normative rules. Existing retrieval-augmented generation (RAG) methods often rely on unstructured text retrieval and struggle with questions requiring joint reasoning over textual provisions, visual specifications, and document structure. We propose GEMS-RAG (Graph-grounded Engineering Multimodal Standards Retrieval-Augmented Generation), a framework for grounded reasoning over engineering standards. GEMS-RAG combines hierarchical parsing, fine-grained visual entity extraction, multimodal knowledge graph construction, and graph-grounded hybrid retrieval. Instead of treating figures as monolithic images, it decomposes them into meaningful visual entities linked to textual provisions, engineering concepts, and regulatory rules. During inference, dense, sparse, and visual retrieval are combined with graph expansion to identify relevant evidence across text, figures, and document structure. We evaluate GEMS-RAG on the Manual on Uniform Traffic Control Devices (MUTCD), a large engineering standard containing egulations, technical drawings, and standardized traffic-control devices. Results show that graph-grounded multimodal retrieval improves retrieval effectiveness and question-answering performance over text-only, multimodal, and graph-based RAG baselines. Beyond MUTCD, GEMS-RAG offers a general framework for reasoning over engineering codes, technical manuals, regulatory documents, and scientific standards in safety-critical applications.