Agents · 推理

WFM：面向复杂 Agent 推理的 Wiki Foundation Model

Junnan Dong、Linhao Luo、Senlei Zhang、Gong Chen、Taian Guo、Yifei Yu 等。

策展人观点

腾讯优图实验室和莫纳什大学的 Junnan Dong、Linhao Luo、Senlei Zhang 及其同事提出了 WFM，即 Wiki Foundation Model，用于编码和检索 LLM Wiki 知识库。这类知识库将高语义密度的 Markdown 文档与显式链接结合起来。

核心要点

01

Wiki Graph 模式。 该模式在保留高密度文本语义的同时，维持显式的链接拓扑结构。

02

查询条件化聚合。 消息传递采用查询条件化注意力机制，并引入注意力方差正则化。

03

分布式训练。 一种 NCCL 边界交换协议使用固定形状的 GPU 到 GPU 集合通信，避免 CPU 序列化，训练速度提升 10.5 倍；该方法还在五项 Agent 记忆和多跳推理基准上取得了强劲结果。

摘要

现实世界中的 Agent 从根本上需要持久化的非参数知识，以支持动态推理，即长期记忆和检索增强生成。尽管图结构已展现出在提供结构化证据方面的可靠优势，但稀疏图表示天然限制了复杂 Agent 工作流所需的机器可读性和语义密度。受这一限制推动，整个行业正从传统稀疏图转向 LLM Wiki：这是一种面向 Agent 的原生知识表示，将高密度文档上下文与包含多层拓扑链接的 Markdown 文件相结合。然而，如此丰富的语义难以通过传统稀疏图嵌入进行参数化和编码。与此同时，使用现有图编码器学习 LLM Wiki 可能带来过高的分布式系统开销，从而阻碍其在大规模商业场景中的部署。

为此，我们提出了一种新范式——Wiki Foundation Model，即 WFM，专为可扩展的、面向 Agent 的表示与检索而设计。具体而言，(i) 我们形式化了一种 Wiki Graph 模式，可无缝连接细粒度结构与高密度上下文，在维持显式拓扑结构的同时保留连续语义；(ii) 我们为丰富的 Wiki 消息传递设计了查询条件化注意力聚合机制，并加入显式注意力方差正则化；(iii) 我们构建了一种基础设施级 NCCL 边界交换协议，该协议将静态分区索引预先提升，并利用固定形状的 GPU 到 GPU 集合通信，绕过 CPU 序列化和内存拷贝开销。在五项长期 Agent 记忆与多跳推理基准上的广泛评估表明，WFM 取得了显著性能，同时在分布式集群上实现了 10.5 倍的训练加速。

Agents · Reasoning

WFM: Wiki Foundation Model for Complex Agentic Reasoning

Junnan Dong, Linhao Luo, Senlei Zhang, Gong Chen, Taian Guo, Yifei Yu, et al.

The curator’s take

Junnan Dong, Linhao Luo, Senlei Zhang and colleagues at Tencent Youtu Lab and Monash University propose WFM, a Wiki Foundation Model for encoding and retrieving LLM Wiki knowledge bases that pair dense markdown documents with explicit links.

Key points

01

Wiki Graph schema. The schema keeps explicit link topology alongside dense text semantics.

02

Query-conditioned aggregation. Message passing uses query-conditioned attention with an attention-variance regularizer.

03

Distributed training. An NCCL boundary exchange protocol with fixed-shape GPU-to-GPU collectives avoids CPU serialization and gives 10.5x faster training, with strong results on five agent memory and multi-hop benchmarks.

Abstract

Real-world agents fundamentally require persistent non-parametric knowledge for dynamic reasoning, i.e., long-term memory and retrieval-augmented generation. While graphs have shown reliable advantages in providing structured evidence, the sparse graph representations naturally restrict machine readability and semantic density required for complex agentic workflows. Driven by this limitation, the entire industry is witnessing a paradigm shift from traditional sparse graphs to LLM Wiki, an agent-native knowledge representation that couples dense document contexts with markdown files containing multi-layered topological linkages. However, parameterizing such rich semantics is challenging to encode dense textual contexts using traditional sparse graph embeddings. Moreover, learning LLM Wiki with existing graph encoders could overwhelm distributed system overheads that hinder deployment in large-scale commercial scenarios. To this end, we propose a novel paradigm Wiki Foundation Model, i.e., WFM, tailored for scalable, agent-native representation and retrieval. Specifically, (i) we formalize a Wiki Graph schema that seamlessly bridges fine-grained structures with dense contexts, maintaining explicit topologies alongside continuous semantics; (ii) A query-conditioned attentive aggregation is tailored for rich wiki message passing and explicit attention variance regularization; (iii) We engineer an infrastructural NCCL boundary exchange protocol that hoists static partition indices and leverages fixed-shape GPU-to-GPU collectives, bypassing CPU serialization and memory copy overheads. Extensive evaluations across five long-term agent memory and multi-hop reasoning benchmarks demonstrate the remarkable performance of WFM, while achieving a 10.5 times training acceleration on distributed clusters.