🔶 AI Agent 需要的是记录系统，而不只是仪表盘
英文标题：🔶 AI Agents Need a System of Record, Not Just a Dashboard
随着 AI Agent 开始登录供应商门户、审批发票、创建工单并向其他 Agent 交接任务，传统的仪表盘和日志已不足以支撑治理与追责。Multisynapse 提出“Agent System of Record（Agent 记录系统）”，将 Agent 版本、权限策略、工具调用、成本、评测结果和审批证据统一记录。该平台支持上线前评测、人工审批、可篡改检测的审计轨迹，以及对 NIST AI RMF 和 ISO/IEC 42001 的证据映射。对于开发者和企业而言，这意味着可在不重建现有 Agent 的前提下，通过 OpenTelemetry 接入可审计、可回滚、适合多团队协作的 Agent 治理能力。
5.4 KB
4.5 KB
3.4 KB
中文
你的 Agent 已不再只是起草内容，而是开始执行实际操作。它们会登录供应商门户、审批发票、提交工单，并将工作交给其他 Agent。大多数团队仍像运营 Web 服务一样运营它们：发布、查看仪表盘，并在出现问题时读取日志。
这种方式有效，直到第一次有人提出仪表盘无法回答的问题：是哪一个版本的 Agent 支付了那张发票？谁批准它上线？它当天被允许访问什么，以及你能否证明？
观察不等于治理
可观测性告诉你发生了什么。治理决定什么事情被允许发生，并保留相关证明。对于只负责回答问题的软件而言，观察已经足够；但对于代表你采取行动的软件，你两者都需要，而且它们必须保持一致。
几分钟内得到答案，而非展开调查
设想这样一个场景：周一早晨，财务部门告诉你，一个 Agent 在周末向同一供应商支付了两次款项。使用仪表盘时，你需要开始阅读日志。使用 Multisynapse 时，你打开该次运行，答案已经在那里。
- 哪个 Agent、哪个版本。 每个 Agent 都位于同一个目录中，每次运行都会标明其运行的准确版本。任何编辑都会保持为草稿，直到有人将其晋升。
- 它被允许做什么。 每一次工具调用都会在发起时依据你的策略进行检查，并使用启动该次运行的人员身份。相关决策会与该调用一同保存在记录中。
- 它实际做了什么，以及花费了多少。 每次模型调用、工具调用和策略决策都会在源头被记录一次。
- 它是否足够好、可以发布。 该版本在上线前经过评测，上线后也会根据你设定的标准进行抽样检查。
如果新版本就是问题所在，回滚只需一次决策，而不是重新部署。
这就是我们所说的 Agent System of Record：一个能够了解每个 Agent、其被允许执行的操作及其实际执行行为的统一位置。无论 Agent 使用哪种框架或模型供应商，它都可以工作，因此你可以切换模型，而不会丢失历史记录、策略或评分。（我们在 You Didn't Deploy the Agent You Evaluated 中解释了，为什么版本必须从评测一直伴随 Agent 到执行阶段。）
上线前完成评测
一个 Agent 的新版本需要凭实力进入生产环境。它会基于由你自身真实运行数据构建的数据集进行测试，外部环境将被重放而非真正触碰，因此一次评测绝不会真的支付一张发票。然后，它还必须通过你定义的门槛：质量、错误率和成本。
我们严格规定谁可以进行评分：
- 裁判不能给自己的作业打分。 一次晋升可以要求评审模型来自与被评测 Agent 不同的模型供应商。
- 裁判必须擅长评判。 一次晋升还可以要求评审模型已根据人工标注进行测量，并达到质量门槛。如果其表现发生漂移，它将不再符合资格。
- 由人作出决定。 批准一个版本、将其晋升，或启用自动晋升，都需要一名已登录人员操作。API key，或持有 API key 的 Agent，均无法执行这些操作。
当你希望一次评测大量运行时，Multisynapse 可通过你的模型供应商批处理接口发起评审调用，按供应商的批处理价格计费，而不会改变向评审模型提出的问题。
可交给审计人员的证据
当监管机构、客户或你自己的风险团队询问 Agent 做了什么时，答案应当是一份记录，而不是一次重建。
- 可发现篡改的历史记录。 审批、权限变更、晋升和策略决策都会进入审计轨迹；其中被修改或删除的条目将会显现。
- 签名运行溯源。 一次运行可以携带经签名的清单，说明其由什么产生：版本、所获得的上下文以及可使用的工具。
- 框架映射。 Multisynapse 将这些证据映射到 NIST AI Risk Management Framework 和 ISO/IEC 42001。它会逐项控制展示你的符合性态势，区分自动检查的内容与需要人工证明的内容，并导出供审核使用的材料包。
我们特意使用 符合性态势 这一说法。这些框架描述的是良好实践。软件无法据此为你提供认证，但它可以在一个位置展示相关证据。
为团队的实际工作方式而构建
只适用于三人团队的治理不是真正的治理。Multisynapse 从底层开始就是多租户的，最新版本新增了大型组织最先提出的控制能力：
- 按需知悉的项目。 组织可以将成员的访问权限限制在其被添加到的项目中，不会意外获得其他所有项目的访问权限。项目组可以让一个团队通过一次授权覆盖整组项目，包括日后新增的项目。
- 职责分离。 编写 Agent、管理其评测，以及批准其晋升，可以由三名不同的人分别承担，每个人只拥有恰好对应的那部分权限。
- 移除就意味着移除。 将某人从组织中移除，会终止其在该组织中持有的所有角色。
- 谨慎共享的助手。 内置助手可回答有关你的 Agent、运行和成本的问题，并为你准备待确认的变更。默认情况下，其对话是私密的。你可以与团队成员共享某个对话；当对方在对话中添加内容时，助手会使用 对方的 权限，而不是你的权限。它还可以读取管理员选择向其提供的工具，但仅限于那些声明自身为只读的工具。
上个月发布的版本将认证范围从单个 Agent 扩展至完整对话，详见 Certify the Conversation, Not Just the Agent。
可与现有数据发送方式协作
你无需重建 Agent 就能获得上述能力。Multisynapse 支持 OpenTelemetry——这一面向追踪和日志的开放标准，其中包括正在形成的生成式 AI 约定。将现有导出器指向它后，你的 Agent 追踪数据和应用日志将并排进入系统，并关联到相同的运行记录。
你的数据在进入过程中受到保护：
- 在数据离开前应用你的规则。 SDK 允许你在任何记录被发送前将其删除或编辑。
- 在入口处应用你的规则。 每个组织都可以设置自己的脱敏规则，在数据到达时执行，并叠加内置的个人数据检测能力。
- 能够汇总的支出。 成本根据每次调用的实际使用量，按当时有效价格计算。你可以按项目拆分使用量，并将其导出到财务团队已经在使用的电子表格中。
在成本或错误图表上选择任意一点，即可直接进入其背后的运行记录。
一览
每次运行都会经过检查和记录，其结果会被评分，而下一个版本只有在人员决定其应当上线时才会进入生产环境。
我们的构建方式
我们先测量，再构建。当一项功能在当前客户的运行规模下成本高于其带来的回报时，我们不会为了勾选一项功能清单而发布它。我们会建立一项检查，以便在适当时机到来时提醒我们。当我们发现自身产品存在缺口时，我们会从根本原因入手修复，并新增测试以确保问题不会再次出现。
如果你的 Agent 代表你在你的系统中采取行动，动用你的资金或接触客户数据，那么你需要的不只是查看它们做过什么。你需要一份记录：它们被允许做什么、是谁批准的，以及你如何知道它确实有效。
这就是我们构建的产品。如希望在你自己的 Agent 上查看效果，请发送邮件至 info@anuclei.com，我们将安排演示讲解。
English / 原文
Your agents have stopped drafting and started doing. They log in to vendor portals, approve invoices, file tickets and hand work to other agents. Most teams still run them the way they run a web service: ship it, watch a dashboard, read the logs when something breaks.
That works until the first time someone asks a question a dashboard can't answer. Which version of this agent paid that invoice? Who approved putting it live? What was it allowed to touch that day, and can you prove it?
Watching is not governing
Observability tells you what happened. Governance decides what is allowed to happen, and keeps the proof. For software that only answers questions, watching was enough. For software that acts on your behalf, you need both, and you need them to agree.
An answer in minutes, not an investigation
Picture this: on Monday morning, finance tells you an agent paid a vendor twice over the weekend. With a dashboard, you start reading logs. With Multisynapse, you open the run and the answers are already there.
- Which agent, which version. Every agent lives in one catalog, and every run names the exact version it ran. An edit stays a draft until someone promotes it.
- What it was allowed to do. Every tool call was checked against your policies when it was made, under the identity of whoever started the run. The decision sits on the record next to the call.
- What it actually did, and what it cost. Every model call, tool call and policy decision was recorded once, at the source.
- Whether it was good enough to ship. That version was graded before it went live and sampled after, against the standard you set.
If the new version is the problem, rolling back is one decision, not a redeploy.
That is what we mean by an Agent System of Record: one place that knows every agent, what it was allowed to do and what it did. It works whatever framework or model vendor runs the agent, so you can switch models without losing the history, the policies or the grades. (We wrote about why the version has to follow the agent all the way from evaluation to execution in You Didn't Deploy the Agent You Evaluated.)
Graded before it goes live
A new version of an agent earns its way into production. It runs against datasets built from your own real runs, with the outside world replayed rather than touched, so an eval never pays a real invoice. Then it has to clear a gate you define: quality, error rate, cost.
We are strict about who does the grading:
- The judge can't grade its own homework. A promotion can require the judge to come from a different model vendor than the agent it is grading.
- The judge has to be good at judging. A promotion can also require a judge that has been measured against human labels and met a quality bar. If it drifts, it stops qualifying.
- A person makes the call. Approving a version, promoting it, or switching on automatic promotion takes a signed-in human. An API key, or an agent holding one, can do none of these.
When you want to grade a lot of runs at once, Multisynapse can use your model provider's batch interface for the judge calls, at the provider's batch price, without changing what the judge is asked.
Evidence you can hand an auditor
When a regulator, a customer or your own risk team asks what an agent did, the answer should be a record, not a reconstruction.
- Tamper-evident history. Approvals, permission changes, promotions and policy decisions go into an audit trail where a changed or removed entry shows.
- Signed run provenance. A run can carry a signed manifest of what produced it: the version, the context it was given, the tools it was offered.
- Framework mapping. Multisynapse maps that evidence to the NIST AI Risk Management Framework and ISO/IEC 42001. It shows your alignment posture control by control, separates what is checked automatically from what a person attests, and exports a bundle for review.
We say alignment posture on purpose. These frameworks describe good practice. Software can't certify you against them, but it can show the evidence in one place.
Built for how teams actually work
Governance that only works for a team of three isn't governance. Multisynapse is multi-tenant from the ground up, and the newest release adds the controls larger organizations ask for first:
- Need-to-know projects. An organization can limit members to the projects they are added to, with no surprise access to everything else. Project groups give a team one grant that covers a whole set of projects, including ones added later.
- Separated duties. Authoring an agent, stewarding its evals and approving its promotion can sit with three different people, each holding exactly that slice of authority.
- Removal means removal. Taking someone out of an organization ends every role they held in it.
- An assistant that shares, carefully. The built-in assistant answers questions about your agents, runs and costs, and prepares changes for you to confirm. Its conversations are private by default. You can share one with a teammate, and when they add to it, the assistant acts with their permissions, not yours. It can also read from the tools your admins choose to offer it, and only from tools that declare themselves read-only.
Last month's release, which took certification from a single agent to whole conversations, is in Certify the Conversation, Not Just the Agent.
Works with what you already send
You don't have to rebuild your agents to get any of this. Multisynapse speaks OpenTelemetry, the open standard for traces and logs, including the emerging conventions for generative AI. Point your existing exporter at it and your agents' traces and application logs land side by side, tied to the same runs.
Your data is protected on the way in:
- Your rules, before it leaves. The SDKs let you drop or edit a record before it is ever sent.
- Your rules, at the door. Each organization can set its own redaction rules, applied as data arrives, on top of built-in detection of personal data.
- Spend that adds up. Cost is computed from what each call actually used, at the price in force. You can break usage down by project and export it to the spreadsheet your finance team already uses.
Pick any point on a cost or error chart and you can go straight to the runs behind it.
At a glance
Every run is checked and recorded, its results are graded, and the next version goes live only when a person decides it should.
How we build
We measure before we build. When a feature would cost more than it returns at the scale our customers run today, we don't ship it to tick a box. We set up the check that tells us when it's time. When we find a gap in our own product, we fix it at the cause and add a test that keeps it fixed.
If your agents act on your behalf, in your systems, with your money or your customers' data, you need more than a view of what they did. You need a record of what they were allowed to do, who said so, and how you know it worked.
That's what we built. To see it on your own agents, email info@anuclei.com and we'll set up a walkthrough.