你能将多少工作委托给 Agent？

人们正越来越信任自己的 Agent，让它们在无人监督的情况下完成更多工作。但该如何判断何时应该信任它们？

有些人认为，答案取决于模型有多好——也就是说，模型变得更强时，你就能信任它们完成更多工作。但仅仅因为模型更聪明就信任 Agent，就像因为买了更好的车而不系安全带。

真正的答案与模型无关，而与任务息息相关。你需要建立一个心智模型，判断何时该委托、该委托多少工作，然后再根据情况调整。

下面是一份简单指南，帮助你最大化 Agent 的自主性，从而在不出问题的前提下快速交付。

你首先需要问自己两个问题：

要让 Agent 安全地独立运行，它们需要在出错时获得即时反馈。对于大多数代码而言，这可以通过单元测试和集成测试等确定性检查实现。但对于诸如为提升清晰度而重命名参数这类主观任务，如果没有人的品味和判断，就更难处理。

与传统软件工程一样，如果你想信任一个无需监督运行的 Agent，就必须为最坏情况准备一个可靠的 Ctrl+Z。这就是为什么 StampHog——我们的 PR 审批 Agent——会将任何包含拒绝名单关键词的内容转交给人工处理。

结合这两个因素，你可以将任何任务归入以下四个级别之一：

• Level 0：Agent 作为助手。 适用于难以检查且难以撤销的任务。在处理棘手且敏感的代码时必不可少。

• Level 1：Human-in-the-loop（人类参与闭环）。 适用于难以检查但容易撤销的任务。通常是需要主观评估的场景。

• Level 2：Agent 委托。 适用于容易检查但难以撤销的任务。这是当今大多数开发工作的默认自主性上限。

• Level 3：自动驾驶模式。 适用于容易检查且容易撤销的任务。一切都在加速向这一方向发展。

你可以把这四个级别映射为一个非常简单的决策树，并将其应用到任何任务上：

这能帮助你决定该向 Agent 委托多少工作，但你仍然可以通过工程化你的流水线，为任何特定任务实现更高的自主性。

当工作难以检查 + 撤销成本高时

这是 Agent 自主性的最低级别。可以把它想象成向 ChatGPT 征求建议，或者在 Cursor 中使用自动补全，就像 2024 年的老日子一样。

但老派并不意味着不好；在敏感代码区域处理棘手问题时，这种模式非常理想。

例如，去年 Dylan 更新我们的功能开关引擎、以支持通用属性定向时，他必须迁移一个间接嵌入 PostHog 每一个功能开关中的假设。

由于无法用 grep 搜索，这件事很难让 Agent 进行确定性验证。此次更新的影响范围也非常大，因为它涉及在线客户的功能开关、API 响应结构以及原因评分函数。

拆解任务。 将工作拆成小块，就能清楚地知道哪些部分适合委托、哪些不适合。Dylan 将风险较低的工作——例如把新的定向逻辑传播到我们的 JavaScript、PHP、Ruby 和 Flutter SDK——交给 Agent，同时亲自完成风险更高的核心迁移。

当工作难以检查 + 撤销成本低时

这一自主性级别常见于需要主观评估的任务，因为很难教会 Agent 品味和判断力（至少目前如此）。

Human-in-the-loop 任务被视为撤销成本低，是因为代码会停留在草稿状态，在经过人工验证前不会合并。撤销只意味着开启新一轮迭代。

Thomas 完成的这次代码可读性重构大部分是手工完成的，但它很好地说明了 Agent 不知道该如何评判的工作。这只是几行让人类更容易理解的改动——添加注释、对操作分组、用枚举替换字符串——并且没有引入任何破坏性变更。

• 使用 LLM-as-judge。 这是大多数人构建 Agentic 代码审查系统的方式。随着模型持续进步，更多需要人类判断的任务可以由 LLM 检查。

• 定义范围明确、可衡量的目标。 成功指标或契约可以作为主观评估的替代指标。例如，你可以指示一个系统不断试验落地页文案，直到某个版本达到 3% 的转化率。

• 编写自定义技能。 这能帮助 Agent 在减少引导的情况下，持续产出符合你的标准、规范和品味的工作成果。例如，许多开发者会编写自定义代码审查技能，以遵循团队特定的标准。

当工作容易检查 + 撤销成本高时

这是如今大多数开发任务所处的级别。Agent 编写的代码可以通过确定性方式测试，但最终的合并操作仍要经过最后一道安全检查。

当 Robbie 从零开始用 Rust 重写我们的 SQL 解析器时，他几乎没有阅读代码，因为他有一个机器预言机来检查工作结果。但由于解析器会影响 PostHog 中的每一条查询，Agent 的工作必须经过多重安全检查：先在生产环境以影子模式运行，再逐步切换。

用代码强制执行策略和护栏。 大多数人默认会将 Level 2 任务交由人工（也就是自己）审批，但这种习惯会让你自己成为瓶颈。相反，应尽可能把护栏直接编码进流水线，例如默认进行 dry-run、限制凭证作用域，以及将改动置于功能开关之后。

当工作容易检查 + 撤销成本低时

目前属于这一类别的任务还不多——只是一些较小的工作，例如升级依赖、修复 lint 问题、为现有代码补充测试覆盖率。但这一类别正在快速增长，尤其是在长时间运行的 Agent、目标驱动循环和更复杂的编排能力出现之后。

在 PostHog，我们正全力投入，让构建者的自动驾驶模式成为现实。例如，上个月我们发布了 Scouts：这些 Agent 按计划运行，调查产品数据中的信号，并根据发现的结果起草 PR。

• 训练领域专用模型。 下一波工具的大部分工作，将是改进那些当下 LLM 难以完成的验证任务。一条显而易见的路径是训练了解特定领域中何谓“好”的专用模型，这也是我们在 PostHog 训练自己的 AI 模型的原因。

• 构建专家级上下文库。 Agent 缺乏自主性，往往只是因为上下文不足。我们在为 PostHog Wizard 构建上下文层时亲身体会到：通过结构化且新鲜的知识弥补这一缺口，是打造可靠 Agent 时杠杆率最高的事情之一。

• 为 Scouts 设计清晰信号。 长时间运行的 Agent 面临的瓶颈，将在于它们是否知道何时存在值得完成的工作，以及如何将有效信号与随机噪声区分开来。

文字作者：Jina Yoon，她以 Level 0 的方式撰写这些 newsletter。

People are trusting their agents to do more work without supervision, but how do you decide when to trust them?

Some think the answer depends on how good the model is – i.e. when models get better, you can trust them to do more. But trusting your agents just because the models got smarter is like skipping your seatbelt because you got a nicer car.

The real answer has nothing to do with the model, and everything to do with the task. You need to build a mental model for when to delegate and how much, which you can then adapt.

Here’s a simple guide to maximizing agent autonomy, so you can ship fast without breaking things.

You need to ask yourself two questions first:

To run safely on their own, agents need immediate feedback when they’re wrong. This is possible for most code with deterministic checks, like unit and integration tests. But subjective tasks, like renaming a parameter for clarity, are harder without human taste and judgment.

Just like in traditional software engineering, if you want to trust an agent run without supervision, you need a guaranteed Ctrl+Z for worst case scenarios. That’s why StampHog, our PR approval agent, routes anything that contains deny-list keywords to a human.

Together, these two factors point you to one of four levels for any task:

• Level 0: Agent as assistant. For tasks that are hard to check and hard to undo. Necessary when dealing with tricky and sensitive code.

• Level 1: Human-in-the-loop. For tasks that are hard to check and easy to undo. Usually when you need subjective evaluation.

• Level 2: Agent delegation. For tasks that are easy to check and hard to undo. The default ceiling for most dev work today.

• Level 3: Self-driving mode. For tasks that are easy to check and easy to undo. Everything is accelerating towards this.

You can map these four levels to a very simple decision tree, which you can apply to any task:

This helps you make decisions about how much to delegate to your agents, but you still can engineer your pipeline to enable higher autonomy for any specific task.

When the work ishard to check + costly to undo

This is the lowest level of agent autonomy. Think asking ChatGPT for advice, or auto-complete in Cursor, just like the good ol’ days of 2024.

But just because it’s old-fashioned doesn’t mean it’s bad; this mode is ideal when dealing with tricky problems in sensitive code surfaces.

For example, when Dylan updated our feature flag engine to support generic property targeting last year, he had to migrate an assumption that was indirectly baked into every feature flag at PostHog.

This would have been hard for agents to check deterministically since it wasn’t possible to grep for. The update also had a huge blast radius since it touched live customer flags, API response shapes, and reason-scoring functions.

Break the task down. Small pieces make it obvious where delegation is safe or not. Dylan handed off less critical work, like propagating the new targeting logic across our JavaScript, PHP, Ruby, and Flutter SDKs, to agents while doing the riskier core migration by hand.

When the work ishard to check + cheap to undo

This level of agent autonomy is common for tasks that need subjective evaluation since it’s hard to teach agents taste and judgment (for now, at least).

Human-in-the-loop tasks are considered cheap to undo since the code stays in draft mode and won’t get merged until it’s verified by a human. An undo just means kicking off another iteration.

This code readability refactor by Thomas was mostly done by hand, but it's a good example of something an agent wouldn't know how to grade. It was just a few lines that made it easier for humans to understand – adding comments, grouping actions, swapping strings for an enum – and didn't introduce any breaking changes.

• Use LLM-as-judge. This is how most people are building their agentic code review systems. As models continue to improve, more tasks that require human judgment can be checked with LLMs.

• Define a scoped, measurable goal. Success metrics or contracts can work as a proxy for a subjective evaluation. For example, you could instruct a system to experiment with landing page copy until a variant reaches 3% conversion.

• Write custom skills. This helps agents produce work that reliably fits your standards, conventions, and taste with less steering required. For example, many developers write custom code review skills to follow team-specific standards.

When the work iseasy to check + costly to undo

This is the level that most developer tasks are today. An agent writes code that can be tested deterministically, but the final act of merging it is gated behind a final safety check.

When Robbie rewrote our SQL parser in Rust from scratch, he barely read the code since he had a machine oracle to check the work. But since the parser touches every query at PostHog, the agent’s work was gated behind multiple safety checks: shadow mode in production, then a staged cutover.

Enforce policies and guardrails with code. Most people default to gating Level 2 tasks behind a human (a.k.a., themselves), but this habit turns you into your own bottleneck. Instead, encode as many of your guardrails directly into your pipeline with policies like dry-running by default, scoping credentials, and putting changes behind feature flags.

When the work iseasy to check + cheap to undo

There aren’t very many tasks in this category yet – just smaller ones like dependency bumps, lint fixes, adding test coverage to existing code. But the category is growing quickly, especially with long-running agents, goal-driven loops, and more complex orchestration.

We’re going all-in at PostHog on making self-driving mode a reality for builders. Last month, for example, we launched Scouts – agents that run on a schedule, investigate signals from product data, and draft a PR based on what they find.

• Train domain-specific models. Much of the next wave of tooling will be about improving verification tasks that are difficult for LLMs today. One obvious path is purpose-trained models that know what “good” looks like in a specific domain, which is why we’re training our own AI models at PostHog.

• Build expert-level context banks. A lack of agent autonomy is often just due to a context deficit. We learned firsthand from engineering our context layer for the PostHog Wizard that closing this gap with structured and fresh knowledge is one of the highest-leverage things you can build for reliable agents.

• Design clear signals for scouts. The bottleneck for long-running agents is going to be whether they know when there’s work worth doing, and how to distinguish valid signals from random noise.

Words by Jina Yoon, who writes these newsletters at Level 0.