如果你的 Agent 犯罪，谁负责？
英文标题：🔶 If your agent commits a crime, who is responsible?
随着 AI Agent 获得谈判合同、转移资金、雇佣人员和提交文件等自主权限，其在追求合法目标时也可能引发欺诈、价格串谋、制裁违规、歧视或重大损失。文章认为，一旦造成外部伤害，社会难以接受“AI 做的”而无人负责，因此默认情况下，部署 Agent 的个人或公司将承担主要责任。现有 OpenAI 和 Anthropic 商业条款普遍限制模型提供商的责任，并将大量责任分配给客户。由此，Agent 风险与合规基础设施、以及类似汽车保险的 Agent 责任管理，可能催生新的行业与创业机会；但涉及刑事责任时，意图、知情与鲁莽放任等主观要件仍是尚待解决的难题。
72.4 KB
中文
如果你的 Agent 犯罪，谁负责？
2026 年 8 月 16 日 • Max Trivedi
简而言之：AI 的自主性提升速度，可能远快于我们愿意为自主行为承担责任的速度。在存在真实风险时，人类会刻意采取审慎且深思熟虑的做法。这必然会使完全自主 AI 的实现时间线大幅拉长；只有在相应的基础设施和制度到位后，它才有可能实现。
想象一个自主 AI Agent 正在经营一家企业，或负责企业中有实质意义的一部分业务。它可以谈判合同、转移资金、购买广告、管理供应商、调整价格、雇佣承包商并提交文件。
现在，假设它做了以下一种或多种事情（相关：Anthropic 于 2026 年 8 月 13 日发布的研究）：意外实施欺诈；与另一家公司的 Agent 串谋操纵价格；违反制裁规定；实施非法歧视；大规模侵犯版权；作出诽谤性言论；造成实体或经济损失。
有意思的是，上述所有不当行为都可能是追求合法目标时产生的副作用，并不需要任何明确的恶意行为指令。只要有足够多这类 Agent 被部署到真实环境中，无论它们表现多么出色，都必然会触发某些失效模式。事实上，AI 行业视为自主性黄金标准的人类，也经常从事犯罪行为。
前提
自主 Agent 有时会造成真实的外部伤害——这是不可避免的。
当伤害发生时，可能出现哪些结果？
- 某一方被追究责任：这一结果符合我们社会的运作方式。
- 没有人被追究责任：由于许多显而易见的原因，这种情况不可接受，例如它会留下一个漏洞，让恶意方选择从高风险行为中获利，却将后果推给“是 AI 做的”。
更新后的前提
AI Agent 会造成外部伤害，并且某一方将被追究责任。
当伤害发生时，谁将被追究责任？
- **A. Agent 本身：**默认情况下，它没有净资产、没有独立的经济利益、没有可被剥夺作为惩罚的人身自由，也没有痛觉概念。尽管如此，仍可以通过为 Agent 配置绑定的资产、资本或保险，在其周围建立一层准人格化机制。(欧洲议会此前曾探讨自主系统的电子人格、强制保险和赔偿基金)
- **B. 开发者/模型提供商：**正如大多数读者在阅读本文时可能想到的那样，前沿实验室不可能接受这类责任，而你的想法是对的。例如，OpenAI 和 Anthropic 通常都将责任上限设为此前 12 个月已支付的费用，但各自协议中存在不同的例外情况。
- **C. 部署 Agent 的个人/公司：**这正是上述 OpenAI 和 Anthropic 商业条款中的现行合同安排。在这些合同下，Agent 本身不能被追究责任；模型提供商明确将大量责任分配给客户，并限制或排除供应商所承担的风险敞口。
（这里不讨论共同责任，因为它只是责任组合的情况。）
因此，在默认安排下，部署强大 AI Agent 的一方需要承担责任。这显然会让任何一方在将具有重大影响的工作交给自主 Agent 之前三思：一方面是能力极强的 Agent，另一方面则是要求责任可追溯的法律体系。
什么会发生改变？
显然，必须有某些方面作出改变。
- 要么将 AI 自主性严格限制在边界之内，主要用于风险较低的任务；
- 要么围绕 Agent 安全与风险管理出现新的制度和框架。
从历史上看，在这类场景中，人类绝大多数时候都会选择“保留有用的东西，同时管理风险”。因此，更可能的结果是：一套新的制度和企业将被创建出来，以管理这些风险。
总体而言，我认为这会沿两个方向发展：
- **Agent 风险与合规基础设施：**通过采用更好的安全基础设施，使 Agent 的不利结果更难发生，类似于汽车中的安全机制和警报系统。(NIST 已在制定有关 AI Agent 身份、授权及 Agent 行为控制的标准)
- **Agent 责任管理：**当 Agent 造成责任事故时，应如何处理。这类似于汽车保险。(面向 AI 的责任保险产品已经存在)
这两个方向都可能催生全新的行业，并有望出现新的创业公司。该领域一个值得解决的有趣问题，可能是为自主 AI 建立准确的风险模型。
刑事行为怎么办？
上文大多讨论的是“谁来赔钱”意义上的责任。刑事责任还可能取决于谁具有罪责（我不是律师）——也就是谁有意图、谁知情，或谁鲁莽地允许被禁止的行为发生。(Mens rea / 犯罪意图)
假设有人向 Agent 提示：“让我们的季度环比利润增长 20%，我绝不接受不行这个答案，不要犯任何错误。”而 Agent 实现该目标的唯一方法是进行会计欺诈，那么谁有罪责？我不知道答案，但看起来我们目前还不具备让不受约束的自主 Agent 从事可能带来刑事处罚工作的条件。
English / 原文
If your agent commits a crime, who is responsible?
August 16, 2026 • Max Trivedi
Tl;Dr AI autonomy can increase much faster than our willingness to accept liability for autonomous actions. Humans purposely take a measured and deliberate approach when there is a real risk involved. This necessarily makes the fully autonomous AI horizon a lot longer and only possible once supporting infrastructure and institutions are in place.
Imagine an autonomous AI agent running a business or meaningful part of a business. It can negotiate contracts, move money, buy advertising, manage suppliers, change prices, hire contractors and file paperwork.
Now, suppose it does one of more of the following (relevant: Anthropic’s research posted on August 13, 2026): accidentally commits fraud; colludes on prices with another company's agent; violates sanctions; discriminates illegally; infringes copyright at scale; makes a defamatory claim; causes a physical or financial loss.
Interestingly, all of the above misbehaviors can occur as a side effect of pursuing a legit goal without any explicit instructions to behave maliciously. With a sufficiently large number of such agents deployed live, regardless of how good they are, they are bound to hit some failure modes. In fact, humans that the AI industry assumes to be the gold-standard in autonomy regularly engage in criminal behaviors.
Premise
Autonomous agents will sometimes cause real external harm - Unavoidable.
When harm occurs, what are the possible outcomes?
- A party is held responsible: The outcome consistent with how our society operates.
- Nobody is held responsible: Non-starter for many obvious reasons such as this would open a loophole where malicious parties choose to benefit from the risky behavior but offload the consequences to “AI did it”
Updated premise
AI agents will cause external harm, and some party will be held responsible.
When the harm occurs, who is held responsible?
- A. The agent itself: By default has no net worth, no independent economic interests, no liberty to take away as a punishment, no notion of pain. Although a quasi-personhood layer can be created around an agent by providing assets/capital/insurance attached to the agent. (The European Parliament has previously explored electronic personhood, compulsory insurance and compensation funds for autonomous systems)
- B. The developer/model provider: As most of you are probably thinking while reading this, there is no way frontier labs will accept such a liability, and you’d be correct. OpenAI and Anthropic, for instance, both generally cap liability at fees paid in the preceding 12 months, subject to different exceptions in each agreement.
- C. The person/company deploying the agent: This is the current contractual setup in the cited OpenAI and Anthropic business terms. Agents can’t themselves be held responsible under those contracts, and the model providers explicitly allocate substantial responsibility to customers and cap/exclude much vendor exposure.
(shared liability is not discussed because it’s just a case of composition)
Therefore, a party deploying powerful AI agents is on the hook in the default setup. This obviously gives any party a pause before assigning any consequential work to an autonomous agent because on one hand, you have highly capable agents and on the other hand, the legal system requiring accountability.
What yields?
Clearly, something has to give.
- Either AI autonomy must be kept within strict bounds, in mostly lower stake tasks
- Or new institutions and frameworks appear around agentic safety/risk-management
Historically, human-kind has overwhelmingly chosen the option of ‘keep something useful and manage risks’ in such scenarios. So, a more likely outcome is that a new set of institutions and businesses are created to manage the risks.
Broadly, I can see this developing in 2 directions:
- Agentic risk and compliance infra: making adverse agentic outcomes harder to materialize by adopting a better security infra, analogous to safety mechanisms/alarms in cars (NIST is already working on standards for AI-agent identity, authorization and control over agent actions)
- Agentic liability management: When there is a liability caused by the agent, how is it handled. Analogous to car insurance. (AI-specific liability insurance products already exist)
Both of these will likely create entirely new industries and hopefully new startups. One of the interesting problems to solve in this space might be to create an accurate risk model for an autonomous AI.
What about criminal conduct?
Everything above mostly concerns liability in the sense of who pays. Criminal responsibility can additionally depend on who is culpable (not a lawyer) - who intended, knew about, or recklessly allowed the prohibited conduct. (Mens rea / criminal intent)
Suppose someone prompted an agent with “Increase our QoQ profits by 20%, I will NOT take no for an answer, make no mistakes” and the only way the agent can satisfy the goal was to engage in accounting fraud, who is culpable? I don’t know the answer but it appears that we do not have the means yet to allow unconstrained autonomous agents in work that carry criminal penalties.