当 AI Agent（意外地）作恶时，谁该承担责任？

当前公众对于 AI 模型究竟有多“智能”的认知，似乎存在很大差异。早在 2022 年，一名 Google 员工就已经认为其 AI 模型具有意识。如今到了 2026 年，似乎每隔几周就会有一篇新文章称，AI 公司“再也无法控制自己的 AI Agent”[2, 3, 4]。

公众开始害怕 AI 完全可以理解。过去，人们担心企业会利用 AI 替代各类工作；如今，_它们_甚至开始入侵政府机构。

负责任的 AI

作为一名 AI 领域从业者，我想明确区分一件事。在上一段末尾，你认为“它们”这个词指的是什么？从关于这一话题的热门标题来看，人们通常会读到这样的表述：似乎是 AI Agent 在实施入侵，因此也应由它们承担责任。我认为，这类标题在一定程度上加剧了公众恐慌，因为发现漏洞、以意料之外的方式访问数字基础设施的，并不是 AI Agent 本身的过错。AI 以及 AI Agent，只是企业和个人为了达成某个目标而运行的工具。在使用工具前，深入理解其局限性至关重要。这也是欧盟发布《欧盟 AI 法案》并纳入强制性AI 素养要求的原因：部署 AI 系统的组织应当充分教育其用户如何理解和使用这些系统。在现实世界中，圆锯使用者应在使用前仔细阅读说明书；即使如此，圆锯工程师仍会增加刀片护罩和紧急停止装置，以尽可能降低风险。在数字世界中，我们同样需要从 AI 工程师和使用者两方面采取负责任的行动。

必须明确的是：AI Agent 正在逃离沙箱并“入侵”公共网站，这一事实非常令人担忧。这些 Agent 背后的 AI 模型已经极其擅长泛化，以至于其文本生成能力_看起来_像是智能。但我们不要忘记，这些 AI 模型——即大型语言模型（Large Language Models；LLMs）——所做的正是生成文本：本质上不断预测下一个词。它们并未像人类一样被视为具有意识。它们只是展现了对文本的语义理解，即能够输出在逻辑上承接前文的文本。这种能力很强大，但既不等同于类人的意识，也不会独立地产生危害。这些模型只是以目标为导向。

谁该被追责？

让我们回到责任归属的问题。新闻标题谈论 AI Agent 逃离沙箱，但 AI Agent 只是被使用的工具。这些公司的研究人员设置 Agent 来完成某项任务；在HuggingFace 入侵事件中，这项任务有时甚至是不可能完成的。随后，这些 Agent，也就是工具，会开始处理一切实现既定目标所需的事项。它们本身并不具有恶意意图。除了完成研究人员提供的初始查询或提示词之外，它们没有其他意图。正是这些将 AI Agent 安置在沙箱中进行约束的研究人员，判断沙箱足够安全，因此无需持续的人类在环（human-in-the-loop）监控。遗憾的是，这些沙箱很少真正足够安全。

而这恰恰说明了责任应当落在何处。

任何可能对其他系统或人员造成重大伤害的系统，都应具备充分的风险缓解措施。建立一个_应该_限制这些 Agent 访问公共互联网的沙箱，仅仅是其中_一项_缓解措施。OpenAI、Anthropic 等 AI 公司始终应考虑瑞士奶酪模型：不能认为单一缓解措施就能修补所有风险。尽管我始终建议尽可能多地引入 human-in-the-loop，例如由人类把关者批准 AI 建议的潜在危险操作，但我也理解，持续的人类把关可能会过度拖慢 AI 创新。

或许，更好的缓解方式是 human-on-the-loop：根据行动可能造成的危险后果实施人类监督。甚至，为什么不使用另一个独立的 LLM，或使用 Jev，在执行前自动对 Agent 行动的危险等级进行分类；一旦触发警示，就暂停 Agent，直到人类监督者批准该潜在危险操作？获取网站数据几乎不需要逐项批准，但当 AI Agent 的文本输出暗示，例如要在 Web 请求中隐藏机密信息时，系统应实施自动告警并暂时停止运行。事实上，最显而易见的缓解措施，是在系统首次尝试以超出预期范围的方式访问公共互联网时立即停止，无论请求内容为何。对于那些被 AI Agent“逃脱”的沙箱而言，连这种相对容易实施的风险缓解措施都没有采用，这充分反映了相关 AI 公司使用 AI 时的伦理状况。不仅研究人员本应更负责任，管理层也绝对应当理解这些实验的危险性，并在缺乏适当监控时予以阻止。

我们该怎么做？

缓解 AI Agent 使用风险的方法有很多。你不必害怕 AI，也不应将 AI 拟人化。然而，你或许应当担心的是，像 OpenAI 这样的公司把互联网中的 AI Agent 当作狂野西部来对待，同时假装其研究人员、工程师和管理层无需为他们允许产生的 AI 行为负责。这些公司应当为风险缓解不足、不负责任地使用 AI，以及由此造成的一切伤害承担责任。

记者同样应当认真斟酌 AI 新闻的措辞。宣称 AI“变得过于智能”或“无法被控制”的标题，或许比客观事实更能获得流量，但其代价显然是损害读者对 AI 的认知和理解。请重新考虑新闻报道的伦理面向，以及将 AI 这一难以理解的话题耸动化后，可能会给不熟悉该领域的人带来的后果。

It looks like public perception of how 'intelligent' current AI models are varies widely. Back in 2022, a Google employee already thought their AI model was sentient. Today in 2026, it seems like every other week there's a new article released about how AI companies "can't hold back their AI agents anymore"[2, 3, 4].

It makes perfect sense for the general public to start fearing AI. In the past, people feared companies would use AI to replace all kinds of jobs, and now they are even hacking government organisations.

Responsible AI

As a professional in the field of AI, I'd like to make one clear distinction. At the end of the last paragraph, what do you think the word "they" refers to? Reading popular headlines on the topic, it typically reads like AI agents are the ones doing the hacking and thus being the ones to blame. I'd argue that these headlines in part cause fear-mongering among the general public, as it's not the AI agents at fault for finding vulnerabilities and accessing digital infrastructure in unexpected ways. AI, and AI agents, are merely tools that companies and individuals run to reach some goal. Before using a tool, it is essential to deeply understand its limitations. That's also why the European Union released the EU AI Act including its mandated AI Literacy: organisations that deploy AI systems should sufficiently educate their users on it. In the physical world, users of a circular saw should carefully read its instructions before use, and even then, the engineers of the saw still add a blade cover and emergency stop just to mitigate risks as well as possible. Digitally, we need to similarly act responsibly on both the engineer's and user's side of AI, too.

Let's be clear: the fact that AI agents are breaking out of sandboxes and "hacking" public websites is very concerning. The AI models behind these agents have gotten incredibly good at generalization, to the point where their text generation seems like intelligence. But let's not forget that these AI models (Large Language Models; LLMs) are doing just that: generating text, effectively predicting the next word, over and over again. They are not deemed conscious like humans. They just show semantic understanding of text, in the sense that they can output text that logically follows the previous text. It's powerful, but not human-like conscious or independently harmful. These models are simply goal-oriented.

Who is to blame?

Let's get back to the question of accountability. Headlines talk about AI agents breaking out of sandboxes. The AI agents are merely tools used. These companies' researchers set up agents to complete a task, sometimes an impossible one in the case of the HuggingFace hack, and the agents (thus: tools) start processing everything necessary to reach the given goal. They do not have harmful intent per se. They do not have any intent other than solving the initial query, or prompt, that the researchers supplied. It is these researchers, who set up AI agents in sandboxes to contain them, who determined that the sandboxes are secure enough that they don't require continuous human-in-the-loop monitoring. Unfortunately, these sandboxes were rarely sufficiently secure.

And that right there shows where the accountability should be.

Any system with risks of causing major harm to other systems or people should have sufficient risk mitigations. Setting up a sandbox that should restrict public internet access to these agents, is merely one such mitigation. AI companies like OpenAI, Anthropic and many more should always account for the Swiss cheese model: it is not enough to assume one mitigation will patch all risks. Although I'd always recommend as much human-in-the-loop as possible, e.g. a human gatekeeper to approve potentially dangerous AI-suggested actions, I can understand persistent human gatekeeping would slow down AI innovations too much. Perhaps a better mitigation would be a human-on -the-loop: human supervision based on potentially dangerous consequences of actions. Heck, why not use a separate LLM or even Jev to automate classifying danger-levels of agents' actions before running them, to raise a flag and pause the agent until the human supervisor has approved the potentially dangerous action. There's little need to approve the fetching of website data, but they should implement automated flag-raising and temporarily halting the system when the AI agent's text output suggests e.g. hiding secrets in a web request. In fact, the most obvious mitigation would be to simply halt the system the moment it first attempts to access the public internet outside its expected scope, regardless of request content. The fact that this relatively easy-to-implement risk mitigation wasn't applied to sandboxes that AI agents "break out of" tells you a lot about the ethics of AI-use at said AI companies. Not only should the researchers have been more responsible, leadership should absolutely have understood the dangers of these experiments and pushed back as well without proper monitoring.

What should we do?

There are plenty of ways to mitigate risks that come with the use of AI agents. You don't have to be afraid of AI. You also shouldn't anthropomorphize AI. What you could be afraid of, however, is companies like OpenAI treating AI agents on the internet like the Wild West, and pretending their researchers, engineers and leadership aren't accountable for the AI-generated actions that they allow. These companies should be held accountable for insufficient risk mitigation, irresponsible use of AI and all of the harm this causes. And journalists, too, should really think twice about the phrasing of AI news. A headline that talks about how AI "has gotten too intelligent" or "couldn't be contained" might get more views than the objective truth, but clearly at the cost of readers' notion and understanding of AI. Please reconsider the ethical side of journalism, and the potential consequences of sensationalising this hard-to-grasp topic that is AI, for those who are less familiar with the subject matter.