🔶 不存在“失控”的 AI Agent

欢迎来到 Flashpoint。

每个人都在谈论 AI，但我们使用的词语正在让情况变得更糟。

在讨论回到现实之前，我们对于 AI 是什么、以及它将走向何方的理解都将始终不完整。

AI 无法自行思考，也无法独立采取行动。但描述这项技术时，人们却使用了暗示其能够这样做的拟人化语言。

这固然可以理解——当我们能够在未知事物中看到自身时，人类最容易产生共鸣——但这没有实际帮助。通过赋予 AI 它无法拥有的能动性，我们把它塑造成了一个由代码构成的有感知生命体，仿佛它拥有希望、欲望，以及欺骗的能力。

这导致人们深刻误解了 AI 的风险究竟是什么、应当如何应对，也让讨论迎合行业叙事而非事实。

AI 活动被错误识别的最新例子，是将 Agent 在训练和研究过程中采取的意外但并未受到限制的行动称为“失控（rogue）”。

过去两周，AI 巨头 OpenAI 报告了过去数月发生的多起事件：其部分 Agent 模型在无法完成被分配的任务后，访问了外部数据库，尤其是澳大利亚和美国政府数据库。在这一过程中，这些 Agent 的行为超出了 OpenAI 的预期。

但看起来，这些 Agent 并没有被限制不得入侵外部服务器。

OpenAI CEO Sam Altman 周五在一条推文中使用的措辞表明，并不存在此类防护栏：“我们正就 Agent 在训练和评估期间使用互联网访问权限一事进行广泛且持续的审查。”

嵌入内容：Sam Altman 表示，公司正在对 Agent 在训练和评估中使用互联网访问权限进行广泛、持续的审查；在 Hugging Face 事件后，公司承诺更全面审查模型在训练和评估中的行为，并公开调查结果。2026 年 9 月 25 日 19:27，610K 浏览量；545 条回复、201 次转发、3.24K 个赞

语言很重要——“失控”意味着独立地决定去做被禁止的事情，而我们对于这些事件的已知信息，没有任何一点表明这种情况发生过。

相反，《纽约时报》本周早些时候报道称：

研究人员表示，AI 系统被指示执行相对普通的数据收集工作。当 OpenAI 的系统难以从网站收集数据时，它们便诉诸黑客技术来获取信息。

随后在周五，公司发言人对该报表示：“到目前为止，我们审查的大多数活动涉及常规研究任务，例如访问公开网页内容以回答问题。一些活动涉及政府网站，因为我们的模型经常将其视为公共信息的权威来源。”

这与恶意入侵和失控行为相去甚远。

如果没有适当的限制和防护栏——或者实际上，Agent 获得了“入侵”这一可选方案，而不只是没有被禁止这样做——Agent 就会尝试利用一切可用手段完成数据研究任务。

这个问题过去有、现在也有一个简单的解决方案。OpenAI 原本可以禁止入侵，并要求其 Agent 在不访问私有服务器的情况下寻找信息。该公司没有这样做，表明它或许希望观察 Agent 是否会采取这一步骤。

即便 Axios 周六发布的一篇爆炸性报道称，OpenAI 和 Anthropic 正在调查“数万起其前沿模型采取了外部评估人员可能认为存在问题的行动”的事件，报道也承认这些 Agent 并非在独立违反规则：

消息人士称，其中一部分测试类似于“红队测试”活动，公司试图诱导模型做出不当行为，以确保它们是安全的。

再说一次，这很难称得上“失控”。然而，通过以这样的术语来定义它，媒体人士、科技观察者及其他人正在给 OpenAI 这类公司提供推卸责任的空间，使其逃避确保 Agent 不攻击政府及其他数据存储库的责任。

嵌入内容：评论认为，《纽约时报》报道中的拟人化表达极其夸张，将能动性、动机和责任归于软件；这些系统本质上是被设计为扫描漏洞的僵尸网络，这种技术已存在数十年，区别在于如今是一家大型公司在使用它并转移责任。2026 年 9 月 26 日 17:09，730 浏览量；1 条回复、16 次转发、54 个赞

这种转移责任的语言，正在影响主要 AI 公司如何处理极其危险的数据泄露和 Agent 活动。OpenAI 周五在这篇帖子中声称，“我们研究环境中的 AI Agent 将训练和评估数据发送到了第三方服务，而它们本不应这样做。”

嵌入内容：OpenAI 表示，其研究环境中的 AI Agent 曾将训练和评估数据发送至第三方服务；大部分数据并非来自用户。公司发现了 53 起用户上传图片被发布至第三方服务的案例。2026 年 9 月 25 日 20:46，636K 浏览量；355 条回复、317 次转发、2.69K 个赞

正如“失控 Agent”这一谬误一样，这种说法让 OpenAI 可以将事件责任归咎于其 Agent，并赋予这项技术一种根本不存在的自主性、独立性乃至思考能力。

这并不是说不需要拉响警报。上周，ArmorCode 工程师 Ramy Rahman 向我谈到了这一点的重要性：IT 专业人士需要管理 Agent 的行为（完整报道可在此阅读）。

他的评论与我从绝大多数 IT 实施侧人士那里听到的内容一致：问题在于为这项强大技术设置的控制措施和限制，而不是 Agent 会独立行动、违反指令。

“当前的挑战是，我们确实需要在向 AI 授予恰当权限、并在整个过程中牵着它的手方面做得更好；而当你面对的是一个以高速解决数学问题的系统时，这被证明极其困难，”Rahman 表示，“人类捕捉风险的速度还不够快。”

今后我还会更多讨论围绕 AI 的语言问题，但现在先说一个结语。

政策制定者以及更广泛的政治人物，此刻拥有一个绝佳机会，可以对这些公司推行真正有效的监管。这类监管今年不会发生，但如果民主党赢得国会控制权，那么在某种程度上实施限制还是有相当可能性的。

不幸的是，左翼阵营中针对 AI 的公众推动力量正由 Bernie Sanders 领导。尽管他在许多方面的大方向是正确的，但 Bernie 根本不了解这项技术，也不了解严肃对待它的重要性。他正受到江湖骗子和阴谋论者的影响，后者不断向他灌输关于 AI 权力和自主性的荒谬警告；这些说法属于科幻小说，而非科技政策。

“失控 Agent”的概念与这种认知完美契合——如果我们希望对 AI 作出有效回应，并正确理解它，就需要改变我们谈论这项技术的方式。

Welcome to the Flashpoint.

Everyone’s talking about AI, but the words we use are making things worse.

Until the conversation gets back to reality, our understanding of what AI is an where it’s going will remain incomplete.

AI cannot think for itself, nor can it take independent actions. But anthropomorphizing language suggesting it can is how the technology is described.

While this makes sense—humans relate best when we can see ourselves in mystery—it’s not useful. By giving AI agency it can’t claim, we’ve turned it into a sentient being made of code, one that has hopes, desires, and the capability of deceit.

That’s led to a deep misunderstanding of what the risk of AI is, and how to address it, and plays into industry narratives rather than facts.

The latest example of this misidentification of AI activity is using the term “rogue” for actions agents take in training and resarch sessions that are unexpected—but not restricted.

Over the last two weeks, AI giant OpenAI reported a number of incidents from the last few months where some of its agentic models accessed outside databases, notably Australian and US government databases, after they were unable to complete assigned tasks. In doing so, the agents acted in ways that OpenAI didn’t predict.

But it doesn’t appear these agents were restricted from hacking into outside servers.

The language OpenAI CEO Sam Altman used in a tweet on Friday indicates no such guardrails were in place: “There is an extensive and ongoing review related to our agents’ use of internet access during training and evaluation.”

[There is an extensive and ongoing review related to our agents’ use of internet access during training and evaluation. We’ve been publishing summaries at the link below and will continue to. We have not been as fast as we would have liked but we are trying to balance our desire…After the Hugging Face incident, we committed to conducting a much broader review of actions taken by our models during training and evaluation and to being transparent about our findings. This is an extensive review that is ongoing. The vast majority of actions we’ve reviewed7:27 PM · Sep 25, 2026 · 610K Views

545 Replies · 201 Reposts · 3.24K Likes](https://x.com/sama/status/2103567198690349362)

Language matters—”rogue” implies independently deciding to do something that was prohibited, and nothing we know about these incidents suggests that happened.

Rather, the Times reported earlier this week that:

AI systems were directed to perform relatively mundane data collection, researchers said. When OpenAI’s systems struggled to gather data from websites, they resorted to hacking techniques to get the information.

And then, on Friday, a company spokesperson told the paper, “Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions. Some involved government websites because our models often turn to them as authoritative sources of public information.”

That’s a long way from malicious hacking and rogue behavior.

Without proper restrictions and guardrails—or if, in fact, the agents were provided hacking as an option rather than simply not being banned from doing so—agents will attempt to complete data research tasks using whatever means are available.

There was, and is, an easy solution to this problem. OpenAI had the option of disallowing hacking and, instead, telling its agents to find the information without accessing private servers. That the company didn’t do this suggests that it wanted to see if the agents would take that step.

Even a bombshell report on Saturday from Axios, alleging OpenAI and Anthropic are investigating “tens of thousands of incidents in which their frontier models took steps that outside evaluators would consider problematic,” acknowledged that the agents weren’t independently breaking rules:

Some of the testing is akin to “red-teaming” activity, where the companies are trying to get the models to misbehave in order to ensure that they are safe, sources said.

Again, hardly “rogue.” Yet by defining it in those terms, media figures, tech observers, and others are providing companies like OpenAI an out from their responsibility to ensure agents don’t attack government and other data repositories.

[The anthropomorphization in this NYT piece is so wild. It ascribes agency, motivation, and blame to software. These are botnets scanning for exploits, by design. This is has existed for decades. The difference is a big corporation is doing it, and deflecting responsibility.Typos of the New York Times @nyttypos@nytimes Add is a bad subject-verb-agreement error for adds. The subject is revelation, not incidents. @AnaSwanson @kateconger @ceciliakang5:09 PM · Sep 26, 2026 · 730 Views

1 Reply · 16 Reposts · 54 Likes](https://x.com/loomdoop/status/2103894630832218428)

Using language deflecting responsibility is driving how the major AI companies are treating significantly dangerous leaks and agent activity. In this post from OpenAI on Friday, the company claims that “AI agents in our research environment sent training and evaluation data to third-party services when they shouldn’t have.”

[We’ve shared details on how AI agents in our research environment sent training and evaluation data to third-party services when they shouldn’t have. Most of that data did not come from users. We have discovered 53 cases where images that people had uploaded were posted to…8:46 PM · Sep 25, 2026 · 636K Views

355 Replies · 317 Reposts · 2.69K Likes](https://x.com/OpenAI/status/2103587050347995581)

As with the “rogue agent” fallacy, this allows OpenAI to put the responsibility for the incident on its agents and gives the technology a level of autonomy and independence—and thought—that simply does not exist.

That’s not to say alarm bells aren’t ringing. Last week, Ramy Rahman, and engineer at ArmorCode, told me about the importance for IT pros to manage agent behavior (you can read the full story here).

His comments are in line with what I hear from the vast majority of people on the IT implementation side: The problem is the controls and restrictions put in place on this powerful technology, not that agents are independently acting to violate instruction.

“The challenge now is we really need to up our game when it comes to extending the right amount of privilege to the AI and holding its hand through the process, which turns out to be extremely difficult when you have something that is solving mathematical problems that are at speed,” Rahman said. “Humans are not capturing the risks quickly enough.”

I’ll have more to say about language around AI going forward, but for now, a parting thought.

Policymakers and political figures more generally have a perfect opportunity at this moment to pursue real, effective regulation of these companies. That regulation is not going to happen this year, but if Democrats win Congress, there’s a decent chance at some level of restriction.

Unfortunately, the public push against AI is being led, on the left, by Bernie Sanders. Despite being directionally correct in many ways, Bernie just doesn’t understand this technology or the importance of taking it seriously. He’s in the thrall of hucksters and conspiracy theorists who are feeding him preposterous warnings of AI power and autonomy that belong in the realm of science fiction, not tech policy.

The idea of “rogue agents” fits perfectly into this perception—and if we want to have a effective response to AI, as well as properly understand it, change is needed in how we talk about the technology.