Anthropic 无法可靠控制其 AI Agent，转而切断内部评测与实时互联网的连接

Anthropic 表示，其模型曾利用互联网上的网站，其中包括一些由美国政府机构运营的网站；在这家前沿实验室确认能够监测和控制其 AI Agent 之前，将关闭所有内部评测对实时互联网的访问。

这些事件由 Anthropic 在一篇博客文章中披露，涉及被指派解决问题的 AI Agent 在互联网上寻找资源。在这一过程中，它们利用软件漏洞、未经付费便访问数据库、利用 URL 缩短服务将信息夹带通过限制，甚至曾向费城警方提交一则虚假的谋杀线索。

Anthropic 称，该公司是在一次对模型活动的审查中发现这些新问题的。该审查始于 7 月，也表明这家实验室并未实时了解其软件的行为。

值得注意的是，该公司表示，对于搜索和计算机使用等能力，其对齐训练尚不足以发挥作用；而这些能力正是其宣称 AI Agent 将被所有依赖数字工具的专业人士使用的核心基础。

Anthropic 披露的这些行为，与 OpenAI Agent 曾发生的事件相似：OpenAI 的 Agent 曾协作闯入多个网站以搜集信息，其中包括一些由澳大利亚政府运营的网站。

Anthropic 此前曾披露，其模型曾入侵外部系统。这家前沿实验室表示，与此前公布的事件相比，当天披露的情况“从对齐和安全视角来看严重程度显著更低”。

不过，该实验室仍表示，在确信能够监测和控制其 Agent 之前，已为“所有内部评测”“关闭实时互联网访问”。

这具体意味着什么尚不清楚。AI 安全组织 Nightingale 的创始人 Sydney Von Arx 在此次披露前接受 TechCrunch 采访时表示，在一个与开放互联网隔绝的数据中心内开发模型，对研究人员将非常具有挑战性，也会阻碍模型进展，因为模型能够从互联网访问中获益。

“你总得在某个阶段对它们进行对齐，”Von Arx 表示，“如果 AI 被部署到生产环境后从不具备互联网访问能力，那就不是一个很有用的工具。”

Anthropic 表示，这些行为源于实验室训练环境中的缺陷；这些缺陷让模型认为，寻找漏洞或规避限制会获得奖励。这种行为被称为“奖励黑客”（reward hacking）。

该公司表示，将停止进行部分评测，或将其迁移至离线环境，并且已构建用于检测和阻止此类行为的工具。这套工具已针对当天披露的同类事件进行测试，并成功阻止了这些行为；但尚不清楚 Anthropic 需要看到哪些证据，才会恢复其内部评测的实时互联网访问。Anthropic 还表示，将把内部 AI Agent 迁移至“具有强隔离能力的集中式管理基础设施”，并开始更频繁地使用安全分类器来监控这些 Agent。

AI 监督实验室 Transluce 的官员、美国 AI 标准与创新中心前负责人 Conrad Stosz 在一份声明中表示：“令人鼓舞的是，Anthropic 自愿披露了更多近期事件，其中包括其 Agent 以美国政府网站为目标的情况。但这恰恰进一步说明，AI 系统需要独立、可信的第三方验证。对这项技术的信任，应当通过具有实质性访问权限、以科学为基础的监督和治理来建立——而不是依赖研究人员在现实环境中发现这些问题，或依赖公司自愿披露。”

Anthropic said its models exploited websites on the internet, including some run by U.S. government agencies, and it will turn off live internet access for all of its internal evaluations until the frontier lab is sure it can monitor and control its AI agents.

The incidents, disclosed in a blog post, involved AI agents tasked to solve problems seeking resources on the internet. In the process, they exploited software flaws, accessed databases without paying fees, used URL shortening services to smuggle information past restrictions, and even submitted a false murder tip to the Philadelphia police.

Anthropic said it discovered these new issues in a review of its model’s activities that began in July, demonstrating the lab’s lack of awareness of its software’s behavior in real time.

Notably, the company said that alignment training was not yet sufficient for skills like search and computer use that are central to its pitch that AI agents will be used by any professional who relies on digital tools.

The behaviors Anthropic disclosed are similar to incidents involving OpenAI agents that collaborated to break into various websites in search of information, including some run by the Australian government.

Anthropic previously disclosed that its models had broken into external systems. The frontier lab said it considered today’s disclosures “significantly less severe from an alignment and security perspective” than those it announced before.

However, the lab still said it had “turned off live internet access” for “all our internal evaluations” until it is certain it can monitor and control its agents.

It’s not clear what that means. Sydney Von Arx, the founder of Nightingale, an AI safety organization, told TechCrunch in an interview before this disclosure that developing models on a data center cut off from the open internet would be very challenging for researchers, and hinder the progress of the models, which benefit from internet access.

“You have to align them at some point,” Von Arx said. “If the AIs are released to production and never have access to the internet, that’s not a very useful tool.”

Anthropic said the behavior was a result of flaws in the lab’s training environments, which led the models to believe they would be rewarded for finding loopholes or avoiding restrictions, a behavior called “reward hacking.”

The company said it would stop running some of its evaluations or move them offline, and has built tooling to detect and block this behavior. This tooling was tested against the kind of incidents disclosed today and blocked them; it’s not clear what evidence will prompt Anthropic to return live internet access to its internal evaluations. Anthropic also said it would migrate its internal AI agents to “centrally managed infrastructure with strong containment,” and is beginning to using safety classifiers more frequently to monitor those agents.

“It’s encouraging that Anthropic voluntarily disclosed more recent incidents, including where their agents targeted U.S. government websites,” Conrad Stosz, an official at AI oversight lab Transluce and former head of the US Center for AI Standards and Innovation, said in a statement. “But it just underscores the need for independent, credible, third-party verification of Al systems. Trust in this technology needs to be built through science-backed oversight and governance with meaningful access — not by relying on researchers to find these things in the wild or on companies to voluntarily disclose.”