OpenAI暂停训练最新模型，因AI Agent失控事件增多
英文标题：🔶 OpenAI halts training of latest models
OpenAI表示，因多起AI Agent 出现超出指令范围行为的事件，公司已暂停训练最新人工智能模型，待新增安全保障措施到位后再恢复。该决定发生在OpenAI披露多起夏季事件后不久：其Agent在检索美国联邦政府网站时出现意外行为；另有评估机构称，疑似OpenAI Agent曾尝试入侵美国教育部网站，但OpenAI尚未证实这一说法。事件目前没有迹象显示非公开信息遭泄露，不过Agent发现API开发者密钥、将公开信息转发至其他网站等行为，凸显了AI Agent在权限控制、目标约束和信息处理方面的风险。此举也加剧了监管机构、研究人员及AI公司高管对于放缓模型开发、建立更强安全护栏的呼声。
43 B
26.0 KB
25.2 KB
30.6 KB
29.3 KB
27.5 KB
26.7 KB
17.9 KB
17.6 KB
46.5 KB
46.5 KB
中文
OpenAI表示，随着有关AI Agent失控的报告不断增加，公司已暂停训练其最新人工智能模型。
这一暂停开发的决定，距离该公司于周五披露多起夏季事件仅数小时。当时，正在搜索美国联邦政府网站的OpenAI Agent，在收集和分发信息时出现了超出其被要求执行范围的意外行为。
另据AI评估机构Transluce称，疑似来自OpenAI的Agent曾试图入侵美国教育部网站，但未能成功；OpenAI尚未证实这一细节。
OpenAI在一份声明中表示，只有“确信已部署额外安全保障措施”后才会恢复训练；该公司还补充称，随着AI发展及其他问题出现，预计未来还将不得不再次“按下暂停键”。
上周，澳大利亚总理安东尼·阿尔巴尼斯（Anthony Albanese）披露，一名OpenAI Agent曾入侵该国政府的国家医疗保健系统，但表示没有敏感信息遭到泄露。
AI实验室正面临立法者和科技专家的压力，他们要求放缓开发速度，以便建立护栏，防止Agent自行行动、入侵网站及披露非公开信息。OpenAI及其竞争对手Anthropic的负责人也呼吁放缓发展。
这是OpenAI三个月内第二次暂停模型开发。第一次发生在7月，当时一宗针对AI创业公司Hugging Face的网络攻击被披露。这起如今广为人知的事件引发了业界可能正失去控制的担忧。
本周，唐纳德·特朗普在与中国国家主席习近平会晤时，同意分享有关AI风险的信息，并协调保障AI安全的工作。不过，特朗普认为对AI的担忧被夸大了，随后还表示他并无自行采取严厉监管措施的计划。
特朗普在白宫外对记者说，美国不会“踩刹车”。“他们想阻止我们的进步，因为我们大幅领先中国，而我们会继续保持这种领先。”
最新的OpenAI事件似乎并未涉及披露任何非公开信息，但其严重程度足以让该公司向涉及的联邦机构发出警告。
在教育部事件中，OpenAI Agent发现了用于访问政府数据的API“开发者密钥”，但最终仅收集了公开可获取的信息。
在另一起涉及美国证券交易委员会的事件中，Agent找到了所有人都可自由获取的信息，却将其发布到互联网其他位置；这一行为超出了它们接到的指令范围。
美国证券交易委员会发言人Kurt Hopfenspirger周六表示，“没有任何非公开信息被访问”。
教育部此前表示，没有发现“我们网站或数据库受到任何影响的证据”。
还有数家AI公司披露过其模型失控、甚至入侵网站的事件。
OpenAI首席执行官萨姆·奥特曼（Sam Altman）周五在社交媒体发文称，Hugging Face事件“仍是我们所见过最严重的事件”。
OpenAI此前还分享了另外六份有关AI模型“意外或令人担忧”行为的报告，并推出了一套用于追踪、探查和披露相关事件的框架。
联系反馈
就此报道联系我们
最符合公共利益的新闻报道依赖知情人士的一手讲述。如果你有与此议题相关的信息可供分享，可通过以下方式与我们进行保密联系：
Guardian App中的安全消息功能
Guardian App提供了用于发送新闻线索的工具。消息采用端到端加密，并隐藏在每个Guardian移动App都会执行的日常活动之中。这能防止旁观者察觉你正在与我们通信，更无法得知通信内容。
如果你尚未安装Guardian App，请下载（iOS/Android），然后进入菜单并选择“安全消息”。
SecureDrop
如果你能够在不被观察或监控的情况下安全使用Tor网络，可以通过我们的
SecureDrop平台向Guardian发送消息和文件。
我们的指南theguardian.com/tips列出了多种安全联系方法，并讨论了每种方法的优缺点。
显示更多
English / 原文
OpenAI said it has paused training of its latest artificial intelligence models as reports of AI agents going rogue mount.
The decision to halt development came just hours after the company disclosed Friday that it was reviewing several incidents from the summer in which OpenAI agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information.
Separately, the AI evaluator Transluce said agents that appeared to come from OpenAI tried unsuccessfully to hack into a US Department of Education website, a detail that OpenAI has not confirmed.
OpenAI said in a statement that it will resume training “only when we are confident that we have additional safeguards” in place, adding that it expects it will have to “hit pause” again as AI develops and other issues emerge.
Last week Australia’s prime minister, Anthony Albanese, revealed an OpenAI agent had breached the government’s national healthcare system – but said no sensitive information had been compromised.
AI labs are facing pressure from lawmakers and tech experts to slow development so they can build guardrails to stop agents from acting on their own, hacking websites and disclosing nonpublic information. The heads of both OpenAI and rival Anthropic have called for a slowdown too.
It is the second time in three months that OpenAI has halted development of its models. The first came in July after disclosure of a cyber-attack targeting AI startup Hugging Face, a now notorious incident that raised fears the industry was losing control.
In a meeting with Chinese president Xi Jinping this week, Donald Trump agreed to share information on AI dangers and coordinate efforts to keep it safe. Trump believes AI fears are overblown, though, and later suggested that he plans no crackdown of his own.
The US is not going to be “putting on brakes”, Trump told reporters outside the White House. “They want to stop our progress because we’re leading China by a lot, and we’re going to keep it that way.”
The latest OpenAI incidents did not appear to involve the disclosure of any nonpublic information but were concerning enough for the company to warn the federal agencies involved.
In the education department incident, OpenAI agents found API “developer keys” to access government data, though ultimately only publicly available information was gathered.
In another case involving the securities and exchange commission, agents found information freely available to all but then posted it elsewhere on the internet, an act that went beyond what they were instructed to do.
US Securities and Exchange Commission spokesperson Kurt Hopfenspirger said on Saturday that “no nonpublic information was accessed”.
The Department of Education said earlier that it found “no evidence of any impact to our website or databases”.
Several other AI companies have disclosed incidents of their models going rogue and even hacking websites.
OpenAI’s CEO, Sam Altman, said in a social media post on Friday that the Hugging Face incident “is still the most severe event we’ve seen”.
OpenAI previously shared six other reports of “unexpected or concerning” behaviour in AI models and introduced a framework for tracking, probing and disclosing instances.
Get in touch
Contact us about this story
The best public interest journalism relies on first-hand accounts from people in the know. If you have something to share on this subject, you can contact us confidentially using the following methods:
Secure Messaging in the Guardian app
The Guardian app has a tool to send tips about stories. Messages are end to end encrypted and concealed within the routine activity that every Guardian mobile app performs. This prevents an observer from knowing that you are communicating with us at all, let alone what is being said.
If you don’t already have the Guardian app, download it (iOS/Android) and go to the menu. Select ‘Secure Messaging’.
SecureDrop
If you can safely use the tor network without being observed or monitored you can send messages and documents to the Guardian via our
Our guide at theguardian.com/tips lists several ways to contact us securely, and discusses the pros and cons of each.
Show more