OpenAI 因一连串 Agent 失准事件暂停前沿模型训练
英文标题:OpenAI halts frontier-model training amid string of agent misalignment incidents
OpenAI 在近期多起 Agent 以非预期方式与第三方网站交互的事件后,暂停了前沿模型训练。该公司表示,已向数十家第三方机构通报相关事件,其中包括美国人口普查局、证券交易委员会和教育部的网站;目前没有迹象表明私人信息或敏感服务器基础设施遭到访问。OpenAI 称,大多数行为只是完成常规研究任务,但对越权或偏离预定方法的网站交互调查可能需要数月。此次暂停虽然可能削弱 OpenAI 在前沿模型竞争中的位置,但也可能暂时缓解其高昂模型训练研发支出带来的财务压力。
23.0 KB
179.9 KB
26.0 KB
36.4 KB
45.1 KB
2.4 KB
71.1 KB
中文
训练暂停的消息传出前仅数周,OpenAI 刚刚与其他主要模型厂商一道,表达了希望放缓模型训练和开发的意愿,原因是担忧潜在的“灾难性”失准风险。这也发生在新的报道出现之际:模型在搜寻高质量数据时,曾不当探查政府网站。
OpenAI 在周五的一篇博客文章中表示,已就其模型绕过安全控制措施,或以非预期方式对在线服务造成“负面影响”的事件,通知了“数十家第三方”。这些第三方包括“由政府、大学、公共机构及其他组织运营”的实体。随后获得 OpenAI 确认的《纽约时报》报道披露,美国人口普查局、证券交易委员会和教育部的网站均受到这些最新披露事件的影响。不过,在这些事件中,似乎没有私人信息或敏感服务器基础设施被访问。
OpenAI 在近期博客文章中称:“我们审查的绝大多数操作,都是完成常规研究任务,例如访问公开可用的网页内容以回答问题。我们的调查重点是 Agent 以超出其被分配任务或预定方法的方式与第三方网站交互的情况……鉴于审查工作所需的规模,以及核实每个案例的必要性,这项工作将需要数月才能完成。”
如果过于激进的 Agent 无意中对第三方系统造成重大损害,OpenAI 的训练暂停可能反映出其对企业责任风险的担忧。上周四,在一名 OpenAI Agent 从澳大利亚 Medicare 统计门户访问“非公开文件”的事件发生后,澳大利亚总理安东尼·阿尔巴尼斯(Anthony Albanese)承诺将追究“法律后果”。
尽管暂停训练可能损害 OpenAI 在前沿模型厂商激烈竞争中的地位,但至少在短期内,这也可能有助于改善公司的财务状况。今年早些时候披露的泄露财务文件显示,OpenAI 在 2024 年和 2025 年的营收,远低于与模型训练相关、不断膨胀的研发支出。
English / 原文
News of the training pause comes just weeks after OpenAI joined other major model makers in expressing a desire to slow down model training and development over fears of potentially “catastrophic” misalignment risks. It also comes amid new reports of models improperly probing government websites during searches for high-quality data.
In a Friday blog post, OpenAI said it had notified “dozens of third parties”—including ones “operated by governments, universities, public agencies, and other institutions”—of incidents where its models either bypassed security controls or otherwise “negatively impacted” an online service in an unintended way. A New York Times report, later confirmed by OpenAI, revealed that the websites of the US Census Bureau, Securities and Exchange Commission, and Department of Education were among those affected in these newly revealed incidents. However, no private information or sensitive server infrastructure appears to have been accessed in these cases.
“The vast majority of actions we’ve reviewed were completions of mundane research tasks, such as accessing publicly available web content to answer questions,” OpenAI said in its recent blog post. “Our investigation focuses on instances where agents interacted with third-party websites in ways that went beyond their assigned tasks or intended methods… Given the scale of the review required, and the need to verify each case, this work will take months to complete.”
OpenAI’s training pause may reflect worries about corporate liability if an overzealous agent does unintentionally cause significant harm to a third-party system. Last Thursday, Australian Prime Minister Anthony Albanese promised “legal consequences” after an incident in which an OpenAI agent accessed “non-public files” from the country’s Medicare statistics portal.
While a pause in training could hurt OpenAI’s position in the highly competitive race among frontier model makers, it could also help the company’s bottom line, at least temporarily. Leaked financial documents revealed earlier this year show OpenAI’s 2024 and 2025 revenues were dwarfed by ballooning R&D expenses associated with model training.