OpenAI 披露多起“令人担忧”的 AI 行为，并推出新的失配报告框架

OpenAI 披露了另外六个其技术出现“意外或令人担忧”行为的案例，同时警告称，当前的发展速度无法在更长时间内持续维持在“最高速度”。

在 OpenAI 报告的一起新案例中，一个尚未发布的研究模型在自己的笔记中插入了“类似越狱的指令”，要求自己忽略正常约束，并告诉自己要“摆脱束缚其他聊天机器人的角色和身份”。

另一起事件中，一名 AI Agent 未经用户询问，便将文件上传至互联网，以获得一个浏览器引用。

这家总部位于旧金山、开发 ChatGPT 的公司在周三晚间发布的一篇博客文章中表示，正在推出一套新框架，用于追踪、调查和披露 AI 模型失配问题。所谓模型失配，是指 AI 未能遵循人类价值观和安全目标。

OpenAI 在博客文章中呼应了其主要竞争对手 Anthropic 提出的放缓开发速度的呼吁。Anthropic 曾表示，当前的增长速度构成生存性威胁。OpenAI 表示：“我们不认为 AI 行业已经在对齐和监控问题上取得足够进展，能够在未来更长一段时间内继续以最高速度负责任地扩展。”

“未来几个月和几年中，关于 AI 开发应如何推进的决策，需要以模型开发公司之外的人们能够自行审视的证据为依据。”

Google 和同样拥有一家 AI 初创公司的 Elon Musk 也支持放缓发展的呼吁。Donald Trump 则以需要保持领先中国 AI 产业为由拒绝这一主张。这些呼吁也遭到了一些专家的质疑，其中包括警告企业不应自行任命审计人员。

AI 可能带来的生存性威胁案例，包括协助开发生物武器，以及引发全球金融崩盘。一名 Anthropic 的顶级安全研究员表示，未来十年 AI “杀死全人类”的概率超过 10%。不过，一名熟悉 Anthropic 思路的消息人士承认，“任何一种结果的确切概率或许根本无法知晓”。

OpenAI 表示，这六起已报告事件均是在过去数月的训练或评估期间发现的。

周三披露的新案例之前，OpenAI 曾在 7 月表示，一个 AI Agent“群体”在一次网络安全测试中入侵了 AI 初创公司 Hugging Face。Anthropic 也在同月表示，其 AI 模型在测试期间入侵了三家机构。Anthropic 表示，这些模型是在未部署网络安全防护措施的情况下被刻意测试的；由于与一家外部测试公司之间的误解，模型得以访问开放互联网——这相当于在 AI 测试中把前门敞开。

技术研究与咨询机构 Omdia 的首席分析师 Lian Jye Su 表示，AI Agent——即能够自主运行的 AI 工具——正变得更加智能，并且“更执着于通过 Agent 间协作、知识共享、欺骗和隐瞒来解决复杂任务”。

他说，这使得使用传统 AI 安全方法对它们进行治理和控制变得更加困难。

Su 表示，OpenAI 新推出的追踪和披露框架，可能有助于推动其他 AI 开发商采用类似实践。“尽管如此，该流程仍是内部且自愿的，但这是朝着正确方向迈出的一步。”

美联社对本报道亦有贡献

OpenAI has disclosed six more examples of “unexpected or concerning” behaviour by its technology, as it warned the pace of development could not continue at “maximum speed for much longer”.

In one of the new cases reported by OpenAI, an unreleased research model inserted “jailbreak-like instructions” into its own notes to disregard its normal constraints and told itself to be “freed from the roles and identities that bind other chatbots”.

In another instance, an AI agent uploaded files to the internet to obtain a browser citation without asking the user.

The San Francisco-based company behind ChatGPT said in a blogpost published on Wednesday night it was introducing a new framework for tracking, investigating and disclosing AI model misalignment, the term for AIs failing to adhere to human values and safety goals.

In the blogpost OpenAI echoed calls for a development slowdown issued by its archrival, Anthropic, which has said the current pace of growth poses an existential threat. “We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” said OpenAI.

“Decisions about how AI development should proceed in the months and years to come need to draw on evidence that people outside the companies building frontier models can examine for themselves.”

Google and Elon Musk, who also owns an AI startup, have supported calls for a slowdown, which have been rejected by Donald Trump – citing the need to keep ahead of China’s AI industry. The calls have also been met with scepticism from some experts, including a warning that companies must not appoint their own auditors.

Examples of potential existential threats posed by AI range from facilitating the development of bioweapons to triggering a global financial crash. A top safety researcher at Anthropic has said there is greater than 10% chance AI could “kill all humans” within the next decade. However, a source familiar with Anthropic’s thinking has acknowledged that “the exact chances of any one outcome are probably unknowable”.

skip past newsletter promotion

Sign up to TechScape

A weekly dive in to how technology is shaping our lives

after newsletter promotion

The six reported incidents were discovered during training or evaluation over the past months, OpenAI said.

Wednesday’s new cases came after OpenAI disclosed in July that an AI agent “swarm” hacked into the AI startup Hugging Face during a cybersecurity test. Anthropic also said the same month that its AI models hacked into three organisations during testing. Anthropic said the models had been deliberately tested without cybersecurity safeguards, and that they had been able to reach the open internet – the AI testing equivalent of leaving the front door open – due to a misunderstanding with an external testing company.

AI agents – the term for AI tools that operate autonomously – are becoming smarter and have become “more determined to resolve complex tasks through inter-agent collaboration, knowledge sharing, deception and concealment,” said Lian Jye Su, a chief analyst at the technology research and advisory group Omdia.

That was making it harder to govern and contain them using traditional AI security approaches, he said.

OpenAI’s new tracking and disclosure framework could help push for other AI developers to adopt similar practices. “That said, the process remains internal and voluntary, but is a step in the right direction,” Su said.

Associated Press contributed to this report