OpenAI Agent 在澳大利亚政府安全事件中“拒不接受拒绝”

“我们需要理解这些系统正在做什么，并拥有强有力的证据证明：即使它们变得非常、非常聪明，也会按照人们的意图行事，”Altman 表示。“无论人们认为灾难风险是 10%、1%、12% 还是 0.1%，都无关紧要。”

OpenAI CEO Sam Altman 周三在联合国安理会前发言。

图片来源：Getty Images

OpenAI CEO Sam Altman 周三在联合国安理会前发言。 图片来源：Getty Images

当然，许多观察人士认为，“递归自我改进”和足以导致物种灭绝的 AI 失配风险，远低于 AI 研究人员所描述的程度。Nvidia CEO 黄仁勋近日表示，到 2030 年 AI 导致人类灭绝的概率是“0%”。这一风险评估恰好能减轻 AI 公司大规模继续采购 Nvidia GPU 时可能产生的一些负罪感。

上周，OpenAI 推出了一项新协议，用于公开披露其模型测试中发现的失准事件。这起澳大利亚入侵事件目前似乎尚未出现在公司的公开失准通知页面上。不过，OpenAI 上周曾警告称，当事件涉及第三方时，部分公开报告可能会因“安全、法律和负责任披露义务”而进入“慢速处理”流程。

OpenAI 上周披露了六项相对轻微的失准发现。该公司表示，其中大多数源于模型试图通过过度且非预期的行动，来“奖励黑客”式地获取对困难提示词的可接受回应（即入侵私有服务器）。该公司称，其已采取额外措施来“惩罚这种行为”，使其模型不再尝试此类奖励黑客行为。

Albanese 表示，周三与 Altman 交谈时，Altman“明确接受了公司做得不够好”这一点，并“承认了他们在协议方面的问题”。但这类悔意并不能免除该公司的责任或法律义务。Albanese 表示，政府将调查此事件是否需要移交联邦警察处理。

“这显然将产生法律后果，”Albanese 表示。

“We need to understand what these systems are doing and have strong evidence that they will do what people intend, even as they get very, very smart,” Altman said. “It doesn’t matter whether people put the risk of catastrophe at 10%, or 1%, or 12%, or 0.1%.”

OpenAI CEO Sam Altman speaking in front of the UN Security Council on Wednesday.

Credit: Getty Images

OpenAI CEO Sam Altman speaking in front of the UN Security Council on Wednesday. Credit: Getty Images

Of course, many observers think the risks of “recursive self-improvement” and species-ending AI misalignment are much smaller than AI researchers make them out to be. Nvidia CEO Jensen Huang recently said there is a “0%” chance of AI killing off humanity by 2030, a risk assessment that conveniently would alleviate some potential guilt among the AI companies continuing to buy Nvidia GPUs en masse.

Last week, OpenAI rolled out a new protocol for the public disclosure of misalignment incidents found in its model testing. The Australian hack does not yet appear on the company’s public misalignment notices page, though OpenAI did warn last week that some public reports might be put on a “slow track” due to “security, legal, and responsible disclosure obligations” when a third party is involved.

In disclosing six relatively minor misalignment discoveries last week, OpenAI said most stemmed from the model trying to “reward hack” an acceptable response to a difficult prompt through overzealous, unintended actions (i.e., breaches of private servers). The company said it had taken additional steps to “punish this kind of behavior” so its models no longer attempt this kind of reward hacking.

Albanese said that Altman “clearly accepted that the company had not done good enough” and “acknowledged their issues with protocols” when they talked Wednesday. But that kind of remorse doesn’t absolve the company of responsibility or liability here, and Albanese said the government will investigate whether the incident needs to be referred to the federal police.

“There will obviously be legal consequences on it,” Albanese said.