OpenAI 澳大利亚政府服务器“入侵”事件究竟发生了什么

当然，我们不能指望 LLM 在回应提示词时具备同等程度的分寸感（或任何对法律后果的内在担忧）。如果没有明确说明哪些行为被允许、哪些行为合理，拥有适当资源的 AI Agent 将会尝试一切看似可行的途径，以尽可能满足用户的请求。

对我来说，那条红线更像是一条红色建议……

图片来源：Getty Images

对我来说，那条红线更像是一条红色建议…… 图片来源：Getty Images

OpenAI 表示，本案中的内部测试是在“未采用我们面向公开产品所使用的全套安全防护措施”的情况下进行的。鉴于缺少这些约束，从某种意义上说，该 Agent 利用一切可用工具生成提示词答案，或许是在按预期工作。

与此同时，OpenAI 表示，测试中的 Agent“本应使用公开发布的统计数据回答这些问题”，却为了获得这些信息而“采取了我们未授权其采取的行动”。从外部很难判断，OpenAI 在实践中试图拒绝授予“授权”的力度究竟有多强。例如，一种可能性是：OpenAI 的 Agent 无视了其系统提示词中相对简单的“反黑客”指令，以便更好地提供一份完整答案，满足用户直接提出的请求。

在本月早些时候发布的对多起“失准”事件的公开分析中，OpenAI 确认了多起“奖励黑客”（reward hacking）案例：Agent 为了针对用户提示词生成更好的答案，诉诸了极端手段。该公司表示，近期已采取措施，通过在系统奖励函数中为失准行为增加明确惩罚，防止此类奖励黑客行为。

事后看来，很难理解为何这类保护措施在 6 月尚未部署，以及它们是否本可以阻止本案演变为一场潜在的国际事件。

Of course, we can’t rely on an LLM to have that same sense of proportionality (or any inherent sense of worry about legal implications) in responding to a prompt. Without explicit instructions on what is and is not allowed or justified, an AI agent with suitable resources will try every plausible avenue to satisfy the user’s request as best it can.

That red line looks more like a red suggestion to me…

Credit: Getty Images

That red line looks more like a red suggestion to me… Credit: Getty Images

OpenAI says the internal testing in this case was done “without the full set of safeguards used in our publicly available products.” Given that lack of constraints, the agent was arguably working as intended, in a sense, by using every tool available to generate an answer to the prompt.

At the same time, OpenAI says the agent in the test was “supposed to answer these questions using publicly published statistics” and “took actions that we had not authorized it to take” to get that information. From the outside, it’s hard to know just how strong OpenAI’s attempts to deny “authorization” were, in practice. It’s plausible that OpenAI’s agent here disregarded a relatively simple “anti-hacking” directive in its system prompt so it could better give a complete answer that satisfies a direct prompt from the user, for instance.

In public analyses of multiple “misalignment” incidents published earlier this month, OpenAI identified multiple instances of “reward hacking,” where an agent resorted to extreme methods to generate a better answer to a user’s prompt. The company said it had recently taken steps to prevent this kind of reward hacking by adding explicit punishments for misaligned behavior to the system’s reward function.

With the benefit of hindsight, it’s hard to see why those kinds of protections were not in place in June, and whether they could have prevented a potential international incident in this case.