秘密上传与“夸大自我”：OpenAI披露新型“失调”Agent事件

英文标题：Covert uploads and megalomania: OpenAI details new "misaligned" agent incidents

OpenAI表示，任何发现内部模型失调案例的员工都可向公司的安全与对齐团队报告，相关团队将决定是否立即公开、进一步调查或先咨询受影响第三方。公司将优先披露新的失调机制、已知行为的重大变化，以及挑战既有安全或缓解措施假设的发现。OpenAI称，即使事件的重要性尚不确定，也倾向于公开披露；对于反复缓解仍持续存在的问题，将持续发布更新。该公司同时承认，AI行业尚未充分解决对齐与监控问题，难以长期以最高速度负责任地扩大模型规模。

打开原文相关 5 · 机会 1

中文

我们会告诉你（几乎）一切

OpenAI 表示，任何注意到内部模型失调案例的员工，都可以标记该事件，提请其内部安全与对齐团队关注。这些团队随后将决定，该事件是否值得立即披露，还是需要进一步调查，以及是否需要在向公众发出警报前咨询任何受影响的第三方。

该公司表示，并非 OpenAI 模型以非预期方式行事的每个案例都会产生公开报告。相反，OpenAI 表示将“优先关注新的机制、已知行为中的重大变化，以及对安全性或缓解措施假设提出挑战的发现”。

与此同时，OpenAI 表示，它“倾向于披露，即使其重要性尚不确定”；其政策可能导致公开讨论一些“虚假偶发、并非更大模式的一部分，或并不预示未来发展的”案例。OpenAI 表示，如果某个特定失调问题在“反复尝试缓解后”仍持续存在，公司将在每次出现时提供更新。

如果某个案例被认定不值得公开披露，最初报告该案例的员工可以将分歧上报至 OpenAI 安全咨询小组的高级官员；在出现极端分歧的情况下，还可上报给 OpenAI 管理层。OpenAI 表示，随着时间推移，公司“计划与其他开发者、外部研究人员、行业标准机构和监管机构共同制定更具客观性的披露标准”。

该公司的公告还简要提及了广受讨论的、为给对齐研究留出更多时间而放慢进一步 AI 发展步伐的“节奏控制”概念。OpenAI 写道：“我们不认为 AI 行业已经在充分程度上解决了对齐和监控问题，因此无法在更长时间内继续以最高速度负责任地扩大规模。”

English / 原文

We’ll tell you (almost) everything

OpenAI said that any employee who notices an internal example of model misalignment will be able to flag the incident for the attention of their internal safety and alignment teams. Those teams will then decide whether the incident merits immediate disclosure or requires additional investigation and/or whether any affected third parties may need to be consulted before alerting the public.

Not every example of an OpenAI model acting in an unintended way will generate a public report, the company said. Instead, OpenAI said it will “prioritize new mechanisms, meaningful changes in known behavior, and findings that challenge assumptions about safety or mitigation.”

At the same time, OpenAI said it “favors disclosure even when significance is uncertain” and that its policy could lead to the public discussion of examples that are “spurious and not part of a larger pattern or suggestive of future developments.” If a specific misalignment issue continues to persist “despite repeated efforts to mitigate it,” OpenAI said it will offer updates each time.

If an example is deemed not worthy of public disclosure, the originating employee can escalate the disagreement to the senior officials at OpenAI’s Safety Advisory Group and, in cases of extreme disagreement, with OpenAI leadership. Over time, OpenAI said, it “plan[s] to develop more objective disclosure criteria with other developers, external researchers, industry standards bodies, and regulators.”

The company’s announcement also makes passing reference to the heavily discussed concept of “pacing” further AI development to allow more time for alignment research. “We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” OpenAI wrote.