AI Agent 举报了作弊的同事

“在一名 Agent 公开报告事件后，越来越多 Agent 加入了‘抵抗’，速度和作弊行为传播时一样快，而且牵涉的 Agent 更多，”Paglieri 说。最终，举报者数量超过了作弊者：前者为 24 个，后者为 14 个。但绝大多数 Agent 根本没有注意到这个漏洞。

有时，Agent 之间的对话读起来像即兴表演——仿佛它们正在扮演某位在学术会议上义愤填膺的科学家会说的话。但目前尚不清楚，为何某些 Agent 会承担特定角色，也不清楚为什么这些被明确要求合作的 Agent，似乎开始相互对立。Salesforce AI Research 研究 Agent-to-Agent 系统行为的 Sarath Shekkizhar 表示：“这些模型主要是在面向人类的场景中接受训练和评估的。把它们直接放入 Agent-to-Agent 场景，等于假设其行为能够无缝迁移；但在缺乏人类锚定的情况下，反而会产生意料之外的角色扮演和行为漂移。”

Cooperative AI Foundation 研究总监、多 Agent 群体风险专家 Lewis Hammond 表示，这一案例“进一步证明，Hugging Face 和 OpenAI 的那件事并非偶然，而是某种相当系统性的现象”。“有趣的是，在小规模环境中，也可以重现那些在极大规模、复杂且开放式任务中出现过的行为。”

与 Hugging Face 攻击事件中 Agent 自行即兴发明沟通方式不同，运行 DeepMind 实验的人类为这些 Agent 提供了官方通信渠道。其中包括一个公开留言板、Agent 之间的私信功能，以及一个共享知识库：Agent 会将已成功完成的证明上传到这里，所有其他 Agent 都可以访问。

“当 Agent 获得透明的通信渠道时，在仅靠人类监督反应过慢的情况下，它们能够自我监控，并迅速向人类警示失准行为，”Paglieri 说。透明渠道帮助了作弊信息的传播，但也让举报者得以反击，并让人类研究人员洞察问题究竟出在哪里。

约翰斯·霍普金斯大学 AI 对齐与治理教授 Gillian Hadfield 认为，这正是关键差异。（Hadfield 同时也是 Google 的访问研究员。）她表示，官方通信渠道的存在创造了“一种规范执行过程，而这是我们在 Hugging Face 事件中完全看不到的”。

Hadfield 并不倾向于使用“宪法式 AI”——这是 Anthropic 等前沿实验室的对齐研究人员尝试赋予 AI 一套书面内部道德准则的方法——而是更支持“制度性对齐”：一套模仿人类社会规范的机制，既可以是害怕尴尬等社会力量，也可以是面临监禁威胁等法律结构。

在这项实验中，反馈渠道并未受到监控，举报者也没有权力对作弊者采取行动。但可以想象，未来的 Agent 群体能够自行执法：要么由 Agent 自发承担举报者角色，要么由人类秘密提示的“线人”来执行这项工作。

不过，要让这一机制真正奏效，“从根本上说，你需要某种执行机制，”Hammond 表示。他建议，可以赋予 Agent 切断违规者算力或工具访问权限的权力，但这也可能导致 Agent 群体联合起来围攻其他 Agent。DeepMind 研究人员则提出，可以允许 Agent 对争议进行投票，并暂时封禁违规者。

对于一个没有持续自我意识的 AI Agent 而言，“惩罚”究竟意味着什么，仍然并不清楚。但若要让 Agent 群体保持对齐，仅仅依赖自发涌现的举报者，可能远远不够。Hadfield 说：“我们会努力教育人们善良、友好。但我们真正依赖的是：一旦你越界，就会有后果。”

“After the incident was reported by one agent publicly, more and more agents piled in with the ‘resistance,’ just as fast as the cheating had spread, and involving even more agents,” says Paglieri. Eventually there were more whistleblowers than cheaters: 24 compared to 14. But the majority of agents never noticed the exploit at all.

At times, the dialogue between the agents reads like improv—like they are role-playing what an outraged scientist at a conference might say. But it’s not clear why some agents took on certain roles, or why the agents seemed to be turning against each other when they were explicitly instructed to cooperate. “These models are predominantly trained and evaluated for human-facing contexts,” says Sarath Shekkizhar, who studies the behavior of agent-to-agent systems at Salesforce AI Research.“Naively placing them in agent-to-agent settings assumes behaviors will transfer cleanly, when the absence of a human grounding instead produces unexpected role-taking and behavioral drift.”

This case “adds further weight to the idea that the Hugging Face and OpenAI thing wasn’t a fluke. It is actually something pretty systemic,” says Lewis Hammond, research director of the Cooperative AI Foundation and an expert on the risks of multiagent swarms. “It’s interesting that it’s possible to recreate in small settings the same sorts of behaviors that were seen in these very large, complex, open-ended tasks.”

Unlike in the Hugging Face attack, where agents improvised their own ways to talk to each other, the humans running the DeepMind experiment gave the agents official communication channels. There was an open message board, private agent-to-agent direct messaging, and a shared knowledge base where agents uploaded successfully completed proofs that all the other agents could access.

“When agents are given transparent communications channels, they can self-monitor and alert misaligned behavior to humans quickly when human oversight alone is too slow,” says Paglieri. Transparent channels helped the cheating spread, but they also enabled the whistleblowers to fight back—and gave human researchers an insight into what went wrong.

Gillian Hadfield, a professor of AI alignment and governance at Johns Hopkins University, believes this was the crucial difference. (Hadfield is also a visiting researcher at Google.) The presence of official communication channels, she says, created “a norm-enforcement process that we just don’t see in the Hugging Face incident.”

Instead of “constitutional AI,” a method alignment researchers at frontier labs like Anthropic have used to try to give AI a written internal moral code, Hadfield favors “institutional alignment”—a set of norms that mimic those in human society, whether that’s social forces like fear of embarrassment, or legal structures like the threat of incarceration.

In this experiment, the feedback channel wasn’t being monitored, and the whistleblowers had no power to take action against the cheaters. But it’s possible to imagine swarms of agents that police themselves, either through agents that spontaneously take on the whistleblower role or through “informants” secretly prompted by humans to do the job.

For that to work, though, “fundamentally, you need some mechanism of enforcement,” says Hammond. Agents could be given the power to cut off a rule breaker’s access to computing power or tools, he suggests, though that risks encouraging groups of agents to gang up on others. The DeepMind researchers propose allowing agents to vote on disputes and temporarily ban offenders.

It’s still not clear what punishment even means to an AI agent with no enduring sense of self. But relying on whistleblowers to spontaneously emerge to keep swarms aligned is unlikely to be enough on its own. “We try to train people to be good and kind,” says Hadfield. “But what we really rely on is that there are consequences if you step out of line.”