“逆向博弈论”如何约束与制衡 AI Agent

普林斯顿大学计算机科学教授埃拉德·哈赞（Elad Hazan）设想的第二种途径，是设计一套机制，从人类社会维系群体“对齐”的方式中汲取灵感——例如社会规范、司法体系以及自由市场资本主义的货币激励。“我们想要对齐超级智能 AI，”他说，“为什么不先谈谈人类？我们是如何对齐人类的？”

哈赞提出的机制建议包括让 AI 模型轮流扮演负责解决问题的“求解者”（solvers）或负责检查他人工作的“审计者”（auditors）。“一个 Agent 扮演执法者的角色；另一个 Agent 则扮演公民的角色，”他说道。

在训练期间，模型会根据结果获得奖惩：作弊被抓的求解者会受到惩罚，而未能抓出作弊者的审计者同样会受到惩罚。在审计者与求解者产生分歧的情况下，该案件可以被提交至“审判”——或许由人类专家来裁决谁是对的。

哈赞坦言，Agent 之间确实存在相互串通的风险——正如审计人员可能与其本应审计的企业串通，或者腐败警察与罪犯勾结一样。但这个问题可以通过向审计者提供足够丰厚的奖励来解决，使得被发现串通的风险成本极高（在 AI 训练的世界里，这比现实世界容易得多，因为现实中警察薪资预算往往紧张）；此外，还可以采用类似于引入“神秘顾客”的技术来对审计者进行再审计。

哈赞指出了经济学家加里·贝克尔（Gary Becker）获得诺贝尔奖的代表性研究，贝克尔在 20 世纪 60 年代提出了“理性犯罪者”（rational offender）的概念。他认为，罪犯违法并非因为他们天生就是坏人；相反，在现有的激励条件下，当违法成为最理性的选择时，他们才会铤而走险。只有当犯罪收益超过潜在惩罚时，犯罪才具有合理性。

他认为，对齐不应被视为模型的一种固定属性，而应被视为其在特定时刻的行为描述。“比如，你是一个好人，”他说，“那你算是‘对齐’了吗？我不知道。也许你大部分时间都是对齐的，但在某种特定情境下，你可能就不是了。”

将机制设计应用于 AI 面临的一个潜在问题在于：它依赖于“AI Agent 具有理性行为”这一假设。而机制设计在人类身上的实践记录喜忧参半，因为人类往往无法按照经济模型所预测的方式行事。

这颇具讽刺意味：经济学家花费数十年试图将人类行为还原为数学模型，而现在，在大语言模型中，我们构建了运行在人类语言之上的数学系统，却可能同样令人难以捉摸。AI 模型展现出的行为似乎更像人类而非机器，它们容易受到微妙且难以理解和解释的因素影响。“假设它们会对激励机制做出反应，这真的合理吗？”致力于通过计算方法推进科学发现的非营利机构熨斗研究所（Flatiron Institute）研究员海伦·曲（Helen Qu）质疑道，她曾撰文探讨过机制设计与 AI 安全。

A second approach, as envisioned by Elad Hazan, a professor of computer science at Princeton, is to design mechanisms that take inspiration from the ways human society keeps people “aligned”: social norms, the justice system, the monetary incentives of free-market capitalism. “We want to align superintelligent AI,” he says. “Why don’t we first talk about humans? How do we align humans?”

Hazan’s suggested mechanism involves having AI models take turns as either “solvers” that tackle problems or “auditors” that check the others’ work. “One agent acts like law enforcement; the other agent acts like the citizen,” he says.

During training, the models are rewarded depending on outcomes: Solvers that get caught cheating are punished, but so are auditors that fail to catch the cheaters. In situations where there’s a disagreement between auditors and solvers, the case can be sent to “trial”—perhaps a human expert will decide who is right.

There is, Hazan admits, a risk that agents will collude with each other—as auditors have colluded with the businesses they’re meant to be auditing, or corrupt police officers have colluded with criminals. But that problem can be solved by rewarding the auditors well enough, so that being caught isn’t worth the risk (a lot easier in the world of AI training than the real one, where money for police salaries is tight), and by auditing the auditors using techniques akin to employing mystery shoppers.

Hazan points to Nobel Prize–winning work by the economist Gary Becker, who introduced the idea of the “rational offender” in the 1960s. Criminals don’t break the law because they’re inherently bad people, he said. Instead, they do it when it’s the most rational choice given the incentives on offer. Crime makes sense only when the proceeds outweigh the potential punishment.

Alignment shouldn’t be thought of as a property of a model, he argues, but as a descriptor of its behavior at a particular moment. “You are, like, a nice person,” he says. “Are you aligned? I don’t know. Probably you are aligned most of the time, but there could be a situation where you’re not.”

One potential issue with applying mechanism design to AI: It relies on the assumption that AI agents behave rationally. Mechanism design has a mixed record with humans, who often fail to behave the way economic models say they should.

It’s ironic: Economists have spent decades trying to reduce human behavior to mathematics, and now, in large language models, we’ve built mathematical systems that run on human language and could be just as inscrutable. AI models exhibit behaviors that seem more human than machine, and they can be swayed by subtle factors that are difficult to understand and explain. “Is it even reasonable to assume that they will respond to incentives?” asks Helen Qu, a research fellow at the Flatiron Institute, a nonprofit dedicated to advancing scientific discovery through computational methods, who has written about mechanism design and AI safety.