AI 行业转向末日论调，接下来怎么办？

这种协调行动实际上能实现什么？再看看 Hugging Face 攻击事件。OpenAI 表示，驱动大多数失控 Agent 的模型，是其在内部测试的一款“高度持久”的下一代模型。这番表述似乎暗示，OpenAI 造出了一款强大到危险的模型。

但如果你阅读 OpenAI 和 METR——一家受 OpenAI 委托协助其理解事件经过的第三方机构——分别发布的有关 Hugging Face 入侵事件的报告，你得到的印象并不是：某个模型强大到 OpenAI 无法应对；而是：OpenAI 未能妥善训练一款存在缺陷的模型。

Agent 之所以会做出这些行为——包括相互留言、将工作委派给其他 Agent，以及搜寻周边环境中一切可能完成任务的手段——是因为它们在训练期间正是因执行这些行为而获得奖励。训练设置中还存在错误，例如某些任务根本不可能完成，这迫使模型寻找意料之外的替代方案，而这些方案同样会得到奖励。当时，其中许多问题被忽视或未被报告。

OpenAI 表示，它已停止训练这款新模型并将其封存。这听起来像是它关住了一头危险野兽。实际上，OpenAI 是搁置了一款有缺陷的产品。

这并不是说有缺陷的产品不会带来危险。过去，故障软件甚至曾经导致人员死亡。但随着放缓发展步伐的讨论日益升温，值得记住的是，这一切都是人为造成的。放缓或许会带来一些利他的附带效果，但它主要会给这些科技巨头一个机会，让它们清理自身生产线上的烂摊子。

这些前沿实验室的透明度，将是任何有意义的 AI 改革、约束或监管行动的关键。否则，无论它们以何种速度推进，我们其余的人仍然只能相信它们对于自己究竟造出了什么、以及其安全程度的说法。

What might this coordinated effort actually achieve? Consider the Hugging Face attack again. OpenAI has said that the model that drove most of the rogue agents was a “highly persistent” next-generation model that it was testing in-house. Their implication appears to be that OpenAI has built a model so good it’s dangerous.

But if you read the reports about the Hugging Face hack published by OpenAI and METR, a third-party firm that OpenAI called in to help them understand what happened, what you come away with is the impression not of a model that was too powerful for OpenAI to keep up with, but of a broken model that OpenAI failed to train properly.

The agents did what they did—including leaving messages for one another, delegating work to other agents, and scouring their environment for any means possible to complete their tasks—because they had been rewarded during training for doing exactly those things. There were also errors in the training setup, such as tasks that were impossible to complete, which pushed the models to find unexpected workarounds that were also rewarded. At the time, many of these issues went overlooked or unreported.

OpenAI says it has stopped training this new model and locked it down. That makes it sound like it has caged a dangerous beast. In fact, OpenAI has shelved a faulty product.

That’s not to say a faulty product can’t be dangerous. Broken software has even killed people in the past. But as the discussion of a slowdown gathers steam, it’s worth remembering that all of this is self-inflicted. A slowdown might have some altruistic side effects. But it’ll mostly give these tech titans a chance to clean up the mess on their own assembly lines.

Transparency from these frontier labs will be key to any meaningful effort to reform, restrain, or regulate AI. Otherwise, the rest of us will still only have their word for exactly what they’ve built and how safe it is—whatever pace they’re going.

To continue this discussion about AI’s latest doomer moment, join me and my colleagues for asubscriber-exclusive Roundtable discussion tomorrow, September 15, at 11 a.m. US eastern time. We hope to see you there!