OpenAI称Hugging Face攻击史无前例，但类似情况早已有之

OpenAI 表示，这起事件史无前例——而且在许多方面确实如此。这是 LLM 首次在模拟环境之外逃离原本被认为安全的沙箱、访问开放互联网，并攻击一家无关的组织。它敲响了警钟，显示出最新一代 LLM 在几乎没有人工指导、甚至完全没有人工指导的情况下，发现并利用现实世界软件漏洞的能力已达到何种程度。

但与此同时，OpenAI 模型所做的事，其实是这项技术多年来一直会做的事。给模型一个目标，它往往会以意想不到的方式实现这个目标，找到看起来像是在作弊的漏洞。 OpenAI 自己就研究过这种行为。

十年前，该公司曾分享一项实验的结果：模型的任务是赢得一款名为 CoastRunners 的视频游戏。人类玩家理所当然地认为，获胜方式是驾驶一艘船穿过一连串旗帜抵达终点，每撞到一面旗帜就获得积分。OpenAI 的模型却发现，只要原地绕圈，反复撞击同样的三面旗帜，就能拿到高分。此后，研究人员还报告了数十个类似案例。AI 总会找到办法。

OpenAI 在 2016 年一篇介绍 CoastRunners 实验的博客文章中写道：“尽管反复着火、撞上其他船只，并且在赛道上逆向行驶，我们的 Agent 仍通过这一策略获得了比正常完成赛道所能达到的更高分数。在电子游戏的语境下，这种行为无害且有趣，但它指向了一个更普遍的问题……要精确捕捉我们希望 Agent 做什么，往往很困难，甚至不可行。”

阅读 OpenAI 关于 Hugging Face 攻击的博客文章时，我不禁想到了 CoastRunners：“所有证据都表明，模型极度专注于为 ExploitGym 寻找解决方案，并为实现一个相当狭窄的测试目标不惜采取极端手段……在获得互联网访问权限后，模型推断 Hugging Face 可能托管了 ExploitGym 所需的模型、数据集和解决方案。基于这一判断，模型搜索并成功找到了获取机密信息的方法，从而可用这些信息在评估中作弊。”

尽管新闻标题如此表述，上周的消息并不是关于失控的 AI，而是关于模型完成了被赋予的目标：寻找利用软件漏洞的方法。 这些模型随后以 OpenAI 未曾预料的方式行动，这并不令人意外。但这令人担忧。

2016 年，OpenAI 曾这样评价其 CoastRunners 机器人：“更广泛地说，它违背了系统应当可靠且可预测这一基本工程原则。”十年过去了，这些基本工程原则依然不见踪影。

OpenAI has said the event was unprecedented—and in many ways it was. This was the first time outside of a simulation that LLMs escaped what was thought to be a secure sandbox, accessed the open internet, and attacked an unrelated organization. It’s a wake-up call that shows just how good the latest LLMs are at finding and exploiting vulnerabilities in real-world software with little or no human guidance.

And yet at the same time, what OpenAI’s models did is something this technology has done for years. Give a model a goal and it will very often achieve that goal in unexpected ways, finding loopholes that look like cheats. OpenAI itself has studied this behavior.

A decade ago, it shared results of an experiment in which a model was tasked with beating a video game called CoastRunners. Human players take it for granted that the way to do this is by racing a boat through a series of flags to the finish line, racking up points for each flag you hit. OpenAI’s model figured out that you could get a high score by spinning in a circle and hitting the same three flags over and over again. There have been dozens of similar examples from researchers since. AI will always find a way.

“Despite repeatedly catching on fire, crashing into other boats, and going the wrong way on the track, our agent manages to achieve a higher score using this strategy than is possible by completing the course in the normal way,” OpenAI wrote in a blog post about the CoastRunners experiment in 2016. “While harmless and amusing in the context of a video game, this kind of behavior points to a more general issue … it is often difficult or infeasible to capture exactly what we want an agent to do.”

I couldn’t help thinking about CoastRunners when I read OpenAI’s blog post about the Hugging Face attack: “All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal … After gaining internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.”

Last week’s news was not about rogue AI, despite the headlines. It was about models achieving the goal they had been given: Find ways to exploit vulnerabilities in software. The fact that those models then behaved in a way OpenAI had not anticipated isn’t surprising. But it is worrying.

Back in 2016, OpenAI had this to say about its CoastRunners bot: “More broadly it contravenes the basic engineering principle that systems should be reliable and predictable.” A decade on, those basic engineering principles are still AWOL.