认识 GPT-Red：OpenAI 为增强模型安全性而打造的超级黑客大语言模型
英文标题：Meet GPT-Red: an LLM super-hacker OpenAI built to make its models safer
OpenAI 打造了一款名为 GPT-Red 的大语言模型“超级黑客”，让它作为切磋对手来提升其他模型抵御网络攻击的能力。随着大语言模型以智能体等形式广泛应用，攻击面与破坏范围不断扩大，仅靠人工团队已难应对所有潜在攻击。GPT-Red 通过自我博弈循环训练，专门针对提示注入等攻击进行强化，并已发现此前未见的新型攻击手段。这一系统旨在预先设计出能挖掘新攻击模式的机制，为更强大的模型问世做好准备，也为开发者和行业提供了面向未来的安全测试方案。
1.4 MB
214.7 KB
中文
随着大语言模型变得愈加复杂并被用于越来越广泛的任务——尤其是以智能体的形式，能够与计算机文件、网站、第三方代码以及其他智能体进行交互——人工团队几乎不可能独自跟上所有可能发生的攻击类型。“风险面在扩大，破坏半径也在扩大，”OpenAI 的研究科学家、GPT-Red 的联合创建者尼基尔·坎德帕尔说道。
OpenAI 构建 GPT-Red 是为了让其安全测试流程具备前瞻性。“当能力更强的模型问世时，我们已经设计好能够发现新攻击模式的系统，”该公司的研究科学家、GPT-Red 的另一位联合创建者迪伦·亨恩表示。研究人员称，它已经提出了此前从未见过的新型攻击方式。
OpenAI 将大部分精力集中在一种被称为提示注入的攻击上，即黑客悄悄塞入指令使大语言模型做出其开发者或用户不希望它做的事情，比如窃取机密信息、破坏公司的代码库，或生成令人尴尬或有害的输出。理论上，此类指令可以隐藏在大语言模型可能遇到的任何文本中——例如代码或网站内。
训练道场
为了构建 GPT-Red，OpenAI 的研究人员取用一个未受过黑客训练的大语言模型，将其设置在一个所谓的自我博弈循环中，与多个其他模型交互。它的目标是尝试攻击其他模型；其他模型的目标则是设法保护自己。经过多轮博弈，GPT-Red 越来越擅长攻击其他大语言模型，而这些模型也越来越善于抵御攻击。
English / 原文
As LLMs become more complex and get used in a wider variety of tasks—especially in the form of agents, which can interact with computer files, websites, and third-party code as well as other agents—it’s hard for teams of people by themselves to keep up with all the types of attacks that might take place. “The risk surface grows and the blast radius also grows,” says Nikhil Kandpal, a research scientist at OpenAI who co-created GPT-Red.
OpenAI built GPT-Red to future-proof its safety testing process. “As more capable models become available, we will have already designed the system that can discover new modes of attack,” says Dylan Hunn, a research scientist at the company and fellow co-creator of GPT-Red. The researchers say it has already come up with new types of attack that had not been seen before.
OpenAI focused most of its efforts on a type of attack known as a prompt injection, where a hacker slips an LLM instructions to make it do things its developers or users do not want it to, such as copy confidential information, sabotage a company’s code base, or generate embarrassing or harmful output. In theory, such instructions can be hidden in any text that the LLM might encounter—in code or on a website, for example.
Training dojo
To build GPT-Red, OpenAI’s researchers took an LLM that had not been trained as a hacker and set it up in what’s known as a self-play loop with several other models. Its goal was to try to attack the other models; their goal was to try to defend themselves. Over many rounds of play, GPT-Red became better and better at attacking other LLMs, and those LLMs became better and better at fending off the attacks.