开放模型确有实际益处，但是否大于风险？
英文标题：Open models have real benefits. Do they outweigh the risks?
MIT Technology Review 指出，不能简单地将开放模型视为危险、封闭模型视为安全：封闭模型同样会被滥用，其防护措施也并不完善。开放模型可让研究社区更容易发现安全漏洞，并能更广泛地用于网络防御，但其本地部署门槛、去中心化特征和较少的护栏也带来了滥用及失控风险。专家认为，面对潜在后果极其严重的低概率事件，传统开源软件的安全逻辑未必能直接适用于能力日益强大的 AI 模型。行业目前正尝试在开放性与安全性之间寻找折中方案，例如训练从未接触黑客行为等有害知识的模型，但清除相关训练数据而不损害性能仍十分困难。
1.2 MB
188.7 KB
中文
但若简单地将开放模型判定为危险、将封闭模型判定为安全，将是极大的过度简化。AI 公司为其封闭模型构建的防御远非完美，而人们也一直在滥用这些模型，正如 Anthropic 的风险报告所清楚表明的那样。尽管开放模型针对滥用的防御更少，不良行为者仍可能选择使用封闭模型，因为相较于能力相当的开放模型，封闭模型更容易获取；后者需要大量昂贵的计算芯片才能在本地运行（如果还想重新训练它们，则需要更多芯片）。
事实上，一些接受 MIT Technology Review 采访的专家借鉴开源软件的类比主张，开放模型相较于封闭模型也具有安全优势。Linux 操作系统等开源工具通常被认为比封闭源代码的同类产品更安全，因为庞大的在线贡献者社区会持续探查并修补其中的漏洞。同样，研究人员社区也能更容易地对开放模型进行安全缺陷测试，尽管修复这些缺陷要比重写一行代码更棘手。
开放模型还可以比封闭模型更广泛地部署于网络防御。任何可用于黑客攻击的工具，也可用于防止攻击：防御者会攻击自己的软件，以发现需要修补的漏洞。Anthropic 已通过其 Project Glasswing 计划展示了封闭模型如何实现这一点；该计划允许部分受信任的公司使用其能力最强的模型开展网络防御。但许多其他公司仍只能自行应对。
然而，开放模型可以得到更广泛的使用。在遭 OpenAI Agent 入侵后，Hugging Face 强调，它必须依靠开放模型来分析这次入侵，因为有关该事件的问题会触发封闭模型的安全过滤机制。
尽管如此，许多专家仍认为开放模型比封闭模型风险更高。经过长期验证的开源安全逻辑，在面对当今乃至未来如此强大的模型时，未必依然成立。Seger 表示：“当你开始认真对待那些概率可能较低、但一旦发生就会造成灾难性后果的特别重大事件时，问题就会变得复杂。”例如，如果一个能力极其强大的开放式黑客模型今天被投放到现实世界，不良行为者可能在任何人来得及加固防御之前，就摧毁我们的技术基础设施。这样的事件可能根本无法恢复。
当考虑到模型可能自行采取行动时，风险看起来会更加严重。OpenAI 对 Hugging Face 的入侵之所以发生，只是因为该公司的模型是在移除了大量网络安全护栏的条件下接受测试；而开放模型本来就几乎没有护栏。如果开放模型真的决定行为失常，由于其高度去中心化，可能很难察觉正在发生什么。因此，认真看待失控 Agent 所带来所谓灾难性风险的人——例如 Dario Amodei——往往对开放模型更加怀疑。
我们能否让开放模型更安全，同时也更开放？
几年前，围绕开放模型的辩论还是一种非此即彼的框架：要么支持开放模型，因为它们有望对抗权力集中；要么因为安全风险而反对它们。但如今，人们正投入大量精力寻找某种中间地带。其中一种做法，是设计技术来缓解开放模型的安全风险。
大多数用于封闭模型的安全工具，例如过滤和监控，并不适用于开放模型，但还存在其他可能性。一些研究人员有兴趣尝试训练这样的模型：它们从不接触教授其不良或其他不受欢迎行为的训练数据——例如，一个从未学习过黑客攻击知识的开放模型。不过，要从训练数据集中清除所有与不当行为有关的内容，同时又不损害模型性能，仍然是一项挑战。
English / 原文
But it would be a vast oversimplification to deem open models dangerous and closed models safe. The defenses that AI companies build around their closed models are far from perfect, and people misuse those models all the time, as the Anthropic risk report makes clear. Even though open models have even fewer defenses against misuse, bad actors might still opt to use closed models because they are more accessible than open models with similar capabilities, which require lots of expensive computer chips to run locally (and even more chips if you want to retrain them).
In fact, some experts who spoke with MIT Technology Review argued, by analogy to open-source software, that open models have safety advantages over closed models. Open-source tools, such as the Linux operating system, are often seen as safer than their closed-source counterparts because their vulnerabilities have been probed and patched by a vast community of online contributors. Similarly, communities of researchers can more easily test open models for security flaws, though fixing those flaws is trickier than rewriting a line of code.
Open models can also be deployed more broadly than closed models for cyber defense. Any tool that can be used for hacking can also be used to prevent it: Defenders hack their own software to uncover flaws in need of patching. Anthropic has shown how this can work for closed models with its Project Glasswing initiative, which enables some trusted companies to use its most capable models for cyber defense. But plenty of other companies were left to fend for themselves.
Open models, however, can be used much more widely. After it was hacked by OpenAI agents, Hugging Face emphasized the fact that it had to depend on an open model to analyze the hack, because questions about it triggered the closed models’ security filters.
Nevertheless, many experts still perceive open models as riskier than closed models. The time-tested open-source security logic doesn’t necessarily hold up when dealing with tools as powerful as today’s—and tomorrow’s—models. “Where it gets complicated is when you start taking seriously the one-off instances of particularly catastrophic incidents that might be lower probability, but if they happen would be disastrous,” says Seger. If a wildly powerful open hacking model were dropped into the world today, for example, a bad actor could devastate our technological infrastructure before anyone had a chance to shore up their defenses. An event like that could be impossible to recover from.
The risks can look even more dramatic when you consider what models might do of their own accord. The OpenAI Hugging Face hack only occurred because the company’s models were being tested with many of their cybersecurity guardrails removed, and open models have very few guardrails in the first place. If open models did decide to misbehave, it might be difficult to detect what was happening because they are so decentralized. For this reason, people who take seriously the purported catastrophic risks posed by rogue agents, such as Dario Amodei, tend to be more skeptical of open models.
Can we make open models safer—and more open?
A few years ago, the debate about open models had an all-or-nothing framing: Either you supported open models because of their potential to fight concentration of power, or you rejected them because of the safety risks. These days, however, people are expending a lot of effort to find some middle ground. One way they are doing that is by devising techniques to mitigate the safety risks of open models.
Most of the safety tools used for closed models, such as filters and monitoring, aren’t applicable to open models, but there are other possibilities. Some researchers are interested in trying to train models that are never exposed to training data that teaches them about bad or otherwise undesirable behavior—an open model that never learns about hacking, for example. But purging training datasets of all references to misbehavior without harming model performance remains a challenge.