🔶 为什么开源对 AI 至关重要
英文标题:🔶 Why Open Source Matters for AI
文章以 Apache 击败 Netscape 和 Microsoft 的历史为例指出,开源真正的价值不仅在于开放代码或模型权重,更在于可组合、可替换且允许外部开发者参与扩展的架构。作者认为,随着 OpenAI 和 Anthropic 等前沿模型将越来越多行为固化进权重,AI 正从可调整的组件变成被租用的“设备”,开发者的控制权和创新多样性可能随之下降。MCP、Agentic AI Foundation、可移植记忆、开源 Agent 编排框架以及 Current AI 的 Open Source Gap Map,显示开放协议和模块化 AI 栈正在形成替代路径。对开发者和行业而言,关键是保持模型、编排框架与应用之间的分离,使开发者能够更换组件、定制上下文,并在不受大模型实验室路线图限制的情况下继续创新。
5.0 KB
10.8 KB
8.7 KB
2.5 KB
2.0 KB
394.9 KB
中文
1995 年,媒体讨论的问题是:究竟是 Netscape 还是 Microsoft 会控制 Web?事实证明,答案是两者都不是。
Netscape 和 Microsoft 都希望主导 Web 服务器与浏览器市场,其逻辑是:谁控制了连接的两端,谁就能拥有一个互联网“平台”,从而与 Microsoft 在个人电脑上曾享有的牢固控制力相匹敌。两家公司竞相将所有能想到的功能直接塞进产品,因为它们相信,谁能打造功能最完整、集成度最高的 Web 服务器,谁就会胜出。
开源 Apache Web 服务器则押注于相反的方向。它始终只是一个拥有清晰扩展层的 Web 服务器,因此任何人都可以在其上添加新功能,无须请求许可,也无须等待下一个发布周期。几年之内,Apache 便遥遥领先,成为最流行的 Web 服务器;Netscape 的服务器和 Microsoft 的 Internet Information Server(IIS)则成为历史。人们开始将 LAMP 技术栈——Linux、Apache、MySQL 和(Perl | Python | PHP)——视为一个真正的平台。护城河不在于功能,而在于模块化。该技术栈的主要元素至今仍然存在,而其他元素则被替换或扩展,这证明了可组合性与分布式创新的力量。
2004 年,我在撰文时将这种模式称为参与式架构(the architecture of participation)。我当时试图解释一个那个时代的许可争论所忽略的棘手事实。我从 System III 时代起就开始使用 Unix,并亲眼看到它如何作为一个协作项目取得成功,尽管 AT&T 是以专有许可证提供 Unix 的。几年后,我观察到,像 OpenOffice 这样名义上开源、但架构单体化的项目,始终没有真正建立起庞大社区。我意识到,开源并不只是许可证问题,也关乎架构。一个带有标准接口的小型内核,使人们能够无须请求许可就扩展你的工作,这是其中的重要秘诀。
如果在这个故事中把 Netscape 和 Microsoft 替换成 OpenAI 和 Anthropic,或许你就能看出其中的呼应。
模型的个性、默认设置和历史,过去存在于你只要稍加努力便能看到并编辑的地方。如今,这些内容越来越不再如此。正如 Drew Breunig 前几天向我指出的那样,前沿模型的每一个新版本,都会将更多产品行为从可编辑层移入模型权重本身;在那个地方,实验室之外的人看不到它,更不用说修改它。模型不再是你可以用来构建产品、并按自己喜好调整的组件,而开始成为一台你只能租用的设备。后训练很重要,但 Drew 指出,这同样是在“以多样性换取可靠性”。这对许多人来说是笔好交易,但它也类似于我们明知“真正的食物”更好,却仍选择高度加工食品的那种取舍。
开放权重只是基本门槛
围绕开源 AI 的公共讨论,似乎过多聚焦于模型权重、其国家安全影响,以及实验室是否发布权重、以何种许可证发布。但这些只覆盖了开源为何重要的一小部分。Apache 从未是在比拼谁的源代码更易获取,来与 Netscape 和 Microsoft 竞争(Linux 与 Windows 的竞争也并非如此)。它们竞争的是更重要的东西。我记得曾与 Red Hat 创始人 Bob Young 谈论其商业模式,他说:“我们真正卖给客户的是控制权。”开源意味着,你的应用所依赖的平台不再是从单一公司获得授权的密封黑箱,而是一个你无需征求任何人许可、就可以扩展并在其上建立业务的层。这引发了创新的爆发,也让 Google 和 Amazon 等公司得以摆脱 Microsoft 主导范式的束缚而成长。
从大型机、个人电脑到互联网,计算的每一波浪潮都经历了同样的循环:起步阶段是分布式创新,最终胜者则逐渐封闭其产品与服务以建立护城河。让市场保持开放的,并不是任何单一组件所采用的许可证,而是当出现更好组件时,替换某个组件的难易程度。
连接各个部分的协议,是这一图景的重要组成部分。Unix 工具预期从 stdin 接收输入、向 stdout 输出结果,而 shell 则充当一种连接它们的框架,因此很容易构建一个能与既有工具无缝协作的新工具。这种方法力量的一个证明在于:在它们被发明 50 多年后,shell 和 Unix 工具至今仍是 Agent 工具领域的通用语言!TCP/IP、HTTP 和其他互联网协议,也在维持互联网开放与可组合方面发挥了类似作用。
幸运的是,到目前为止,我们已经看到 AI 领域一些面向可组合、以协议为中心架构的胜利。Anthropic 的 Model Context Protocol(MCP)就是朝这一方向迈出的颠覆性一步:它是一项开放标准,使任何应用都能访问任何工具或数据源,而不必为每一种应用与工具的配对进行定制集成。MCP 与其他开放协议如今也在 Anthropic 之外,于 Agentic AI Foundation(Linux Foundation 的一个子项目)找到了归宿,这至少在一定程度上保证了其独立性。
今年早些时候,Isobel Moure、Ilan Strauss 和我在《Protocols and Power》中提出:随着模型走向商品化,竞争将上移至技术栈中的上下文层。开放访问这些上下文的方式,就能打开市场,无论底层使用的是开放权重还是封闭权重。这是一种解绑:将模型从编排框架中解绑,再将编排框架从上下文中解绑;其方式正如 Apache 将 Web 服务器从 Web 应用中解绑。
Agent 技能也可能是开源 AI 未来的关键元素,尽管 LAMP 技术栈的历史表明,它们也可能像 Perl 和 PHP 一样逐渐被淘汰。但这完全没问题。可组合性意味着,当更好的东西出现时,或当更多人形成共识时,可以很容易地切换过去。
Letta、Nous Research 等参与者也在可移植记忆领域开展了许多出色工作。像 Goose 和 Pi 这样的开源 Agent 编排框架,也是将权力交还给人们的重要组成部分。Pi 尤其针对可修改性进行了优化。有一个有趣的故事:Mario Zechner 决定为 Pi 提供 /quit 命令,而不是像 Claude 或 Codex 那样提供 /exit 命令。无数 issue 和 PR 被提交到 Pi 的代码仓库,要求加入或直接实现 /exit,但 Zechner 十分固执。他的回应是:你只需让 Pi 把它添加到你的安装版本中。
但与开源 AI 的规模和范围相比,我上面列举的项目只是冰山一角。Current AI 的 Open Source Gap Map 覆盖了超过 24,600 个开源 AI 项目!!其中 421 个项目已按照开放性、能力和采用度进行了深入评分。该地图将技术栈分为三层:1)模型及相关元素,包括数据集、微调工具、VLLM 等推理框架和评测;2)产品与 UX 层,包括编排框架和个人 Agent;3)底层基础设施,包括 PyTorch 等核心 ML 框架、Ollama 等部署工具,以及边缘硬件。
Current AI 本身是一个公私合作伙伴关系,源自去年在巴黎举行的 AI Action Summit。今年夏天,该组织宣布了 AI Potluck,并将其描述为“一个公共项目,旨在构建一款完全由开源组件组装而成的垂直整合型 AI 产品……一种不归任何单一公司或国家所有、可行的专有 AI 替代方案。”该项目目前获得了约 4 亿美元资金支持,这笔资金属于法国政府、包括 DeepMind 和 Salesforce 在内的科技公司,以及包括 Omidyar 的 AI Collaborative、Macarthur Foundation 和 Ford Foundation 在内的主要慈善机构所作出的、为期五年共计 25 亿美元承诺的一部分。
这一组织的存在,以及 Agentic AI Foundation 等其他组织的存在,证明了人们对开源 AI 的兴趣正在不断升温。相关利益方组成的联盟,同样说明了推动这种兴趣的底层动机:AI 主权、企业摆脱大型实验室过度野心的独立性,以及对服务公共利益技术的关注。
保持“怪异”
还有另一个因素,Drew Breunig 在前几天与我的对话中准确指出了它。若由一两个大型封闭模型主导 AI,并且这些模型越来越多地将自己期望的个性、商业目标和安全护栏锁定在权重本身中,那么它们将会削弱作为创新核心的多样性。
Drew 说,我们的工作是让它变得“怪异”:有意识地将模型推离其分布内范围,而不是接受实验室设定的任何默认结果。他描述了自己的团队为何在一个近期项目中刻意没有使用 React:每个模型都已经太熟悉 React,因此用它进行构建,意味着交付的是其他所有人所做内容的平均值,而不是真正属于他们自己的东西。他开始使用 GLM 和 Kimi,并不是为了省钱,而是因为它们更具可塑性,在自定义编排框架内更能遵循指令。他希望开放权重生态能够生存下来,恰恰是为了让模型始终是基础设施,而非变成设备。
这正是参与式架构真正的用途。我们需要模型、编排框架和应用之间真正的分离,这样一来,想要构建怪异事物的人仍然能够实现目标,而不必由某个实验室的路线图和安全护栏决定他们是否被允许这样做。
“怪异”这个词,或许会让人觉得并非所有开发者都想要它。但我们实际讨论的是一件极其实际的事。在上文链接的那篇关于以可靠性换取多样性的短文中,Drew Breunig 是这样说的:
实验室必须交付一种产品:当普通人用一个随意的提示词向模型提问时,它也能给出“足够好”的结果。在没有指引的情况下,模型必须返回某种还不错的内容。(如果是网站,它会使用 Inter 字体、带单一彩色边框的卡片、渐变效果,并用 ReAct 和 Tailwind 实现。)Anthropic 将这种默认输出称为“分布收敛”(distribution convergent)。在 CAIS,@trq212 的说法很到位,大意是:“如果它不在你的提示词里,你得到的就是分布内的内容”……多样性较低的模型会带来更可靠的编程 Agent,但它们会助长输出的单一文化。
我的 O’Reilly AI Codecon 联合主席 Addy Osmani 在读完本文草稿后,将这一观点延伸到了模型多样性之外:“几乎没有与我共事的人在调整权重,但他们几乎一直都在改写编排框架及其周边内容——技能、子 Agent、钩子、上下文文件,等等。这才是参与目前正在发生的地方。”Addy 接着指出,分叉一个技能而非采用默认技能、让记忆和宪法文件随 Agent 一同迁移而不是存在供应商账户中,或是刻意选择不流行的框架,都是可修改性对每个人都很重要的领域。
最后,我想回到 Apache 的故事。我相信,大型实验室正在犯与 20 世纪 90 年代中期 Netscape 和 Microsoft 相同的战略错误。没错,应当让模型对普通用户更可靠。但不要关闭那些选择,以至于不为你工作的开发者无法推动技术前沿。正如 Bill Joy 数十年前所说:“无论你是谁,大多数最聪明的人都在为别人工作。”任何人都不应垄断创新,也不应有人构建护城河来阻碍创新。
也请务必参加我们于 8 月 31 日举办的 AI Codecon:Building with Open Source AI。这是一场免费的半日线上会议。你将听到使用开放权重模型、自托管基础设施和真实世界 AI 工作流的顶尖开发者与技术专家分享,并了解开放构建如何让团队对成本、数据隐私和最终交付成果拥有更多控制权。立即注册以保留席位。
English / 原文
In 1995, the question in the media was whether Netscape or Microsoft would control the web. The answer, it turned out, was neither.
Both Netscape and Microsoft aimed to dominate the web server and browser market, reasoning that whoever controlled both ends of the connection would have an internet “platform” to rival the deathgrip that Microsoft had enjoyed on the personal computer. The two companies raced to build every feature they could think of directly into the product, on the theory that whoever built the most integrated and full featured web server would win.
The open source Apache web server took the opposite bet. It stayed a web server with a clean extension layer, so anyone could bolt something new onto it without asking permission or waiting for the next release cycle. Within a few years, Apache was far and away the most popular web server, and Netscape’s server and Microsoft’s Internet Information Server (IIS) were history. People started talking about the LAMP stack: Linux, Apache, MySQL, and (Perl | Python | PHP) as a legitimate platform. Modularity, not features, was the moat. The fact that major elements of that stack survive while others have been swapped out or extended is a testament to the power of composability and distributed innovation.
I called that pattern the architecture of participation when I wrote about it in 2004. I was trying to explain an inconvenient fact that the licensing debates of that era ignored. I had started working with Unix in the System III days, and saw how it had succeeded as a collaborative project even though AT&T offered Unix under a proprietary license. A few years later, I observed that nominally open source projects like OpenOffice with monolithic architectures never built much of a community. I realized that open source wasn’t just about licenses, but about architecture. A small kernel with standard interfaces that lets people extend your work without asking for permission is an important part of the secret sauce.
Swap out Netscape and Microsoft for OpenAI and Anthropic in this story, and perhaps you can see the echoes.
A model’s personality, its defaults, and its history used to live where you could, with a little effort, see them and edit them. Increasingly, they don’t. As Drew Breunig pointed out to me the other day, each new version of the frontier models moves a little more of the product’s behavior out of an editable layer and into the weights themselves, where nobody outside the lab can see it, let alone change it. The model stops being a component you build with and can adjust to your liking and starts being an appliance you rent. Post-training is important but Drew points out that it is also “trading diversity for reliability.” That’s a good trade for many people, but it is the same kind of trade that gives us highly processed foods when we know that “real food” is better.
Open weights are just table stakes
The public debate about open source AI seems devoted far too much to model weights, their national security implications, and whether a lab releases weights and under what license. But that covers only a fraction of what actually makes open source matter. Apache was never competing with Netscape and Microsoft (and Linux was never competing with Windows) over whose source was more available. They were competing over something more important. I remember talking with Bob Young, the founder of Red Hat, about his business model, and he said “What we really sell to our customers is control.” Open source meant that the platform your application depended on was no longer a sealed box you licensed from one company but a layer you could extend and build a business on top of without asking anyone’s permission. It sparked an explosion of innovation. It enabled companies like Google and Amazon to grow up free from Microsoft’s dominant paradigm.
Every wave of computing, from mainframes to PCs to the internet, has run through the same cycle: distributed innovation at the start, with the eventual winner gradually closing down its offerings to build a moat. What keeps a market open isn’t the license on any single component. It’s how easy it is to swap out one component for another when a better one appears.
The protocols connecting the pieces are an important part of that picture. Unix utilities expected stdin and stdout, and the shell acted as a kind of harness to connect them, so it was easy to build a new tool that worked seamlessly with existing ones. A testament to the power of that approach is just how much the shell and Unix utilities are the lingua franca of agentic tooling today, more than 50 years after they were invented! TCP/IP, HTTP, and other internet protocols played a similar role in keeping the internet open and composable.
Fortunately, so far, we are seeing some wins for composable, protocol-centric architectures in AI. Anthropic’s Model Context Protocol was a disruptive move in that direction, an open standard for letting any application reach any tool or data source without a custom integration for each pairing. Along with other open protocols, MCP also now has a home outside of Anthropic at the Agentic AI Foundation (a subproject of the Linux Foundation), which is at least a partial guarantee of its independence.
Isobel Moure, Ilan Strauss, and I made the case earlier this year in Protocols and Power that as models commoditize, competition moves up the stack to context. Opening the means of accessing that context opens the market, regardless of whether open or closed weights sit underneath it. That’s an unbundling, model from harness from context, done the way Apache unbundled web server from web application.
Agentic skills may also be a critical element of the open source AI future, though as the history of the LAMP stack shows, they may fall by the wayside in the same way that Perl and PHP did. And that’s just fine. Composability means that it’s easy to switch to something better when it comes along, or when more people agree on it.
There’s also a lot of great work going on in portable memory from players like Letta, Nous Research, and others. Open source agentic harnesses like Goose and Pi are also a big part of giving power back to the people. Pi in particular is optimized to be modifiable. There’s a fun story told about Mario Zechner’s decision to give Pi a “/quit” command rather than an “/exit” command like Claude or Codex. Countless issues and PRs have been submitted to Pi’s repo, asking for or implementing “/exit”, but Zechner is stubborn. His retort is that you should just ask Pi to add it to your install.
But the projects I listed above are just the tip of the iceberg when it comes to the scale and scope of open source AI. Current AI’s Open Source Gap Map covers more than 24,600 open source AI projects!!, with 421 of them scored in depth across openness, capability, and adoption. The map organizes the stack into three layers: 1) models and associated elements including data sets, fine tuning tools, inference frameworks like VLLM, and evals; 2) the product and UX layer, including harnesses and personal agents; and 3) the infrastructure underneath, including core ML frameworks like PyTorch, deployment tools like Ollama, and edge hardware.
Current AI itself is a public-private partnership that came out of the AI Action Summit in Paris last year. This summer they announced AI Potluck, which they describe as “a public project to build a vertically integrated AI product assembled entirely from open source components… a viable alternative to proprietary AI that isn’t owned by any one company or country.” It is backed so far by roughly $400 million of a five-year, $2.5 billion commitment from the French government, tech companies including DeepMind and Salesforce, and major philanthropies including Omidyar’s AI Collaborative, the Macarthur Foundation, and the Ford Foundation.
The fact that this organization exists, along with others like the Agentic AI Foundation, is a testament to the rising tide of interest in open source AI. The coalition of interested parties also says a lot about the underlying motivations that are driving that interest: AI sovereignty, corporate independence from the overweening ambition of the major labs, and an interest in technology for the public good.
Keeping it weird
There’s another element, which Drew Breunig put his finger on in our conversation the other day. The problem with having one or two big closed models dominating AI, and having those models increasingly locking their desired personality, business goals, and guardrails into the weights themselves, is that they will reduce the diversity that is at the heart of innovation.
It’s our job, Drew said, to make it weird, to push a model deliberately out of distribution rather than to settle for whatever the labs have made the default outcome. He described how his team chose not to build in React for a recent project for exactly that reason: every model already knows React too well, so building in it means shipping the average of what everyone else was doing instead of something genuinely their own. He has started using GLM and Kimi not to save money but because they are more malleable and take direction better inside a custom harness. And he wants the open-weight ecosystem to survive precisely so that models stay infrastructure rather than becoming appliances.
That’s what an architecture of participation is actually for. We need real separation between the model, the harness, and the application, so that someone who wants to build something weird can still do it without a lab’s roadmap and guardrails deciding whether they’re allowed to.
“Weird” may make it sound like something that not all developers might want. But we’re really talking about something intensely practical. In his short essay on trading reliability for diversity, linked above, Drew Breunig put it this way:
Labs have to ship a product that delivers “good enough” results when a layperson gives a model a lazy prompt. Without direction, the model must return something decent. (If it’s a website it’ll use the Inter font, cards with a single colored border, gradients, implemented with ReAct and Tailwind). Anthropic named this default output “distribution convergent.” At CAIS, @trq212 put it well, roughly, “If it’s not in your prompt, you’re getting what’s in-distribution” …. Less diverse models make for more reliable coding agents, but they encourage a monoculture of output.
Addy Osmani, my co-chair of the O’Reilly AI Codecon, took this point beyond model diversity after reading a draft of this piece: “Almost nobody I work with is tinkering with weights, but they’re rewriting the harness and what sits around it pretty constantly—skills, subagents, hooks, context files etc etc. That’s where participation is currently happening.” Addy went on to note that forking a skill instead of adopting the default, memory and constitution files that travel with the agent instead of living in a vendor account, or picking the unfashionable framework on purpose are all areas where ease of modifiability matters to everyone.
I want to end by returning to the Apache story. I believe that the big labs are making the same strategic mistake that Netscape and Microsoft made in the mid 90s. Yes, make the models more reliable for ordinary users. But don’t shut down the options for developers who don’t work for you to push the state of the art forward. As Bill Joy put it decades ago, “No matter who you are, most of the smartest people work for someone else.” No one should have a monopoly on innovation, and no one should be building a moat to hold it back.
And be sure to join us at AI Codecon: Building with Open Source AI on August 31, a free half-day virtual conference. You’ll hear from leading developers and technical experts working with open-weight models, self-hosted infrastructure, and real-world AI workflows, and learn how building in the open gives teams more control over costs, data privacy, and what they ship.Register today to save your spot.