在 Oxide 上运行自托管 AI Agent

我们的 Oxide 云计算机（Oxide Cloud Computer）具备当下 AI Agent 所需的恰当原语——提供强隔离能力的虚拟机、性能稳健的磁盘、完全可控的 VPC 网络，以及用于动态配置一切资源的弹性 API。即便如此，我们仍着眼于未来，致力于满足明日 AI Agent 的更多需求。

与现有及潜在客户就 AI Agent 展开的讨论，主要围绕以下几个核心主题：

• 安全性（Security）
• 规模扩展（Scale）
• 冷启动延迟（Cold-start latency）
• 清理机制（Cleanup）

大多数讨论都始于安全性，尤其是随着整个行业中报告的 CVE 数量持续攀升。我们之前已经探讨过虚拟机带来的强隔离特性，但我们的安全防护远不止于此。Oxide 云计算机拥有用于租户管理与资源配额强制执行的 Silo（孤岛）、用于网络隔离的 Project（项目），以及能够阻断不良网络流量的 VPC 防火墙规则。此外，我们移除了传统 BIOS，并结合 RFD 605 中所探讨的虚拟机远程证明（attestation）工作，为 AI Agent 工作负载打下了坚实的安全基石。

如果你需要运行大量的 AI Agent，你可能会关心规模扩展问题。只要 Oxide 云计算机的物理容量允许，你可以创建任意数量的虚拟机。目前我们正在研发多机架（multi-rack）能力，以将单个 Silo 的容量扩展至单机架之外。该功能发布后，将大大简化跨多机架管理 AI Agent 的复杂性。

从运维角度来看，我们的 API 让虚拟机配置变得十分简单，但在发出 API 请求后，虚拟机完成调度、启动并实际运行工作负载可能需要数十秒甚至数分钟。这会以冷启动延迟的形式影响用户体验。虽然预热池（warm pool）有助于缓解该问题，但无法彻底消除。在这篇博文中，我们接受了这种延迟的存在，但我们的目标是尽可能将其降至最低。

借助 Oxide API，清理虚拟机同样非常简单。然而，根据 AI Agent 的具体实现不同，往往没有一种直接简便的途径来获知工作负载何时已经执行完毕。在本文中，我们采取的做法是将「虚拟机处于停止状态」视为工作负载已完成的信号。这要求我们将 AI Agent 进程包装在一个 systemd 服务中，该服务会在 AI Agent 进程退出后执行 poweroff，同时配合一个 reaper 进程轮询 Oxide API 来识别并清理虚拟机。这种用户体验并不算理想，尤其是当 AI Agent 架构本身没有提供 delete 操作时更是如此；但我们目前正在探索如何让这种体验更像容器——当主进程执行结束时容器便自动退出。

讨论至此，我们尚未提及 microVM，但 microVM 原语正是许多基础设施提供商目前所采用的方案，用以改善运行 AI Agent 的用户体验，同时无需牺牲用户所依赖的虚拟机强隔离特性。

业界对 microVM 的定义各不相同，但在 AI Agent 语境下对它的讨论，促使 Oxide 内部开始思考 microVM 原语在 Oxide 系统中究竟应当以何种形式呈现。我们的虚拟机监控程序（Hypervisor）团队甚至进行了一次探索性预研（Spike），以测试我们究竟能以多快的速度启动虚拟机并运行其工作负载。事实证明，我们的表现与主流的 microVM 实现已经相差无几！

虽然目前还没有最终定论，但关于 microVM 的探讨被证明成效显著，并且得到了客户实际需求的印证。我们已处于极佳的起跑位置，准备从 microVM 中汲取灵感，推出一种更优秀的底层原语来运行 AI Agent 工作负载，重点聚焦安全性、可扩展性、冷启动延迟与清理效率。

如果你有兴趣在 Oxide 上运行 AI Agent，我们非常期待与你交流！欢迎联系我们，向我们介绍你的具体使用场景，并敬请期待后续动态。

Our Oxide Cloud Computer has the right primitives for today’s AI agents—​virtual machines with strong isolation, disks with solid performance, control over VPC networking, and an elastic API to dynamically provision everything. Even so, we’re looking ahead to meet the needs of tomorrow’s AI agents.

Conversations with current and prospective customers about AI agents have centered around the following themes:

• Security

• Scale

• Cold-start latency

• Cleanup

Security is where most conversations start, especially as the number of reported CVEs increases across the industry. We’ve already discussed the strong isolation provided by virtual machines, but our security story doesn’t stop there. Our Oxide Cloud Computer has silos for tenancy and resource quota enforcement, projects for network isolation, and VPC firewall rules to block undesirable network traffic. Additionally, our removal of the BIOS, along with the virtual machine attestation work discussed in RFD 605, provides a secure foundation for AI agent workloads.

If you run many AI agents, you may be wondering about scale. You can create as many virtual machines as you can fit on your Oxide Cloud Computer, but we’re working on multi-rack capabilities to expand a silo’s capacity beyond a single rack. Once released, this will simplify the management of AI agents across multiple racks.

Our API makes virtual machine provisioning trivial from an operational standpoint, but it can take tens of seconds or even minutes after an API request before a virtual machine is scheduled, booted, and running a workload. This affects users in the form of cold-start latency, which a warm pool can help mitigate, but not entirely eliminate. We accepted the latency in this blog post, but our goal is to reduce it as much as possible.

Cleaning up virtual machines is also trivial using the Oxide API, but, depending on the AI agent, there isn’t always a simple way to know when the workload has finished. In this blog post, we handled this by treating a stopped virtual machine as the signal that the workload has finished. This required us to wrap our AI agent process in a systemd service that runs poweroff after the AI agent process exits, along with a reaper process to poll the Oxide API for virtual machines to clean up. Not the best user experience, especially if the AI agent architecture doesn’t provide a delete operation, but we’re exploring how to make this experience behave like a container that exits when the main process is finished.

I’ve made it this far without talking about microVMs, but the microVM primitive is exactly what some infrastructure providers are using to improve the user experience of running AI agents without sacrificing the strong isolation they’ve come to rely on from virtual machines.

The definition of microVMs can vary across the industry, but discussing them in the context of AI agents has led to internal conversations about what a microVM primitive could look like for Oxide. Our hypervisor team even spiked on work to see how quickly we could launch a virtual machine and have it running its workload. Turns out we’re not far off from popular microVM implementations!

We don’t have anything set in stone, but the conversations about microVMs are proving to be fruitful and are being validated by customer requests. We’re well-positioned to release a better primitive for running AI agent workloads with a focus on security, scalability, cold-start latency, and cleanup, taking inspiration from microVMs.

If you’re interested in running AI agents on Oxide, we’d love to hear from you! Contact us, tell us about your use case, and stay tuned for what’s next.