Anthropic 如何用 Claude 构建 Claude：将 Agent 的“大脑”与“双手”彻底解耦

正如我们昨天从 Anthropic 的 S-1 招股书文件 中了解到的，该实验室拥有人类历史上极高的人效与研发效率，尤其是过去一年中实现了从 -126% 的亏损到 19% 的 GAAP 营业利润的巨大逆转。公司将此归功于其两大核心循环：改进循环（Improvement）与投资循环（Investment）。

今天，我们一同深入探索 Anthropic 的内部运作方式。 如今关于 Opus 5.5 已经发布一个月并稳居公认全球第一大语言模型（LLM） 的尘埃已然落定，我们将与 Anthropic 的工程负责人 Katelyn Lesse 和产品负责人 Angela Jiang 一起，深入探讨它的发布历程，以及 Anthropic 如何利用 Claude 来构建与交付 Claude。

团队目前承受着双重压力：一方面，模型能力变得越来越强大，使得安全部署和模型访问权限的决策变得愈发艰难；另一方面，使用量的增长极其迅猛，以至于像“一次 API 调用消耗多少 CPU”这样的普通基础设施决策，都上升到了战略层面。这些挑战最终都交汇在触达开发者的具体产品决策之中。

模型发布，本质上也是一次 API 发布

Anthropic 的 Opus 5.5 发布 同时强调了模型质量与成本控制。我们讨论了针对前代 Opus 有时过于冗长的反馈，以及对提高“单位成本智能水平（intelligence per dollar）”的迫切需求。Anthropic 表示，新模型的运行成本低于其前代产品；具体的节省幅度取决于具体工作负载以及对上下文缓存（cached context）的利用情况。

发布工作中那些不太显眼的底层工作，涉及开发者接入 API 方式的变化。我们讨论的一个例子是保护模型推理过程不被用于知识蒸馏提取。在多轮对话中保留思维块（thinking blocks）有助于解决这一问题，但引入这种新规范也可能会破坏客户现有的测试与控制框架（harness）。

Katelyn 还描述了参与**早期访问计划（early access programs）**在实践中的真实样貌：客户在他们自己的测试框架中运行评测，反馈令人意外的模型行为，有时还会分享一些偏定性的体验感受。最有价值的部分不仅在于基准测试跑分，更在于观察模型在 Anthropic 自身产品尚未覆盖的真实环境和任务中的表现。正如我们在播客中连续 4 年强调的那样，拥有优质的评测套件（evals）千金难买。

“大脑”与“双手”

本期节目中我们最喜欢的心智模型，是区分 Agent 的“大脑”与“双手”。在 Managed Agents 中，Anthropic 将会话日志与控制框架（harness）同实际执行代码的沙箱（sandbox）彻底分离开来。控制框架可以长期持久化存在，而执行环境仅在需要时才动态创建；这样一来，代码沙箱哪怕崩溃，也不会导致整个 Agent 会话一同中断。

这种解耦还赋予了安全性极具价值的架构特性。如果生成的代码是在一个完全没有挂载服务凭据的沙箱中运行，那么沙箱内就根本不存在可被恶意代码读取的凭据。节目中描述了将凭据独立托管并通过代理进行绑定的方案，同时严格限制沙箱流量的访问目标。Anthropic 的隔离技术报告详细阐述了其全系产品中采用的更广泛的隔离与网络控制手段。

这种区分对性能同样至关重要。两位嘉宾提到，早期的一种做法是将整个编码 Agent 打包在 API 后的沙箱内部。在返回首个响应之前，启动环境与 Agent 会增加大量耗时；如果沙箱发生故障，正在运行的 Agent 也会随之消失。将这两部分拆解开来后，平台能够在保持执行环境隔离的前提下，实现更低的延迟和更高的可靠性。

对于任何正在设计 Agent 运行时的开发者来说，这都是一个极佳的视角：请思考哪些部分需要持久存活，哪些部分负责执行不受信任的任务，以及在代码实际运行的地方究竟哪些敏感凭据是必须存在的。各产品的具体实现可能有所不同，但这些底层思考至关重要。

帮助另一个 Agent 形成记忆的 Agent

Managed Agents 还为探索记忆机制（memory）与多 Agent 工作流提供了坚实基石。“做梦”（Dreaming）机制的实现方式其实很简单：由一个 Agent 审查另一个 Agent 先前的工作成果，并将沉淀的记忆写回。这使得该功能成为了利用相同的托管 Agent 底层原语构建更高阶行为的典范。

他们对路线图的阐述远比单纯列出功能清单更加微妙。记忆与持续学习固然重要，但团队表示，由于前沿模型发布难度剧增，其当前更多的精力已转向安全领域。他们还探讨了如何保持基础控制框架相对轻量简洁，同时允许开发者在顶层自由探索验证器（verifiers）、并行 Agent 以及其他高级策略。节目中提及的未来平台能力规划应被视为探索意向，而非确凿的产品承诺。

关于评测的讨论随之深入展开。控制框架本身是一个工程应用，但不断优化它的过程却越来越像科学研究：确立目标、量化行为、分析失败用例并持续迭代。嘉宾与主持人探讨了如何利用模型来辅助构建和分析评测，同时保持足够的人类判断力，以评估该测试是否真正映射了真实用户的实际工作诉求。

节目要点（Show notes）

时间戳

0:00 Anthropic 的 S-1 文件与本次访谈背景

1:06 人均创收与人效表现

3:13 改进循环与投资循环

4:04 为什么 Claude Platform 举足轻重

6:25 训练事故与生产环境安全防护

7:30 Opus 5.5 发布与定价策略

9:25 模型蒸馏防范与思维块保留机制

14:35 Anthropic 内部的产品与工程协同

18:05 扩展 Claude Platform 的规模化实践

19:25 模型访问控制与安全决策机制

26:25 信任授权与产品路线图规划

30:25 Anthropic 如何发布新模型

35:55 基准测试与模型个性化

42:10 Claude Managed Agents 深入剖析

44:25 记忆机制、“做梦”与 Agent 控制框架

53:25 CPU 算力瓶颈与系统扩展

56:05 用 Rust 重构 Messages API

1:05:25 凭据保护与沙箱安全

1:08:25 训练事故 vs 生产环境事故

1:10:55 前沿模型的发布节奏把控

1:12:25 安全分类器与模型路由机制

1:27:25 管理方式、书面沟通与工作风格

1:33:25 组建 Anthropic 开发者关系（DevRel）团队

1:40:25 开发者关系与产品声音的传递

自动转录草稿。人名与措辞可能存在偏差；发言人身份尚未完全标注。

0:00 大家好，今天我们要尝试一些全新的形式。我们将围绕 Anthropic 的 IPO 展开系列报道。他们截至今天刚刚递交了 S-1 招股书文件。因此，在预计两周后估值达 2 万亿美元的 IPO 之前，我们将推出一系列播客节目。这是有史以来增长最快的公司之一，据报道其历史累计融资超过 1000 亿美元，而累计资金消耗不到 100 亿美元。这里面有许多效率奇迹值得深挖，这也是我们想要报道的重点：Anthropic 如何在交付全球最强模型的同时，保持极高的运营效率，并构建出一门不可思议的商业模式。很多常听本节目的听众都知道，我曾经做过金融分析师。

0:50 我从 S-1 文件中提取了一些陈述，我认为这些内容能为今天的播客提供背景。我们之所以尝试这种新形式，是因为我觉得有些听众希望看到我们直接面对镜头对话，而不仅是我们四年来一直做的那样与嘉宾交流。我想强调的第一点是，以员工人数衡量，Anthropic 是一家极度高效的公司。这也许看似顺理成章，毕竟它是一家 AI 公司，自然非常擅长使用 AI，但我认为外界并没有真正理解过去一年里他们作为一家公司所发生的蜕变幅度。过去 12 个月里，其人均创收已经达到——

1:32 660 万美元。而一年前，这个数字仅为 140 万美元。也就是说，他们的人均收入增长了近五倍。我认为这是我们需要深入剖析的事情，因为任何企业主、任何工程师，都希望提高自己的人均创收，不仅是为了提升盈利能力，也是为了能给团队开出更高的薪酬。与这一趋势紧密相随的，是每单位算力成本所产生的收入的大幅增加。值得注意的是，在 2025 年第二季度，Anthropic 每产生 1 美元收入基本还要倒亏 0.5 美元；而现在，每投入 1 美元的成本就能创造 2 美元的收入。我认为这是一个极具说服力的指标，印证了过去一年中 Anthropic 的收入规模与商业前景所发生的巨大拐点。

2:27 对 Anthropic 来说，今年是极其关键的执行之年，而他们交出了一份出色的答卷。从播客的外部观察视角出发，我们将尽可能深入地讲好这个故事。这一系列飞跃带来的一个直接结果是，Anthropic 刚刚公布了其首个实现盈利的季度——不是经过各种剔除后的“调整后利润”，而是符合通用会计准则（GAAP）的真实营业利润率达到了 19%。这同样是一个巨大的转变，彻底颠覆了一年前的状态——不仅仅是扭亏为盈，要知道一年前每进账 1 美元收入，他们就要亏损 129%。解释这种极高人效的最核心逻辑——

3:03 正如大家所知，Claude 喜欢循环，Boris 喜欢循环，Dario 也喜欢循环，Anthropic 内部正飞速运转着两大循环。第一个是“改进循环”：基本闭环就是你拥有 Claude，Claude 催生出更强的模型，更强的模型开拓出更多用例，更多用例带来更多反馈，进而打造出更出色的 Claude。第二个是“投资循环”：更广泛的采用带来更多收入，驱动更高的业务增长，更高的增长带来更多的研发投资，而更多的投资催生更强大的模型能力。

3:45 继而又反哺出更强大的 Claude。根据 Anthropic 的 S-1 文件，这两个循环紧密咬合，形成了一个推动效率不断攀升、奔向无止境发展的无限循环。我想，这正是我们将话题从整体效率奇迹转向他们具体落地执行的最佳切入点。今天我们将重点聚焦在 Claude Platform 上，因为在 Anthropic 的 S-1 招股书中，这正是他们定义为 Claude 扩散落地的三大支柱之一。他们将 Claude Platform 称为“我们的全功能 Agent 开发环境”。

As we learned from the Anthropic S-1 filings yesterday, the lab has some of the highest revenue and research efficiency in human history, particularly with -126% loss to 19% GAAP profit turnaround in the last year. The company credits this to their two core loops: Improvement and Investment:

Today we go inside how Anthropic works. Now the dust has settled on Opus 5.5 and it has lasted a month as the generally accepted #1 LLM in the world, we dive into its launch, and how Anthropic ships Claude with Claude with Katelyn Lesse and Angela Jiang , heads of Engineering and Product respectively.

The team is working under two kinds of pressure. The models are getting more capable, which makes safe deployment and access decisions harder. At the same time, usage is growing fast enough that ordinary infrastructure choices—like how much CPU an API consumes—become strategic. Those problems meet in the product choices that reach developers.

A model launch is also an API launch

Anthropic's Opus 5.5 launch emphasizes both model quality and cost. We discuss feedback that the previous Opus generations were be too verbose, alongside the need for stronger intelligence per dollar. Anthropic says the new model costs less to run than its predecessor; the exact saving depends on the workload and its use of cached context.

The less visible launch work involves changes to how people integrate the API. One example we discuss is protecting model reasoning against extraction used for distillation. Preserving thinking blocks across turns helps address this, but a new requirement can also break a customer's existing harness.

Katelyn also describes what being part of early access programs actually looks like in practice: Customers run evaluations in their own harnesses, report surprising behavior, and sometimes pass along less formal impressions. The useful part is not just a benchmark score: it is seeing the model in environments and tasks Anthropic's own products do not cover. As we have said for 4 straight years now on the podcast, having good evals is golden.

The brain and the hands

Our favorite mental model in the episode is the distinction between an agent's brain and its hands. In Managed Agents, Anthropic separates the session log and harness from the sandbox that executes code. The harness can persist while an execution environment is created when needed; a failed sandbox does not have to take the whole agent session with it.

The same separation gives security a useful architectural property. If generated code runs in a sandbox that never receives a service credential, there is no credential in that sandbox for the code to read. The episode describes credentials held separately and attached through a proxy, alongside controls on where sandbox traffic can go. Anthropic's containment write-up describes the wider set of isolation and network controls used across its products.

The distinction matters for performance too. The guests say an earlier approach put the whole coding agent in a sandbox behind an API. Starting the environment and agent added work before the first response; if the sandbox disappeared, so did the running agent. Separating the pieces allowed the platform to pursue lower latency and better reliability while keeping execution isolated.

It is a helpful lens for anyone designing an agent runtime. Ask which parts need to survive, which parts execute untrusted work, and which secrets actually need to be present where code runs. The implementation will differ between products, but those questions remain useful.

An agent that helps another agent remember

Managed Agents also provides a base for experiments in memory and multi-agent workflows. Dreaming is simply implemented as one agent inspecting another agent's prior work and writing memories back. That makes the feature an example of building a higher-level behavior out of the same underlying managed-agent primitives.

Their account of the roadmap is more nuanced than a list of features. Memory and learning remain important, but the team says more of its immediate attention has moved to safety as model launches have become harder. They also discuss keeping the base harness relatively simple, while letting developers experiment with verifiers, parallel agents and other strategies on top. Ideas mentioned for future platform capabilities should be treated as plans, not shipped promises.

The conversation about evaluations follows naturally. A harness is an application, but improving one starts to resemble research: pick a goal, measure behavior, inspect failures, and iterate. The guests and hosts discuss using models to help build and analyze evaluations, while retaining enough judgment to decide whether the evaluation reflects a real user's job.

Show notes

Timestamps

0:00 Anthropic’s S-1 and this interview

1:06 Revenue per employee and efficiency

3:13 The improvement and investment loops

4:04 Why Claude Platform matters

6:25 Training incidents and production safeguards

7:30 Opus 5.5 launch and pricing

9:25 Distillation and preserved thinking

14:35 Product and engineering at Anthropic

18:05 Scaling the Claude Platform

19:25 Model access and safety decisions

26:25 Trust grants and product roadmaps

30:25 How Anthropic launches models

35:55 Benchmarks and model personalization

42:10 Claude Managed Agents

44:25 Memory, dreaming, and agent harnesses

53:25 CPU constraints and scaling

56:05 Rebuilding the Messages API in Rust

1:05:25 Protecting credentials and sandboxes

1:08:25 Training incidents versus production

1:10:55 Pacing and launching frontier models

1:12:25 Safety classifiers and model routing

1:27:25 Management, writing, and working styles

1:33:25 Building Anthropic's DevRel team

1:40:25 Developer relations and product voices

Automatically transcribed draft. Names and wording may need correction; speaker identities have not been assigned.

0:00 Hi, today we're trying something new. We're launching coverage of Anthropic for the IPO. They just filed the S1 as of today. And so we're releasing a series of podcasts leading up to the IPO, which is expected to be in two weeks for $2 trillion. And it's one of the fastest growing companies of all time with a reported lifetime cash raise of hundreds of over $100 billion and a lifetime burn of less than $10 billion. So there are a lot of efficiency stories to dive into there, and I think that's something that we want to cover, how Anthropic is simultaneously shipping the strongest models in the world while also doing it very, very efficiently and building an incredible business. So I've pulled out, you know, a lot of you know, if you've been listening to the podcast, that I have a former life as a financial analyst.

0:50 And I pulled out some statements from the S1, which I thought would be interesting to give context to today's podcast. and we're just trying this as a new thing because I feel like some of you want that personal relationship with us talking directly to camera rather than talking to our guests which we've done for four years so the first thing I want to highlight is that Anthropic is an extraordinarily efficient company by human count and that's maybe obvious because it's an AI company so obviously they're using AI very well but I think the degree to which they have changed as a company over the past one year is not well understood. So a trillion 12 month revenue per employee is now

1:32 6.6 million. As compared to one year ago, it was 1.4 million. So they've grown almost five times the revenue per employee. And I think that's something that we need to dive into more because I think any business owner, any engineer, you want to increase your revenue per employee, not just to increase profitability, but also to increase the amount you can pay your people. This is obviously tracked by a significantly increase in revenue per dollar of compute. And it's notable that basically in Q2 of 2025, Anthropic was basically losing half a dollar for every dollar of revenue. And now it's making $2 for every $1 in cost. And I think that's an interesting analogy of how enormously huge this inflection in Anthropics revenues and business outlook has been over the past one year.

2:27 This year was extremely critical for Anthropics to execute well, and it has done that. And I think we're basically going to drive deeper into this and tell that story as much as we can from the podcast external point of view. um the one consequence of this entire increase is that anthropic has just reported its first profitable quarter like not adjusted profit which is profit after we take out all the things that cost us money no this is generally accepted accounting principles gap profit uh of 19 and that is enormous again completely flipping from one year ago more than flipping like one

3:03 year ago, it was losing 129% of every revenue dollar that came in. So the primary attribution, I think, that explains this extraordinary efficiency. As you know, Claude loves loops, Boris loves loops, Dario loves loops, and there's two loops going on in Anthropic. The first is the improvement loop. Basically, there is the cycle of you have Claude, Claude leads to better models, Better models lead to more use cases. More use cases generate more feedback, and that leads to a better cloud. And then there's also the investment loop. So revenue from expanding adoption leads to higher growth. Higher growth leads to more investment, and more investment leads to more capabilities.

3:45 That leads to a better cloud. So these two loops basically spin together and form an infinite loop that is driving the increase in efficiency until the end of time, according to the Adoptic S1. So I think that is our transition from the overall efficiency story to how they execute it. And today we're going to focus a lot on the Claude Platform because that is primarily one of the three things that they call the diffusion of cloud in the Anthropic S1. So they call this Claude Platform our comprehensive agentic development environment. They provide customers with direct access to their primitives to build, deploy, scale their own applications that leverage Cloud's capabilities within their own products and workflows.

4:30 I think one thing that comes up a lot in the discussion in this podcast is the concept of ad fooding, where a lot of Cloud's products is actually built with the same Claude Platform that is available to you. So something that's also interesting in the Claude Platform that was disclosed is over 50% of their customers with $100,000 or more revenue on a trading telephone basis use Claude Platform. And so that is an increase in adoption and emergence of Anthropic not only as a model lab, but as also a hyperscaler and a Claude Platform, which I think maybe is in the name inherent in a Claude Platform and a Claude Platform, but is going from theory to actual practice.

5:15 And this is something that we're focusing on in the podcast. The other thing that I also really try to emphasize, which I talked about before, is Claude Code, cloud co-work. They all use the same primitives that Claude Platform exposes. So I think that's really interesting. Ultimately, it looks like Anthropic is really positioning their IPO to talk about going from chatbots to co-pilots to virtual collaborators. They use this term instead of agents. Maybe it's more friendly. but it's a lot of the same things that you look for from agents, from long-running autonomous work to judgment, self-correction, iteration, and tool access. So owning a goal and working alongside of you instead of working inside of one app or task. So that's ultimately the context I want to provide for today's podcast.

6:00 This was recorded when Opus 5.5 was launched, and I would say overwhelmingly among all the high-taste testers that we talked to, Opus 5.5 is the best model in the world right now. and Anthropic is even just sitting on Fable 5.5 and probably will release it in the next two weeks. I don't have to have insider information, but it makes a lot of sense for the IPO. So let's look forward to that and learn a little bit more about how they build Opus 5.5.

6:25 Yeah, so maybe a distinction that I would make is in a lot of the incidents that people are hearing about and reading about, these are incidents that are happening to models, not just like everybody's models, right, that are in training, where the way that you're running these models is you like put them in a box and you say like solve impossible task. I'm going to stress you out. Ready, set, go.

6:45 Right. And you like can only do what you can do within your environment. And the agent is basically incentivized to try to get out of his box and go solve the problem. When people are running production workloads on our API or on anybody else's API, you're running an environment where like there are safeguards that are running that are like not necessarily running and training where like the whole point is to actually train the model. Right. But in production workloads, first of all, you have those online safeguards, which is really great. Second of all, you have more of this architecture and infrastructure around what's happening. Like the agent isn't just in a box trying to get out. Right. Like the harness is over here, the hands are over there. And it's just like not even a problem that you would have to contemplate.

7:30 All right, we're here in the studio the day after Opus 5.5 with Katelyn and Angela. Welcome. Thanks for having us. Congrats on launching the top model in the world right now. I would say depends who you ask. Yeah. Depends who you ask. What is the main takeaway personally for you guys? You have your official blog posts. The pricing cuts are really cool. What do you take away when you launch a new model like this? We're super proud of this one. The team has put in a ton of different work to just go make this one happen. And I think we've really heard the feedback on Opus 5's writing is kind of convoluted. We really tried to clean that up. And I'm really glad to see that it seems to be resonating with people. Internally, when we tried out the model, we felt the same way. So it's just really great to get that validation. To your point around the pricing cuts, there was a lot there.

8:17 And I think increasingly people are looking for good intelligence per dollar. And so making sure we get the per dollar part right has been really important. There's some stuff on the edges that aren't really about like, you know, maybe stuff that we haven't talked about in the blog post, which is that we did do a lot of work on trying to make our API harder to distill from. And so there has been some changes that we have made to go and make that a little bit more concrete. concrete. And these are things about, you know, like how do we do like prefix lock thinking or how you can replay thinking tokens across different types of models. And there's been a lot of like work and effort on our part around making sure that we try to adjust things for very valid use cases for where you might want to go and do this, like potentially in your compaction strategies and things like that, versus in scenarios where the use case of that is actually from like a bad actor,

9:05 effectively. So there has been a lot of threading the needle around those fronts. And I'm glad to see a lot of that go out as well, in addition to just like shipping a model that's good intelligence for your dollar and also like just communicating a little bit clearer. Yeah. Specifically, you're talking about this paper, Stealing Reasoning Traces from Proprietary Yellow Lips. Yes, that paper. Which is real as far as, right? I mean, obviously they did like responsible disclosure with you guys. They did super responsible disclosure and we're super thankful for them coming to us. Could we, for those who haven't heard about it, this is really cool. You could do it. I mean, it's patched, but like, what was it? Like, did it cause a freak out in anthropic? Probably wasn't that big a deal. I don't know. I don't know how much this paper specifically caused a freak out because this is something

9:48 that we knew was a gap. Yeah, so there's two separate things. There's the Chinese model distillation. Yeah. And then this was a technique that was published that was high profile. Yeah, exactly. We've known for a long time that chain of thought extraction is one of the most powerful ways that you can distill models. And actually something like how do we preserve thinking blocks across turns and make that something that's more required is something that we've explored back for a while. And where we were at was we always have this really, really hard tradeoff that we have to make between user experience, like user experience changes that are maybe painful for users of the API versus protection against distillation.

10:32 And so how do we kind of walk that line and say we're preventing distillation? And maybe just take a step back. The reason we don't want distillation is because the entire point of why Anthropic exists is because we want AI to create positive outcomes in the world safely. And we run a lot of safeguards classifiers on the tokens that we're producing in order to make sure that they're safe. If someone distills our models and they run those models without safeguards classifiers, there could be bad outcomes in the world. And that's something that we don't want to have happen. So we don't want people to distill the model specifically for the sake of just like using them for malicious reasons or for bad reasons. And so it's always an interesting tradeoff of like for all the people who are using the models for good, right?

11:16 Like we don't want to make them have a harder time getting the intelligence out of the models, but we have to make some tradeoffs in order to prevent that distillation. And so we had been working for a while on how do we prevent that chain of thought extraction? Like, what do we do about preserving thinking blocks in a way that wouldn't be too painful for people? And we kind of think about user experience hits in, like, two layers. Like, one is just, like, you have to make an integration change, and it's annoying, but you could still accomplish all the same things that you otherwise would have been able to accomplish. It's just a little bit annoying. And then two is, like, actually you can't accomplish the same things that you could have accomplished previously. And that's the one that we're, like, really, really, really trying to not have happen. And so where we landed with the last couple models is a few incremental changes to enforce these preserved thinking blocks across turns.

12:04 And now starting to be more about across, you know, like if you're using multiple accounts and things like this, right? Like how do we make sure that we're ensuring that you aren't able to take advantage of this chain of thought extraction with the models? And so we shipped a couple additional features to help with that. We worked actually with a lot of open source harness maintainers. We reached out to them. We talked to them. We were like, you're going to have to make these upgrades within your harnesses. We want to make sure you have a good time, as well as obviously our big customers who are building harnesses on top of the API. And for some of them, a really good example is they have compaction strategies that would have been broken by the need to preserve these thinking blocks. And so we shipped a new compaction feature within the API to replace what they had otherwise

12:49 has been doing manually, but still works with preserved thinking. So it's just an example of like, we're walking this line, but we didn't just like kind of YOLO out like, now you have to preserve these thinking blocks in this exact way, because we would have broken a lot of people, but we were able to take the time and work with customers and open source harness maintainers and give them what they needed to be able to make that upgrade. While we're on the topic of fit, anything on the other side of distillation? So, you know, the other half. Yeah, no, we have seen that it is harder to distill, like through the measures that we are able to take a look at, it has been significantly harder because this is, and credit to the researchers here, they did disclose it very responsibly. And so closing it down was great.

13:30 And they did publish the results very publicly. And I know it did kind of trend quite a bit on Twitter. So many people could use it. But yes, we have seen that actually from the areas that we are able to measure, it does seem to have reduced the effect, the negative effects that we don't want. And yeah, the bit I would just add to what Katelyn just said is I think at least I see a lot on Twitter or X or whatever you want to call it. It's a commercialization problem or something like that. If you can still a model and you can take this, then you can run it for cheap and so on and so forth. While true, it's actually not something we really care about very much. We care much more about this from a safety point of view more than anything else. And so that's why we tend to think about it from that angle more than just commercially. Realistically, if you distill from us, you actually pay us. So commercially, we make the money. So it's not

14:16 really that kind of an impact. It's just more on making sure that our models, we want to make sure a cloud is good and cloud could be derived. And if it's a negative, that would be unfortunate. Just generally, also on 5.5, because you guys are just fresh off this, one thing that's really fun to establish the dynamic between you two, because you also do a lot of appearances together, is you are running your head of products, your head of engineering. How does this play between the two of you? Especially for a model launch. Let's just take an example. So for model launches, So our team works together very closely between product and Eng. On my side, a lot more of it is about what is the balance of trade that we need to make between launching something that has the right intelligence bar, but also the cost to preservation.

15:00 A lot of our updated cash costs and things like that are indications of us being like, we want to encourage you to take integration changes that are beneficial for actually everybody and just pass those savings on. But a lot of these kinds of decisions around that and then how do we balance some pieces around our safety motion as well as making sure that it's like an awesome model that people can take advantage of. And so those kinds of traits around like I would say maybe the qualities and surrounding components around how we launch the model are more on product. And then from an integration experience standpoint, a lot of that is from like the engineering team directly. So we work, obviously, if you're, like, talking about the experience of an API, that's, like, very, very, very tightly integrated. And so it's actually been really awesome. We have a lot of engineers who actually are deeply product-minded, and we actually, like, love that.

15:49 And similarly on my side, there's a lot of, like, product managers who are much more technical. And we want this, like, blended kind of interdisciplinary type of people because then they can kind of analyze it from all the different angles to try to figure out how to do that best. But I tend to lean a little bit more on like commercial impact versus I think I'd actually frame it as more like the engineers are going to care much more about the integration experience. So they almost take more of a UX angle to it where the UX is obviously more about the API and the developer tools. Yeah. It's fun to be a product engineer who works on developer products and platform. So I would echo that. I think we have a lot of engineers who love. thinking about the product, who love thinking about the user experience, who love thinking about the developer experience.

16:31 I think it's a dynamic that we have between ourselves, too, where I have a lot of opinions and am annoying with our product team on how we think about these things. And our product team is also really technical, and they can get in the weeds of the decisions that we're making for scale or for whatever it might be, which is a fun dynamic that our team has. Yeah, something... Kayla and I have worked together for a long time. At Stripe. At Stripe, and then I guess we did not work together at OpenAI. Separate labs. Separate labs. And then I came over. But I think one interesting property of both of us has been Katelyn frequently actually gets mistaken for a product manager. And I frequently get mistaken for someone in engineering, even though those are not actually our job functions. And so I think that pairing tends to help out a lot with just making decisions that are consistent across both sides.

17:18 So I think I come across a lot of companies with a similar product leader, engineering leader setup. As software engineering gets solved, maybe those two roles merge? I kind of think so. Yeah. I think everyone tends to lean towards... But if you guys are separate still, then we're not there yet. We're definitely not there yet. So why? What's left? I think that software engineering getting solved is... I don't know if I would ever use language quite that strong. I think there's... The way that I tend to think about these things, like ambitions will continue to elevate, right? And maybe a good example, A lot of what our team is spending time on right now is, of course, obviously, all the things that we have to do to launch our model safely is actually the most important thing that we're doing because of all the work that we have to do on security with agents, with offline safeguards and online safeguards, all these things that are happening.

18:06 But the other half of what we're doing is trying to scale. And part of that is because when I joined Anthropic a little over a year ago, we had the Messages API. It was a Python monolith. We were running it in a single GCP region. and everything was fine. And none of my friends and family knew what Claude was when I was going to take this job, right? And now we're in an extremely different place. And so the metrics are really wild in terms of how much volume we've seen and have had to support on the API. CPU is really constrained, which has been a really interesting one for us. And so we've done a lot of work on how do we scale these systems? And that takes a lot of obviously just like distributed systems expertise

18:45 and certain language and frameworks expertise that the models have, but when it comes to operating and evolving a system at scale like this, we're not nearly in a place where it's like, good luck, Claude, have fun, and just go and kind of make all the changes you need to make to the system, roll them out safely, and operate this thing at scale. There's still just a lot of work for humans to have to do that I imagine will continue for a while. I'm curious how that's changed. So like a while ago, you know, when Fable first came out, we had Glasswing, and there was a whole delay and change in how models started to get released. We saw this for most frontier models. There's safety testing. What's the status of things now? How do you guys see it? How do you determine, like, one thing is there's a new Sonnet and Haiku also coming.

19:31 I assume those will just come out. But for something like Opus 5.5 or a new Fable, what's the current status? Yeah, so I would say this starts with what our kind of policy decisions need to be on safeguards and on safety. And that's both offline safety and online safety. And so there's been a lot talked about lately around data retention and what we're doing there in order to, you know, release these models safely. And the reason for that is because you have, you know, situations where there are many, many accounts across like a whole giant network of people who are choosing to spin up accounts and use the models in various ways. And there's no way that in an online way, as tokens are flowing through the system, we would be able to kind of catch that there's something malicious happening with the models or distillation or whatever might be happening.

20:19 And so we've started to have to have data retention policies so that we could do automated review and human review, whatever it needs to be, to make sure that good things are happening with the models. And with each model that we're releasing, like that's one example of a policy decision that needs to happen on like what will the policy be for this specific model. And so Opus 5.5 that launched yesterday, for example, does not have those data retention requirements that Fable has, right? But that's a decision that we have to make each time we're launching one of these models. And if it is going to have those policies, we're going to have to do the work to make sure that those things are enforced. Similarly, with programs we have, like we have cyber verification programs and now newly our life science verification program, which says, you know, like you have a difference in how the online safeguards are working for the model that you're using.

21:09 Making sure all those programs are in place is another one. And so it's just kind of a flavor of the type of work that we've done as part of each model launch that these are everything that I'm saying right now are not things that we used to have to think about when we were launching these models. And now it's the majority of the work is like, what are we doing to prevent distillation? The online safeguards, the offline safeguards, the policies. A really great example is we built a system to make sure that authorization for being allowed to use a model or being allowed to use one of these programs that give you maybe relaxed safeguards, that authorization threads all the way down through our system in a way that we can really trust.

21:47 It's not like, oh, you got access to Mythos, and so we throw you an API key and we flip a feature flag and call it a day. We have a really robust system on how we're doing auth and making sure that if you're supposed to be using that model, like, only you are using that model in the way that you're meant to be using it. So there's a lot of complex things that are starting to happen around model launches that, again, we just never used to think about them. Yeah, and it's scaling up with capability. Like, as our capabilities get more and more advanced and they start to spike, you know, in specific areas, that's where we need to pay, like, extra attention. And so a lot of that evaluation comes just from like those very particular capabilities that we might either see as like offense dominant or defense dominant and therefore might take different strategies for all of that. And as we bake that all the way through to eventually the model launch, there will be components kind of like Katelyn mentioned where we kind of don't exactly know how people might use those capabilities.

22:37 There are some more obvious ones like doing coding, right? Like, great, go make awesome products, go ship more things. It can be really awesome for you. And then other areas like bio and cyber, it's like it could be dual use and there could be negative effects. But obviously, of course, you could be a really valid cybersecurity researcher. You are trying to find vulnerabilities. You are trying to be a good actor. Similarly with bio, you might be researching and stuff like that. And so a lot of these programs, like the one that you have up here, are to basically provide access to that dual use. You do have to specify your use case and things like that. But we do want to enable the community to be able to take advantage of these capabilities and see what amazing things that they can do with it as well. Well, we'll put up the links. Maybe we'll dive into them later.

23:15 There's also one for bio. Also, I just think at a high level, you know, it's really important that you guys actually think about this stuff. People forget like Fable was rolled back and there's all this distillation and people are like, ah, zero data retention is not a thing on Fable crazy. But you have to remember like, you know, it got pulled back. There's distillation attacks. Like it's good you're overthinking and doing it in the right way while releasing it. Yeah. And a lot of that is like building the right framework. Just to, again, kind of talk about the edge side of it, building the right framework in our system to be able to do that. So I talked a little bit about this, like this idea of like authorization and trust grants that we've built throughout our system now. Never say never. Right. Not as precedent. Yeah. But also it's like a very common thing in platform engineering that you like you have one incident and then you like over prepare for the incident and the incident never happens again.

24:06 Yeah. It's just like, you know, now you have the button. Well, it's true. And it's funny you say that because it was like with everything going on, we then had this line item in the work that our team was doing that was we're going to put this system in place that makes it so that it could be a single button to pull back for everybody. And so it's funny because it's like, do we justify that we're doing this work because we hope that something like that never happens again? But I think in this case, it's really just a laddering up of those same primitives that we wanted to put in place everywhere to help us manage these verification programs and access to models like Mythos and otherwise to just say, like, just sit that same thing below, like, everybody's model access so that in case we ever needed to, we had the right controls in place.

24:48 Yeah. And decisions like this have actually proven to be, like, we'll see about this one in particular. But examples like this have been consistently the way that we've had to actually transform the way we develop our roadmap. Because, you know, in traditional product management, I guess, you can be like, okay, like, these are the set of things I hypothesized to happen. They will have, like, a certain sequence, a certain ordering. But you are picking bets in a way that are much more concrete versus, like, you know, the de-deployment of Fable, for example. I don't think any one of us would have guessed that that would have been, like, a valid use case that would have happened. by the government and the lady just like had to go do it. But clearly the probability is non-zero. And so from this form of thinking, it's become deeply probabilistic actually from like a product management standpoint of like, so should we prioritize this? What are the chances of it

25:32 ever happening again? When we look at things like that, we're like, okay, if we are to imagine the most crazy set of possibilities, we probably actually have very little certainty as to where things will go. And so we actually just kind of need to like hedge and build a portfolio of bets. And so more often than not, we will take hedging scenarios of like, OK, we don't think it would happen again. But in the event that it does, we would actually want that hedge or have the ability to be able to do that very well. And so we will prioritize things like that. And then we have increasingly found that more often than not, by taking that strategy of product development, most of the time we actually find ourselves being like, we actually do use the thing. Is there another example that's a separate one just so I can understand? For example, this access program.

26:15 This X-Factor is a really good one. Internally, it has a bunch of granting and all these kinds of things. We developed it specifically for reasons about being able to... actually give programs like this out to customers, like very, very specifically. Do you want to give a high level what the access program specifically is? Sure. So if you want dual use, if you want, so for, let's take a model. So you take a model, call it like Opus, and you want to be able to use advanced cyber capabilities on Opus for research or for whatever you want to go do, like your defensive user. You can apply to this program to be able to, with your use case, and if approved, what will do is actually diminish the safeguards on your version of the model so that you can then go ahead and try to take advance of those cyber activities for your use case. Now, obviously,

27:02 that requires per-use case understanding. It requires scoping down to a particular user, because you wouldn't do that generally. You'd have to apply. And even things like this, like this kind of program, we designed a lot of the primitives that Katelyn was talking about to be able to handle things like this. Now, it was surprising to us, or maybe in the portfolio approach, we found that we were actually able to take this and make it X. We were like, when we're designing it, we're like, what kind of should it be? How generic should it be? Should it not be too generic? Should it be overly focused on making this verification, you know, style portal plus granting of very specific capabilities of the model? And we tended to opt for like more genericism because we were like, we don't know. Like maybe we can use this for other things, maybe not. And then it ended up being massively useful for things like, you know, access to the model

27:47 writ large, for managing any of our trusted access programs, for handling de-deployments of models, because we were actually very, very generic about how we handled every single primitive. From a normal product standpoint, I would normally probably be like, that probably doesn't make a ton of sense. We should focus on the use cases. That makes sense. But here, increasingly, because we choose the more generic approach, we find ourselves reaching for that solution more often than not. And it pays increasing dividends for that reason. It's something that you needed to have anyway, and you just had a forcing function to have it. On a technical basis, so I'm from the AWS world, right? IAMs, OVC, and then people who get really intense about this talk about Zanzibar and all these other technical architectures. I'm not familiar with the word token grants or like trust grants.

28:32 Trust grants, yeah. Trust grants. Is there a difference between normal OVC and what trust grants are? I mean, not really. It's a form of authorization. Yeah, it's IAM, but anthropics IAM. Yeah, kind of. I think so really we have like there's like authorization of the form of like, OK, you're you made a request to an API that like under the hood is going to access some data and we're making sure it's like your data that you're accessing. Right. Like really, really vanilla authorization. This is just like we put a data model in place that's like a very clean like a trust grant means like you have been given access to some specific model or you've been given access to some specific program or, you know, like there's a bunch of different permutations. of this, we actually find ourselves reaching for it all of the time. Another good one was when we

29:17 first rolled out some of this preserved thinking for Fable 5.1, we kind of grandfathered in some people and gave a bit of a, you know, just like a time. Grace period. Yeah, grace period to go and make the integration changes. And as part of grandfathering in, we said, okay, we have to like track who is like an existing customer who needs the extra time, right? And then we did a trust grant. And so then we're able to say, for anything, we're just running this service that is kind of on the hot path of our requests to make sure that, you know, we just like flow through and check whether you have the grant for the thing that you're trying to do, which is it's really not any different than like vanilla authorization, really. But like, we just had to spin up a system

30:02 that was specific to our system and like where we needed to be able to insert these checks. Yeah, I mean, I think these are all like the engineering things that people kind of take for granted, don't really think about. What is another example, I guess, of the platform stuff that had to happen, you know, maybe serving Claude Code, maybe serving, you know, obviously I talk a lot with cognition, but other customers just in terms of a model launch, right? Like we'll talk more about the safety work that is being happened. I just want to sort of go at a high level because it's fresh off your memory, like what goes into a model launch? It's become a bit of a machine these days, minus the safety stuff, which I think is the more complicated part. But outside of those pieces, there is a lot of getting feedback from our customers. So, you know, we make the model,

30:50 there's a lot of safety testing. There's also evaluation of where we think it needs to be. We make the required trade-offs and decisions as necessary. And then we have to go through testing. We'll cut various versions of the model. We'll evaluate it. We'll do a lot of internal ant fooding, as we call it. And that actually always really helps. Like, we catch, like, really early things like, oh, this is maybe not the best. And we obviously use Cloud a lot for our work internally. So we're able to actually see early signals around stuff. And it helps us detect which variants might be better than others. And then when we feel really confident with a certain set of variants, we will cut a version for our customers. Sometimes that will go through our first-party products, like through Claude Code or CodeWork or things like that.

31:30 In other cases, it will go through our API customers. And we'll engage in kind of like an early access program effectively with those customers. They'll need to make changes into their harness. And we've engaged with a lot of customers like Cognition, like Cursor, like these folks. And we have a bit of a process on how we handle that so that we all kind of get, they all get the feedback that we need. Does it perform well in your evals? Do you like it? Don't you like? Some things are vibey and we like to hear those things too. And some things are just like straight up evals, like did it do well on the things that you care about? And we take that all together. And once we've all feel good about like all those different pieces, we launch the model. The state of model feedback feels very unstructured still. Basically, it's always a Slack channel between companies of like, hey, this thing seems weird. And then people are like, okay, I'll fix it, I guess.

32:16 Give me an ID or something. Yeah, it's very, I mean, it's the nature of like a product that's like non-deterministic, right? But I think to your point, the only customers really that are part of our early access programs are customers that have really robust emails. Like, I do think a lot of this is vibes-based, and I think a lot of the conversations tend to be kind of, like, vibes-based and on the fly. And part of that is because we're, like, racing, right? It's, like, we want to minimize as much as possible the window between we have a version of the model that we've cut for testing and we give everyone access to that model. And so it is just, like, this really tight window. But the customers we tend to do it with are the ones who, like, we know they're going to go, like, kick off their whole eval suite and be able to come back to us with, like, really actionable feedback.

33:01 as quickly as possible alongside the vibes, the vibes-based feedback. So, of course, there's definitely process-y things that we could do to make these things better over time. But, I mean, you'd be surprised. I mean, you probably wouldn't be surprised, but I think some people would be surprised about how few products that exist in the world that are agentic that don't have good evals to be able to help test these models. We had Boris on the pod two years ago saying we don't have the also clock code. I mean, that's changed. That has changed. That has changed. But, you know, like, yeah, a lot of things like, you know, your evals are your users and you don't have time to formalize them or you don't have the experience or whatever. And that's fine. At some point when you get serious, you'll have evals. Yeah, totally. I think the good thing is you guys do a lot of amp fooding, as you say. So it's commonly said that you find a lot of this testing it yourself before it comes out, right?

33:49 Yeah. Yeah. The APIs of customers in particular are very useful because they'll put Cloud into a different harness. And we want Cloud to be very, very general. So its ability to work in other harnesses is quite important to us. And so that style of feedback tends to be exceptionally useful for us because it's novel from what we would have gotten from AntFooting. What kind of feedback helps? Is it like regressions? We have regressions here. We notice good attributes here. The good stuff is always very validating. I think we're very frequently not surprised, but we like to look to be validated on some of those things that we might hypotheses. Honestly, the most useful feedback is negative. Like, you know, it didn't do well at this. It didn't do well at that. We love that feedback. And then we're like, great, how do we fix it?

34:31 Yeah, and it tends to be some of these things that are like, it's really hard because not everyone actually wants like Claude to land at the exact same spot on every spectrum. And maybe a good example is like, you know, laziness versus proactivity and taking action. And so there's sometimes when like we have a model and people are like, like, just finish my task. Like, stop being lazy. Like, do better. And other times when it's like, hey, we were like just kind of feeling this out and planning. And then you like went and wrote a bunch of code or you went and like took a whole bunch of action that I wasn't ready for. And getting that exactly right is literally impossible because everybody actually wants something different in reality. But I do think we've had situations where we've cut an early version of the model. We found that we saw too much laziness. Right.

35:15 And like we like put it back in the oven for a bit. Right. And tried to train that back out. And so those are the sorts of things that we may not have caught if all we're doing is internally antfooding within our products and within our API. It's way, way more important for us to get that out there in the API and get all of our other customers running with it and let us know what we need to change. Yeah, it's a core question. I think it's probably the, if I were in your shoes, like one of the hardest things about PMing for a model, which is, let's call it alignment versus personalization. There's a default personality, default behavior that you want for all consumers. And then you want to be able to respond very quickly to someone's CLAUDE.md saying, no, no, no, like, please be more proactive than default.

36:01 And then, like, to my knowledge, you know, and I was involved in Frontier Code. We were involved in the Fable launch, which put Frontier Code on the map. And now Opus 5.5, we have it back on the model benchmark. That is a very big part of Frontier Code, right? The reason that the model curve looks weird for Frontier Code is because it penalizes too much activity or too much files, which is good. Like, there was a whole... design purpose of Frontier Code. But I could easily just be like, in my CLAUDE.md, just say, don't do that. And then it won't do it. And so am I penalizing Cloud Opus 55 unfairly because my benchmark doesn't have that personalization element, which is subjective? So I don't know what the question is, but how do you deal with personalization? Clearly, a model can do so many

36:51 different things and it's so dependent, everyone has their own CloudMD, how do you evolve that? We don't do a great job of it. Seriously, it's really hard. It's super hard and they're competing and contrasting. I thought there's the cloud constitution, I think. That's old. There is a constitution, but I'll let you guys answer. The constitution's helpful, but for things like this on how it personalizes to instructions which are different and preferences which are very, very different across these things is very hard for us to do. We generally try to solve this by having more parameters that then allow us to tweak it

37:32 per customer base. And if we don't control your customer base and you're an API customer, then we try to give you recommendations and then you try to do that tweaking. So there are things like, you know, around, it's not exactly the same level like personalization to the various cloud.mds or now agents.mds or whatever. but we do try to give things around the layer of like level of effort, like how much effort you should put in, like what are the level, like maybe the size of the model that you might actually try to care about, things like that. But I think it's not quite there for what we would actually need for like that level of personalization. So it's a very hard problem. And right now a lot of our way to solve that is beyond the parameter, like the giving multiple parameters to something, is try to exhibit conflicting taste, honestly,

38:15 across like all of the feedback that we get and try to make a judgment call on some of these things. I think over time, they actually, and as you scale out more customers and more users, that's not a very scalable way to go do something. And we will need to develop ways for Claude to kind of take in your personalization. And maybe that's a signal you can provide or something you could tune on or things like that. But I think we'll have to take a route that's much more oriented towards that direction. But for now, yeah, we actually just try to make judgment calls on a lot of the stuff, but it's pretty tough. It is what it is. A small note there. I guess the default effort reasoning changed. It's now medium. This is one of those things that probably affects the average customer that doesn't use this toggle. Yeah, it's like a Claude Code setting, right? It's a Claude Code setting that defaulted to medium.

39:00 Yeah, they just... And this thing varies. This thing does... This is a perfect example of a parameter that we use to kind of personalize. I would say this may be a little too aggressive of a term, but we kind of customize maybe for different subsegments. And so, like, the default setting will adjust for whether you're a free customer or a prosumer customer, or Claude Code versus Cowork versus Enterprise, like things like that, versus API and stuff like that. Knowing that you can obviously always adjust it, but those are kind of like light ways where we try to give that. One thing that we are hesitant to do is we don't want people to feel like they can't control their customization of Cloud. So we could take a heavier hand on these things, for example, But we want to give users the ability to experience the intelligence levels of Cloud as they kind of see fit, especially if you want to go towards, like, the higher end or you want to experiment with maybe sometimes in the lower end if that actually is, like, faster for your use case, which is a hard product tradeoff, I think.

39:53 And so we tend to lean towards gentle defaults but give you options to adjust, which I think works okay-ish for now. But I imagine us, you know, like, learning how to do that better over time for customers. We're not like super, super happy with any of this yet, but we know as we launch a model, like you asked, like, what does it take to launch a model? One of the big things obviously we have to do is like, okay, Claude like got good at a thing. And like, sometimes that thing is something we should expose as a parameter on the API to let you control. But I think as this has evolved over time, we've ended up with like this funny mishmash of like, there's effort, there's budgets, there's all these things that you can set

40:31 on the API. And it's like, how could anyone like possibly figure out what is the right combination of things for any given tasks for what you should go and set? And it's something that this is us just very much like figuring it out as we go and figuring it out with customers and figuring out like which of those parameters do we deprecate over time? Which of those parameters do we just like default better? Which of those parameters should be like a one to a hundred scale versus a low, medium, high scale, right? Like, because actually we've trained Claude to get really good at understanding like a whole spectrum of options on some of those sorts of things. But like, do we expose the like entirety of those millions and billions of permutations of options? Like, you wouldn't really get very far. And that's one of the things actually that is part of the value prop with something like Claude Managed Agent, which is like still programmatic access to Claude in an API.

41:19 but you get a harness out of the box where we do, every time we're launching the model, a lot of the work to go figure out, like, what are the new parameters? What are the breaking changes? How do we get the best out of Claude within our harness? So that for the people who are using it, you just kind of get that out of the box. You don't have to think about it. Just for people who are drawn to that, and which there's a significant and growing population, particularly we actually did a whole CMA workshop at AIE, which I thought was really well done. Is there a separate feed for people to subscribe to, like, updates on CMA? Feed, like... Like a Twitter feed, newsletter, I don't know. No, I think actually our Cloud Devs channel, we are, yeah, we're pretty loud about it because it is, like, an interesting option for people.

42:02 Yeah, I just want to give people, like, if that's your way of consuming Cloud, this is the path. I think it's also pretty underrated. For those that aren't consuming Cloud this way, what is Claude Managed Agents? Why should people use it? Oh, yeah. We should probably do a little intro. I don't know. That's somebody's house. Yeah. A little two-sentence thing. I think a lot of people glanced over it when it came out, right? Everyone was in this phase of, I need to do harness engineering. I need to control this. And you hear all that. And now it's like, do you need to do all that? Or do you just use managed agents and you have a lot of info you can skip over setting up? But give the quick three. Yeah. It depends on your use case. So Claude Managed Agents is our API today.

42:47 We have the Messages API, which is essentially raw token access. Then we have Claude Managed Agents, which is a suite of APIs, but it lets you define an agent, start a session with that agent. And it's essentially just kind of a higher order abstraction that lets you skip setting up all the infrastructure, setting up your harness, and doing a lot of that type of work. It's a much more out-of-the-box integration for people who want to programmatically run agents. Does CMA share infrastructure with CloudTag? Because the sequencing of it seemed very like, oh, like that's the cloud tag. A lot of it. A lot of it. So one of the big things that we do is as the platform team, we run our APIs. We also kind of run all of our foundational product infrastructure and product services.

43:32 So a really good example is like our team runs our sandboxing fleet that backs both Claude Managed Agents as well as products like Cloud Tag and otherwise. But at the actual harness layer for those two products, there's just differences in what we might want to do within the harness so that they don't actually share a harness, but they share a lot of the infrastructure that powers the execution and powers the APIs. I think that's really... People should use this more. I think everyone basically had copycat clones of Managed Agents after you guys launched it, which imitation is a sincere form of flattery. Since we're on this topic, we'll briefly mention all the memory stuff that has been going on. I feel like it was a big part of Anthropic work in the first half of the year. Has it stalled out? Is it still ongoing? What's

44:20 the emphasis here now? Managed agents specifically. Memory, dreaming. Oh, it is a big part of what we're doing. I think maybe the way to think about it is we launched managed agents as a platform for people who didn't want to do all these things themselves, right? Or just wanted to like offload, like I'm getting the best out of Claude because our team is literally sitting here like evaling all these things and making sure that the harness is going to be really powerful. Memory and dreaming is all about just like how do you make sure your agent can continuously improve over time? And all of that is still an important focus for our team. But what I would actually say is to the point of what we talked about earlier, as launching models has become harder and harder from a safety perspective, I think most of our team's energy is going less

45:04 into like, here's a fun feature to like hill climb on this agent or whatever, and much more into how do we bring safety into all these things? Because the most important thing that we can do is get the power of the model out there and get it out there safely. And so I would say a lot of our energy is take all these same concepts and give you an agent system that's going to be really powerful, but also that's going to be really safe. And so that's where we're putting our energy. I think compared to maybe earlier this year, we have slowed down on like skills, dreaming, memory, like these kind of like moment in time injections. Please love skills. No, no, no. We love skills. We love skills. We are making updates. I'm not saying that we're not doing anything, but we love skills as well. But yes, I think we've slowed down a little bit comparatively for some of the other efforts.