🔶 将生产环境 AI Agent 迁移至 GPT 5.6 Sol

即日起，Ploy 的 agent 已运行在 GPT-5.6 Sol 上，那是 OpenAI 今早发布的旗舰级模型。数月以来，由于我们对质量的要求极高，一直找不到能挑战 Claude Opus 的模型。GPT 5.6 Sol 改变了这一局面。在直接与 Claude Opus 对比后，我们已将 GPT 5.6 Sol 设为驱动每个 Ploy 工作区的默认模型。

这次切换比听上去更重要。Ploy 的 agent 负责构建和编辑真实的营销网站：它要规划页面、阅读代码库、编写组件、生成图像、对自己的工作进行截图，并判断何时完成。这份工作为模型设定了极高的门槛，我们会用每一个前沿版本进行测试。在 Opus 占据默认位置的这四个月里（先是 Opus 4.7，后是 4.8），测试过的模型无一胜出。GPT-5.6 是第一个做到这一点的。

并非首次评估就完美。它有真实的失败模式，我们会展示。但总体表现极为出色，并且预示着一个直接而具体的前景：构建完成时间不到原来一半，成本降低 27%，已完成工作的得分不低于甚至超过现有模型。这样的数字足以让一个模型启动真正的迁移工程。

尽管使用了 Vercel 的 AI SDK 这一通用 LLM 开发套件，从 Claude Opus 4.8 切换到 GPT 5.6 Sol 仍然需要逐项分析评估失败，发现那些我们以为属于“模型”的行为其实是供应商特定行为，我们的整个技术栈已悄然围绕它们进行了专项优化：如何填充工具参数、提示缓存如何运作、回合间如何回放自身推理。

以下是所做的关键工作：修复评估框架，接着修复工具 schema，再修复缓存，最后修复推理回放。

第 0 步：在相信任何一个数字之前，先修复评估框架

我们的评估套件会在真实夹具工作区中运行真实的 agent。数百个用例，从“从零构建主页”到“判定此次克隆请求是否安全”。构建类用例由视觉评判器依据参考设计进行二元检查评分，包括 10 个是/否问题（如_“主视觉是全出血摄影场景”或“主要 CTA 是圆角矩形，而非胶囊形状”_），外加内容检查、工具轨迹检查和文件断言。每个失败的用例都会结合完整轨迹进行分诊：实际的工具调用和模型文本，而不仅仅是分数。

在两套模型家族上运行该套件，比任何单独结果都更让我们惊讶：

你的评估框架是围绕当前模型调优的，而你并不知晓。 我们的工具调用预算按 Opus 的顺序执行风格来设定；GPT-5.6 会扇出并行调用，在正确解决用例的同时耗尽了预算。我们的评估执行器不支持批量文件读取，而 Opus 很少使用它，GPT-5.6 却在持续使用。初次跨模型运行时，约三分之一的原始失败可追溯到框架的假设，而非模型行为，并且这些失败在两个模型间并非平均分布。如果你在评估挑战者模型，在信任通过率之前，先对轨迹进行分诊。否则，你是在用新模型模仿旧模型的程度来给它打分。

确保在评估中公平地给模型打分。 一个省略了 minScore 阈值的数据集静默继承了默认值 1.0，因此 GPT-5.6 对一个它得了 0.98 分的主视觉被判为“失败”，而 Opus 在一个通过了所有单项检查的用例中也被判为“失败”。两个设计方向都有道理；而一个隐性的阈值却在暗中作祟。

框架清理完毕后，下面是我们重新设计套件中的一个样本，agent 按照参考设计重建品牌主页：

每次完成构建的平均值	Claude Opus 4.8 (n=11)	GPT-5.6 (n=10)
成本	$3.06	$2.22
墙钟时间	8m 00s	3m 42s
输入 token	2.60M	1.70M
输出 token	33.0K	17.1K
视觉评分	0.936	0.970

这就是前景的轮廓：生成完成页面快 2.2 倍，成本低 27%，输出 token 大约只有一半。GPT-5.6 编写精简的代码。在一组配对示例中，Opus 生成了一个 17,957 字符的 globals.css，包含 174 个 CSS 变量（全色彩渐变，多数未使用），而 GPT-5.6 写了一个 2,508 字符、45 个变量的版本，渲染的页面相当，有时甚至更好。

设计：锐利、干净，但稍显同质

我们对 GPT-5.6 设计工作的总体判断是：_非常_擅长干净、现代、严格网格化的布局，但除非加以良好引导，它倾向于朝那种视觉风格收敛。在我们为 Opus 4.8 设计的旧框架下，GPT 5.6 Sol 倾向于忽略既有的设计系统，转而输出锐利、克制但明显通用的结果。

我们解决这一问题的细节值得另写一篇博文。凭借设计与工程团队的专业知识，我们能够引导模型实现世界级的品牌一致性，这是开箱即用所无法达到的。

第 1 步：设计一个让 GPT-5.6 无法犯规的工具 schema

这个问题在我们发现之前一直在暗中扭曲结果。

我们 agent 的 code 工具有 25 个顶层参数，一个必填（action），其余可选。Claude 只发送它实际使用两三个参数，省略其余。GPT-5.6 会每次都发送全部 25 个参数，为不需要的参数编造看似合理的值：offset: 0、timeout: 120000、siteId: "00000000-0000-0000-0000-000000000000"。

三天生产轨迹中，携带所有属性的 code(read) 调用：

模型	调用次数	携带全部 25 个属性
gpt-5.6	6,635	6,635 (100%)
claude-opus-4.8	2,898	4 (0.1%)
claude-sonnet-5	1,933	0

问题不在于啰嗦。而在于编造的值无法与真实意图值区分。offset: 0 看起来就像一个真实的参数。我们的文件读取实现把它当作真实参数处理，导致 GPT-5.6 52% 到 64% 的文件读取返回空内容。工具两种情况下都返回 success: true，模型无从知晓自己读的是空文件。它只会用更多调用来把工作做得更差。

提示工程无法修复。工具描述中指令“省略未使用的参数”：依旧 25/25。每个属性标注“OPTIONAL, omit if unused”：依旧 25/25。OpenAI 的 strict 模式：行为相同（我们测量过），且采用它会迫使我们从所有 schema 中移除 pattern、format 和数组边界验证。这已内化为模型生成函数调用的方式。无法通过指令消除，只能靠设计来规避。

真正奏效的修复是在供应商边界进行 schema 变换。仅针对 OpenAI 系列模型，我们将每个可选属性重写为必填但可为 null，使用 anyOf: [T, null]，这为模型提供了一种显式表达“未使用”的方式。然后，在每次工具调用都会经过的唯一切入点，我们再在验证之前将 null 值剥离，因此所有工具实现无需任何改动。完整回路：模型看到的是一个可以诚实表达的 schema；工具看到的仍是与之前一致的输入。

// 之前：25 个键，每个都携带一个编造的值 { "action": "read", "file_paths": [...], "offset": 0, "timeout": 120000, ... }

// 之后：25 个键，4 个真实值，21 个显式 null（在工具运行前被剥离） { "action": "read", "file_paths": [...], "offset": null, "timeout": null, ... }

结果：空文件读取从 52% 降至 0%，相同工作所需的工具调用减少了约 30%，因为不再需要反复重新读取返回空白文件的文件。

第 2 步：重建提示缓存

这是最具启发性的工程差异，因为表面上看两个供应商都提供“提示缓存”，但相同的词汇背后是两种完全不同的设计。如果要谨慎迁移一处，请选这里：在我们修复之前，GPT-5.6 成本看起来比 Opus 高约 50%。不是模型定价的问题，而是缓存配置的问题。

我们 agent 的提示以约 29K token 的静态前缀开头（工具 schema 加上核心系统提示），每个对话都完全相同。在 Claude 上，我们用 cache_control 标记缓存断点，这个前缀可以跨整个组织缓存：任何对话、任何工作区，共享一个条目，无需考虑吞吐预算。缓存命中率在 92% 到 96%，缓存退居幕后。

GPT-5.6 改变了我们依赖的 OpenAI 缓存模型。更早的 GPT 模型会对部分前缀匹配进行隐式缓存，免费获得不错的命中率。GPT-5.6 取消了部分前缀匹配：隐式缓存现在仅会基于最后一条消息创建完整提示条目。一个_新的_对话即使共享我们的 29K 静态前缀，缓存命中率也是0%。每次对话都会按未缓存费率重新计费整个前缀，而且在 GPT-5.6 上，每条未缓存的提示还要额外支付 1.25 倍的缓存_写入_附加费，无论你是否使用缓存。

预期的方案是显式的：prompt_cache_breakpoint 标记加上强制的 prompt_cache_key。而 key 正是设计真正分化的地方，因为它属于缓存身份的一部分。相同提示，不同 key：零缓存命中。每个 key 映射到一个缓存节点，该节点在 OpenAI 将流量扇出到拥有独立、冷缓存的其他节点之前，大约能支持每分钟 15 次请求。

于是“启用缓存”变成了一个真正的设计决策：你该将 key 的作用域限定在哪个实体？

• 按对话 key 意味着新对话永远无法命中共享前缀。首次调用命中率：0%。（我们测量过这个错误，代价高昂。）
• 一个全局 key 意味着每个请求都哈希到同一个缓存节点，生产流量会瞬间压垮 15 rpm 的预算；请求溢出到冷节点，回到到处缺失的状态。
• 按工作区 key 是最佳平衡点。客户工作区内所有对话共享条目；每个 key 的流量保持较低。

我们交付的是工作区作用域的 key，并将系统提示拆分为带有断点的多层结构，镜像了我们已为 Anthropic 使用的结构：

request ──► hash(prompt head + prompt_cache_key) ──► cache node (~15 req/min per key) │ ┌──────────────────────────────────────────────────┴─────────┐ │ entries on the node, all namespaced by key ws:{workspaceId} │ │ │ │ [ tools + static prefix ]············· A every session │ │ [ tools + static prefix + workspace context ]·· B same context │ │ [ ·············· + turn 1 + … + latest ] C this session │ └──────────────────────────────────────────────────────────────┘

条目 A 是让会话的_首次_调用变便宜的关键。条目 B 具有自愈能力：当工作区记忆改变时，请求会错过 B 但仍能命中 A，然后写一个新的 B。仅需一次上下文大小的写入，而不是全额 29K 重新计费。条目 C 是 OpenAI 的隐式完整提示链，在我们的会话中运行良好，因为提示是严格追加的。

有一个后果无法绕过：在 OpenAI 上，静态前缀的跨工作区共享在结构上是不可能的。 Anthropic 可以共享，因为它的缓存是组织作用域且无 key 分区。在 GPT-5.6 上，每个工作区在每个空闲窗口都要支付一次约 29K 的冷写入费用，约 $0.18。这是真实成本，但有界且可预测。

调整后的结果：首次调用缓存命中率从约 0% 提升至83.7%，总未缓存输入 token 下降 28%，GPT-5.6 的每套件成本落到了 Opus 之下。我们此前盯着的每一块钱差距，都是缓存配置错误，而非模型定价。如果你在对比模型成本而其中一个模型缓存是冷的，你比较的是自己的配置，而不是模型。

第 3 步：让推理回放自包含

这一步更简短，但它破坏了真实对话。GPT-5.6 的 Responses API 默认将前一轮推理作为服务端条目引用进行回放；我们的会话开始在对话中途间歇性失败，报错 Item 'rs_...' not found。修复方式是 store: false，这会让 SDK 请求加密的推理内容，并回放自包含的 blob，而非指向服务器状态的指针。一个让我们耗费了整个下午调试的推论是：当服务端推理状态参与循环时，即使你发送的字节是追加的，有效提示也可能在你上游发生变化。

GPT 5.6 Sol 已准备好投入 Ploy

自己来试试。 GPT-5.6 今日上线，已经运行在 Ploy 上。你现在就可以免费试用：给它一个网站去构建，看看不到四分钟的构建是什么样。在 ploy.ai 免费开始。

---

Ploy 是营销自动化平台。

As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, we couldn’t find a model that challenges Claude Opus given our incredibly high bar for quality. That changed with GPT 5.6 Sol. After running it head-to-head against Claude Opus, we’ve made GPT 5.6 Sol the default model powering every Ploy workspace.

That’s a bigger switch than it sounds. Ploy’s agent builds and edits real marketing websites. It plans a page, reads the codebase, writes components, generates imagery, screenshots its own work, and decides when it’s done. That job description sets a very high bar for a model, and we test every frontier release against it. For the four months Opus held the default slot (first Opus 4.7, then 4.8), nothing we tested beat it. GPT-5.6 is the first model that did.

Not that the first eval run was perfect. It had real failure modes, which we’ll show you. But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. Numbers like that buy a model a real migration effort.

Despite using Vercel’s AI SDK, a universal LLM SDK, switching from Claude Opus 4.8 to GPT 5.6 Sol required discovering, one eval failure at a time, that the things we think of as “the model” are provider-specific behaviors our whole stack has quietly specialized around: how it fills in tool arguments, how its prompt cache works, how it replays its own reasoning between turns.

Here’s what it took: fix the eval harness, then the tool schemas, then caching, then reasoning replay.

Step 0: Fix your harness before you trust a single number

Our eval suite runs the real agent against real fixture workspaces. Hundreds of cases, from “build a homepage from scratch” to “is this clone request safe to execute.” Build cases are scored by a visual judge running binary checks against a reference design, ten yes/no questions like “the hero is a full-bleed photographic scene” or “primary CTAs are rounded rectangles, not pills” , plus content checks, tool-trajectory checks, and file assertions. Every failed case gets triaged against its full trace: the actual tool calls and model text, not just the score.

Running that suite across two model families surprised us more than any individual result:

Your harness is tuned to your incumbent model, and you don’t know it. Our tool-call budgets were sized for Opus’s sequential style; GPT-5.6 fans out parallel calls and blew through them on cases it was solving correctly. Our eval executor didn’t support batched file reads, which Opus rarely used and GPT-5.6 uses constantly. Roughly a third of the raw failures in the first cross-model run traced back to harness assumptions, not model behavior, and they were not evenly distributed between the models. If you’re evaluating a challenger model against an incumbent, triage the traces before you trust the pass rate. Otherwise you’re grading the new model on how well it imitates the old one.

Make sure you’re grading models fairly in evals. A dataset that omitted its minScore threshold silently inherited a default of 1.0, so GPT-5.6 “failed” a hero it scored 0.98 on, and Opus “failed” a case while passing every individual check. Two defensible design directions; one invisible threshold.

With the harness cleaned up, here’s a sample from our redesign suite, where the agent rebuilds a brand’s homepage against a reference design:

Mean per completed build	Claude Opus 4.8 (n=11)	GPT-5.6 (n=10)
Cost	$3.06	$2.22
Wall-clock time	8m 00s	3m 42s
Input tokens	2.60M	1.70M
Output tokens	33.0K	17.1K
Visual score	0.936	0.970

This is the shape of the promise: 2.2× faster to a finished page, 27% cheaper, and about half the output tokens. GPT-5.6 writes lean code. On one matched pair, Opus produced a 17,957-character globals.css with 174 CSS variables (full color ramps, mostly unused) where GPT-5.6 wrote 2,508 characters and 45 variables for a comparable (and sometimes better) rendered page.

Design: sharp, clean, but a little bit uniform

Our overall read on GPT-5.6’s design work: it is very good at clean, modern, tightly-gridded layouts, but it tends to converge towards that look unless you steer it well. With our older harness designed for Opus 4.8, GPT 5.6 Sol tends to ignore existing design systems and instead produces sharp, restrained, and visibly generic output.

The details of how we fixed this are worth a separate blog post of its own. With the expertise of our design and engineering teams, we are able to steer models to achieve world-class brand adherence that you can’t get out of the box.

Here’s the one that was silently corrupting results before we caught it.

Our agent’s code tool has 25 top-level parameters, one required (action) and the rest optional. Claude sends the two or three it’s using and omits the rest. GPT-5.6 sends all 25, every time , inventing plausible values for the ones it doesn’t need: offset: 0, timeout: 120000, siteId: "00000000-0000-0000-0000-000000000000".

Three days of production traces, code(read) calls carrying every property:

Model	Calls	Carrying all 25 properties
gpt-5.6	6,635	6,635 (100%)
claude-opus-4.8	2,898	4 (0.1%)
claude-sonnet-5	1,933	0

The problem isn’t verbosity. It’s that an invented value is indistinguishable from an intended one. offset: 0 looks like a real argument. Our file-read implementation treated it as one, and 52% to 64% of GPT-5.6’s file reads were coming back empty because of it. The tool returned success: true both ways, so the model had no way to know it was reading blank files. It just did the work worse, with more calls.

Prompting doesn’t fix this. A tool-description directive to “omit unused parameters”: still 25/25. Per-property “OPTIONAL, omit if unused” hints: still 25/25. OpenAI’s strict mode: identical behavior (we measured), and adopting it would have forced us to strip pattern, format, and array-bound validation from every schema. This is baked into how the model emits function calls. You don’t instruct it away; you design around it.

The fix that worked is a schema transform at the provider boundary. For OpenAI-family models only, we rewrite every optional property to be required but nullable , using anyOf: [T, null], which gives the model an explicit way to say “not using this.” Then, at the single seam every tool invocation passes through, we strip the nulls back out before validation, so no tool implementation changes at all. Round trip: the model sees a schema where honesty is expressible; the tools see the same inputs they always did.

// Before: 25 keys, every one carrying an invented value
{ "action": "read", "file_paths": [...], "offset": 0, "timeout": 120000, ... }

// After: 25 keys, 4 real values, 21 explicit nulls (stripped before the tool runs)
{ "action": "read", "file_paths": [...], "offset": null, "timeout": null, ... }

Results: empty file reads went from 52% to 0%, and the agent needed roughly 30% fewer tool calls for the same work, because it was no longer re-reading files that came back blank.

Step 2: Rebuild prompt caching

This was the most instructive engineering difference, because on the surface both providers offer “prompt caching” and the words hide two entirely different designs. If you migrate one thing carefully, make it this: before we did, GPT-5.6 looked about 50% more expensive than Opus. It wasn’t the model’s pricing; it was our cache configuration.

Our agent’s prompt opens with a static prefix of roughly 29K tokens (tool schemas plus the core system prompt) that’s identical for every conversation. On Claude, we mark cache breakpoints with cache_control and that prefix caches across the whole organization : any conversation, any workspace, one shared entry, no throughput budget to think about. Cache hit rates run 92% to 96% and caching fades into the background.

GPT-5.6 changed OpenAI’s caching model out from under us. Earlier GPT models cached implicitly on partial prefix matches, which gave decent hit rates for free. GPT-5.6 dropped partial-prefix matching: implicit caching now only creates whole-prompt entries keyed on the latest message. A new conversation sharing our 29K static prefix cached 0% of it. Every conversation re-billed the full prefix at the uncached rate, and on GPT-5.6 every uncached prompt also pays a 1.25× cache-write surcharge, whether or not you use caching.

The intended mechanism is explicit: prompt_cache_breakpoint markers plus a mandatory prompt_cache_key. And the key is where the design really diverges, because it’s part of cache identity. Identical prompt, different key: zero cache hits. Each key maps to a cache node that sustains roughly 15 requests per minute before OpenAI fans traffic to other nodes with independent, cold caches.

That turns “enable caching” into an actual design decision: what entity do you scope the key to?

• Per-conversation key means a new conversation never hits the shared prefix. First-call hit rate: 0%. (We measured this mistake. It’s expensive.)
• One global key means every request hashes to one cache node, and production traffic obliterates the 15 rpm budget; requests spill to cold nodes and you’re back to misses.
• Per-workspace key is the sweet spot. All conversations in a customer workspace share entries; per-key traffic stays low.

We ship the workspace-scoped key and split the system prompt into breakpointed layers, mirroring the structure we already used for Anthropic:

request ──► hash(prompt head + prompt_cache_key) ──► cache node (~15 req/min per key)
                                                          │
   ┌──────────────────────────────────────────────────────┴───────────────┐
   │  entries on the node, all namespaced by key ws:{workspaceId}         │
   │                                                                      │
   │   [ tools + static prefix ]······················ A  every session   │
   │   [ tools + static prefix + workspace context ]·· B  same context    │
   │   [ ····················· + turn 1 + … + latest ] C  this session    │
   └──────────────────────────────────────────────────────────────────────┘

Entry A is what makes a session’s first call cheap. Entry B self-heals: when workspace memory changes, the request misses B but still hits A, then writes a fresh B. One context-sized write instead of a full 29K re-bill. Entry C is OpenAI’s implicit whole-prompt chain, which works fine within a session because our prompts are strictly append-only.

One consequence has no workaround: cross-workspace sharing of the static prefix is structurally impossible on OpenAI. Anthropic can share it because its cache is org-scoped without key partitioning. On GPT-5.6, every workspace pays one 29K cold write per idle window, about $0.18. A real cost, but bounded and predictable.

Results after the change: first-call cache hits went from roughly 0% to 83.7% , total uncached input tokens dropped 28%, and GPT-5.6’s per-suite cost landed below Opus’s. Every dollar of the gap we’d been staring at was cache misconfiguration, not model pricing. If you’re cost-comparing models and one of them has a cold cache, you are comparing your config, not the models.

Step 3: Make reasoning replay self-contained

Shorter, but it broke real conversations. GPT-5.6’s Responses API replays prior-turn reasoning as server-side item references by default; ours started intermittently failing mid-conversation with Item 'rs_...' not found. The fix is store: false, which makes the SDK request encrypted reasoning content and replay self-contained blobs instead of pointers to server state. A corollary that cost us a debugging afternoon: with server-side reasoning state in the loop, the effective prompt can change upstream of you even when the bytes you send are append-only.

GPT 5.6 Sol is ready to Ploy

Try it yourself. GPT-5.6 launched today, and it’s already live on Ploy. You can try it for free right now: give it a website to build and see what a sub-four-minute build looks like. Start free at ploy.ai.

---

Ploy is marketing run on autopilot: an AI layer that plans, builds, publishes, and optimizes your website and campaigns end-to-end. If debugging cache-node fan-out at 2am sounds like fun,we’re hiring.