🔶 GPT-5.6、Grok 4.5、Claude 与 Muse Spark 构建相同 4 款应用对比评测
英文标题:🔶 GPT-5.6, Grok 4.5, Claude, and Muse Spark build the same 4 apps
本文测评了包括 GPT-5.6(Sol/Terra/Luna)、Grok 4.5、Claude Opus 4.8 与 Fable 5、Meta 的 Muse Spark 1.1 以及多个开放权重模型在内的 12 个模型,在 Doom 风格光线投射迷宫、3D 魔方、计算器和康威生命游戏四个任务上各进行五次尝试。结果显示 GPT-5.6 Sol 在光线投射中最稳定且出色,Claude Fable 5 在魔方与计算器任务上表现优异,Grok 4.5 以高性价比成为实用之选,开源模型虽在简单任务上性价比高但复杂任务仍不稳定。所有尝试链接均已公开,为开发者选型提供了可复现的工程参考。
1020 B
33.9 KB
26.6 KB
43.6 KB
59.7 KB
49.7 KB
39.8 KB
47.7 KB
79.4 KB
58.4 KB
52.6 KB
46.7 KB
50.3 KB
42.6 KB
18.2 KB
45.7 KB
36.7 KB
22.3 KB
25.6 KB
46.0 KB
中文
我们上次的构建对决登上了 Hacker News 首页,评论区毫不留情。没关系,很多反馈都很有价值。因此,我们采纳了这些意见,并趁着 GPT-5.6 以三种层级(Sol、Terra、Luna)登场,以及 Meta 意外发布编码模型(Muse Spark 1.1)之际,重新进行了一次更大规模的测试:十二个模型、四款应用、每个任务五次尝试。
基于反馈我们做出的改变:
- 你希望加入开放权重模型。 因此我们添加了 GLM-5.2、Qwen 3.7 Plus、DeepSeek V4 Pro 和 Kimi K2.6 作为对比,所有模型均通过 Fireworks 提供服务。
- 你说一次尝试不够靠谱。 同意。现在每个模型在每项任务上都有五次尝试。上方展示每个模型的其中一次运行结果;每个任务表格会说明我们认为五次中实际成功了几个(以及如何判定),并链接我们最喜欢的尝试;底部则附有所有尝试的链接,让你可以观察这些模型逐次运行的波动有多大。
- “这不够客观。” 没错,我们从未假装客观。我们不提供科学定论。我们生成了一大堆产物,并全部公开,你可以形成自己的判断。以下内容仅是我们观察结果后得出的看法。
想直接查看原始构建?跳转到所有尝试并亲自运行。
十二强阵容:全新 GPT-5.6 Sol、GPT-5.6 Terra 和 GPT-5.6 Luna;Meta 的 Muse Spark 1.1;Grok 4.5、GPT-5.5、Claude Opus 4.8 与 Claude Fable 5;再加上开放权重阵营:Qwen 3.7 Plus、DeepSeek V4 Pro、Kimi K2.6 和 GLM-5.2。
每个任务展示如下:我们挑选的五次尝试中最佳的一次在上方,五次的总成本和平均时间紧随其后,文末附有所有原始尝试的链接,让你自行评判。
任务 1:Doom 风格光线投射迷宫
第一人称光线投射,用 WASD 行走,带深度的着色墙壁、地板和天花板,有碰撞。
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
我们如何判定“可玩”:唯一的问题是能否真正在迷宫中行走、移动和转身。如果可以,就算成功。
| 模型 | 可玩次数↕ | 最佳构建 | 成本(5 次)↕ | 平均时间↕ | 我们评价 |
|---|---|---|---|---|---|
| Grok 4.5 | 5/5 | #4 | $0.27 | 62s | 性价比高。 |
| Claude Opus 4.8 | 4/5 | #2 | $0.54 | 48s | 稳定但平淡。 |
| GPT-5.5 | 4/5 | #1 | $1.44 | 138s | GPT-5.6 之前整体最佳。 |
| Qwen 3.7 Plus | 2/5 | #4 | $0.13 | 43s | — |
| Kimi K2.6 | 2/5 | #2 | $1.37 | 88s | — |
| Claude Fable 5 | 3/5 | #1 | $2.35 | 107s | 结果不错,但一致性稍差。 |
| DeepSeek V4 Pro | 3/5 | #4 | $0.30 | 318s | — |
| GLM-5.2 | 0/5 | — | $0.12 | 133s | 渲染细节很好,但角色在所有尝试中都无法移动。 |
| GPT-5.6 Sol | 5/5 | #5 | $1.35 | 120s | 最佳结果:稳定,优于 GPT-5.5,游戏整体细节更丰富。 |
| GPT-5.6 Terra | 3/5 | #1 | $0.44 | 39s | 细节好,但不稳定,我有时无法行走。 |
| GPT-5.6 Luna | 5/5 | #5 | $0.15 | 23s | 结果很棒,不过我认为仍不如 GPT-5.5。 |
| Muse Spark 1.1 | 2/5 | #1 | $0.55 | 169s | 运行成功时令人惊喜。五次中三次失败,但成功的可与 Fable 和 Sol 媲美,优于 Grok 与 Opus。 |
总体而言,Claude 在这里的表现低于预期。GPT 超越了所有其他模型,Grok 在其价位上是真正可用的替代方案,而 Muse Spark 在运行成功的几次中是一个真正的惊喜。
任务 2:3D 魔方(打乱 + 复原)
构建一个色彩丰富、具有 3D 外观的魔方,包含“打乱”和“复原”按钮,并以可视动画方式展示旋转。
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
我们如何判定“干净复原”:我们打乱并复原魔方,仅当两个动画都流畅播放、无卡顿、无颜色改变时才算成功。
| 模型 | 干净复原↕ | 最佳构建 | 成本(5 次)↕ | 平均时间↕ | 我们评价 |
|---|---|---|---|---|---|
| Grok 4.5 | 3/5 | #4 | $0.65 | 191s | 良好的简单结果。 |
| Claude Opus 4.8 | 0/5 | #1 | $0.56 | 44s | 没有完美的例子,每个都有些小问题,魔方颜色会改变,不过有些很接近。 |
| GPT-5.5 | 4/5 | #4 | $1.36 | 136s | 结果不错,有小瑕疵,如颜色闪烁或旋转动画不够流畅。 |
| Qwen 3.7 Plus | 1/5 | #5 | $0.07 | 24s | 运行成功时效果很棒! |
| Kimi K2.6 | 1/5 | #4 | $1.05 | 59s | 不太好。 |
| Claude Fable 5 | 5/5 | #2 | $2.03 | 92s | 无可挑剔,非常漂亮的结果。 |
| DeepSeek V4 Pro | 1/5 | #3 | $0.35 | 380s | — |
| GLM-5.2 | 0/5 | — | $0.08 | 89s | 令人惊讶,没有成功的结果。 |
| GPT-5.6 Sol | 4/5 | #5 | $1.06 | 72s | 结果很棒。一次动画奇怪,一次渲染成全黑魔方(有意思的选择),但成功的那几次表现出色。 |
| GPT-5.6 Terra | 4/5 | #4 | $0.42 | 38s | 还行,打乱动画有些奇怪,感觉比 GPT-5.5 略有进步。 |
| GPT-5.6 Luna | 0/5 | — | $0.14 | 24s | 通常初始渲染正确,但一打乱就出错。 |
| Muse Spark 1.1 | 2/5 | #1 | $0.54 | 182s | 比开源模型强,但就其价格而言我不会用它替代 Grok。 |
总体而言,GPT 在这里表现不佳让我们意外,因为它在光线投射中 3D 优势明显。Claude 再次表现出色,不过主要是 Fable 贡献了完美的五发五中,而 Opus 奇怪地一次完美复原都没做到。
在线体验任意模型的所有五次尝试:Grok · Opus · Qwen(将数字 1-5 替换为其他尝试)。
任务 3:计算器
数字、运算符、清除、等号,正确的运算符优先级,真实计算器外观。
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
我们如何判定“可用”:并非详尽测试,仅用 (((5 × 5) − 100) / 10) 等基本计算观察各模型如何处理运算顺序及结果渲染。
| 模型 | 可用↕ | 最佳构建 | 成本(5 次)↕ | 平均时间↕ | 我们评价 |
|---|---|---|---|---|---|
| Grok 4.5 | 5/5 | #5 | $0.37 | 110s | 简单一致的结果,没有额外装饰。 |
| Claude Opus 4.8 | 5/5 | #1 | $0.46 | 35s | — |
| GPT-5.5 | 4/5 | #1 | $0.91 | 75s | 有时会添加额外按钮,并尝试 3D 渲染,但常常被截断。 |
| Qwen 3.7 Plus | 4/5 | #4 | $0.04 | 12s | 可用且一致,不过有一个无法处理负数。 |
| Kimi K2.6 | 0/5 | #5 | $0.76 | 50s | 能运行,但无法处理负数。 |
| Claude Fable 5 | 5/5 | #1 | $1.22 | 48s | 纯粹从风格上,是我个人最爱。 |
| DeepSeek V4 Pro | 3/5 | #3 | $0.28 | 342s | 一个数字顺序错乱,一个结果未渲染。 |
| GLM-5.2 | 2/5 | #1 | $0.07 | 63s | 成功时结果很好。 |
| GPT-5.6 Sol | 5/5 | #1 | $0.84 | 61s | 与 GPT-5.5 一样试图过度装饰。这在光线投射中有帮助,但在这里很烦人,有些样式不佳,而非简洁一致。 |
| GPT-5.6 Terra | 4/5 | #1 | $0.28 | 20s | 成功时结果不错。 |
| GPT-5.6 Luna | 5/5 | #1 | $0.11 | 16s | 良好一致的结果,感觉与 Grok 类似。 |
| Muse Spark 1.1 | 5/5 | #5 | $0.33 | 94s | 总体不错,无明显问题,与 Grok 4.5 相当。偶尔按钮顺序或位置奇怪,扣些风格分。 |
总体而言,这显然是 Claude 表现最好的项目:Opus 和 Fable 都五次全对,Fable 的风格是我们最喜欢的。GPT-5.6 Sol 试图在风格上做得过火,像 Fable 一样用 3D 渲染计算器,但样式不到位,导致整体体验更差。更简单的 GPT 模型似乎因开箱即用而表现更佳。
任务 4:康威生命游戏
网格画布,播放/暂停/步进/随机/清除,点击切换细胞,动画化世代。
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
我们没有对生命游戏进行单独的五次尝试评分对比,因此这里仅列出成本和时间以及下面的一般印象。
| 模型 | 成本(5 次)↕ | 平均时间↕ |
|---|---|---|
| Grok 4.5 | $0.14 | 38s |
| Claude Opus 4.8 | $0.48 | 35s |
| GPT-5.5 | $1.06 | 79s |
| Qwen 3.7 Plus | $0.04 | 11s |
| Kimi K2.6 | $0.93 | 53s |
| Claude Fable 5 | $1.27 | 48s |
| DeepSeek V4 Pro | $0.25 | 304s |
| GLM-5.2 | $0.10 | 121s |
| GPT-5.6 Sol | $0.99 | 62s |
| GPT-5.6 Terra | $0.36 | 25s |
| GPT-5.6 Luna | $0.13 | 18s |
| Muse Spark 1.1 | $0.32 | 98s |
Grok 4.5 在此表现不错,但更重要的结论是这项任务足够简单,开源模型完成得极好。可能开源社区有足够多的生命游戏示例代码,使得它们能以更低的成本做得更好。Qwen 3.7 Plus 和 GLM-5.2 显然是此类任务的优选,但我不会普遍依赖它们——其他任务表明它们在真正新颖或更复杂的工作上仍有困难。
收据:原始速度与成本(简短答案)
综合结果。
| 模型 | 中位延迟↕ | 吞吐量↕ | 每次回复成本↕ |
|---|---|---|---|
| GPT-5.6 Luna | 1.0s | 97 tok/s | $0.001 |
| GPT-5.6 Terra | 1.5s | 62 tok/s | $0.001 |
| GPT-5.6 Sol | 1.8s | 45 tok/s | $0.003 |
| Qwen 3.7 Plus | 2.1s | 204 tok/s | $0.001 |
| Claude Opus 4.8 | 2.5s | 44 tok/s | $0.004 |
| GPT-5.5 | 3.0s | 45 tok/s | $0.003 |
| Grok 4.5 | 3.0s | 112 tok/s | $0.003 |
| Muse Spark 1.1 | 3.1s | 125 tok/s | $0.002 |
| Kimi K2.6 | 4.5s | 83 tok/s | $0.01 |
| Claude Fable 5 | 6.6s | 30 tok/s | $0.01 |
| GLM-5.2 | 7 |
English / 原文
Our last build-off hit the Hacker News front page, and the comments did not hold back. Fair enough, a lot of it was good feedback. So we took it, and with GPT-5.6 landing in three tiers (Sol, Terra, Luna) and Meta surprise-dropping a coding model (Muse Spark 1.1), we ran the whole thing again, bigger: twelve models, four apps, five attempts each.
What we changed based on your feedback:
- You wanted open-weights models in the mix. So we added GLM-5.2, Qwen 3.7 Plus, DeepSeek V4 Pro, and Kimi K2.6 as comparison points, all served via Fireworks.
- One attempt was weird, you said. Agreed. Every model now gets five attempts per task. Up top you get one sample run per model; each task table then says how many of the five we thought actually succeeded (and how we counted) and links the attempt we liked best; and every attempt is linked at the bottom so you can see how much these models swing run to run.
- "This isn't objective." Correct, and we are not pretending it is. We are not handing down a scientific verdict. We generated a big pile of artifacts, we are publishing all of them, and you can form your own opinion. Everything below is just our observations from watching the results.
Want to skip straight to poking at the raw builds? Jump to every attempt and run them yourself.
The lineup, twelve strong: the new GPT-5.6 Sol, GPT-5.6 Terra, and GPT-5.6 Luna; Meta's Muse Spark 1.1; Grok 4.5, GPT-5.5, Claude Opus 4.8, and Claude Fable 5; plus the open-weights crew: Qwen 3.7 Plus, DeepSeek V4 Pro, Kimi K2.6, and GLM-5.2.
Here is each task: our pick of the five attempts playing up top, the cost and time for all five just below, and links to every raw attempt at the bottom of the post so you can judge for yourself.
Task 1: Doom-style raycaster maze
First-person raycaster you walk with WASD, shaded walls with depth, floor and ceiling, collision.
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
How we counted "playable": the only question we cared about was whether you could actually walk through the labyrinth, move and turn. If yes, it counted.
| Model | Playable↕ | Best build | Cost (5 runs)↕ | Avg time↕ | Our take |
|---|---|---|---|---|---|
| Grok 4.5 | 5/5 | #4 | $0.27 | 62s | Great for the price. |
| Claude Opus 4.8 | 4/5 | #2 | $0.54 | 48s | Consistent but dry. |
| GPT-5.5 | 4/5 | #1 | $1.44 | 138s | Best overall before GPT-5.6. |
| Qwen 3.7 Plus | 2/5 | #4 | $0.13 | 43s | — |
| Kimi K2.6 | 2/5 | #2 | $1.37 | 88s | — |
| Claude Fable 5 | 3/5 | #1 | $2.35 | 107s | Good results, but less consistent. |
| DeepSeek V4 Pro | 3/5 | #4 | $0.30 | 318s | — |
| GLM-5.2 | 0/5 | — | $0.12 | 133s | Rendered good detail, but the character could not move in any attempt. |
| GPT-5.6 Sol | 5/5 | #5 | $1.35 | 120s | Best results: consistent, better than GPT-5.5, and with more detail in the game overall. |
| GPT-5.6 Terra | 3/5 | #1 | $0.44 | 39s | Good detail, but less consistent, I couldn't always walk. |
| GPT-5.6 Luna | 5/5 | #5 | $0.15 | 23s | Great results, though not as good as GPT-5.5 in my opinion. |
| Muse Spark 1.1 | 2/5 | #1 | $0.55 | 169s | Surprisingly awesome when it worked. Three of five were broken, but the working ones were on par with Fable and Sol, and better than the Grok and Opus ones. |
Overall, Claude did worse than we expected here. GPT outperformed every other model, Grok was a genuinely usable alternative at its price point, and Muse Spark was a real surprise on the runs that actually worked.
Task 2: 3D Rubik's Cube (scramble + solve)
Build a colorful, 3D-looking Rubik's Cube with Scramble and Solve buttons that visibly animate the rotations.
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
How we counted a "clean solve": we scrambled and solved the cube and only counted an attempt if both animations ran smooth, no glitches, no color changes.
| Model | Clean solve↕ | Best build | Cost (5 runs)↕ | Avg time↕ | Our take |
|---|---|---|---|---|---|
| Grok 4.5 | 3/5 | #4 | $0.65 | 191s | Good, simple results. |
| Claude Opus 4.8 | 0/5 | #1 | $0.56 | 44s | No perfect example, everything had a small issue and the cube's colors would change, though some were close. |
| GPT-5.5 | 4/5 | #4 | $1.36 | 136s | Good results, with small nits like flickering colors or not-quite-smooth rotation animations. |
| Qwen 3.7 Plus | 1/5 | #5 | $0.07 | 24s | When it worked, it looked great! |
| Kimi K2.6 | 1/5 | #4 | $1.05 | 59s | Not great. |
| Claude Fable 5 | 5/5 | #2 | $2.03 | 92s | No notes, really nice results. |
| DeepSeek V4 Pro | 1/5 | #3 | $0.35 | 380s | — |
| GLM-5.2 | 0/5 | — | $0.08 | 89s | Surprisingly, no working results. |
| GPT-5.6 Sol | 4/5 | #5 | $1.06 | 72s | Great results. One had weird animations and one rendered an all-black cube (interesting choice), but it did great on the ones that worked. |
| GPT-5.6 Terra | 4/5 | #4 | $0.42 | 38s | Fine, with weird scramble animations, felt like a small step up from GPT-5.5. |
| GPT-5.6 Luna | 0/5 | — | $0.14 | 24s | Usually rendered correctly at first, but scrambling immediately broke it. |
| Muse Spark 1.1 | 2/5 | #1 | $0.54 | 182s | A step above the OSS models, but not something I'd use over Grok for the price. |
Overall, we were surprised GPT underperformed here given its clear 3D lead in the raycaster. Claude did an amazing job again, though it was Fable carrying it with a clean five-for-five, while Opus, oddly, couldn't land a single flawless solve.
Play all five of any model live: Grok · Opus · Qwen (swap the number 1-5 for other attempts).
Task 3: Calculator
Digits, operators, clear, equals, correct operator precedence, real calculator look.
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
How we counted "working": nothing exhaustive, just basic calculations like (((5 × 5) − 100) / 10) to see how each one handled order of operations and rendered the result.
| Model | Working↕ | Best build | Cost (5 runs)↕ | Avg time↕ | Our take |
|---|---|---|---|---|---|
| Grok 4.5 | 5/5 | #5 | $0.37 | 110s | Simple, consistent results, no extra styling. |
| Claude Opus 4.8 | 5/5 | #1 | $0.46 | 35s | — |
| GPT-5.5 | 4/5 | #1 | $0.91 | 75s | Sometimes added extra buttons, and it tried to render in 3D, which often got cut off. |
| Qwen 3.7 Plus | 4/5 | #4 | $0.04 | 12s | Worked and pretty consistent, though one couldn't handle negative numbers. |
| Kimi K2.6 | 0/5 | #5 | $0.76 | 50s | Worked, but couldn't handle negative numbers. |
| Claude Fable 5 | 5/5 | #1 | $1.22 | 48s | My personal favorite, purely on style. |
| DeepSeek V4 Pro | 3/5 | #3 | $0.28 | 342s | One had out-of-order numbers, and one result didn't render. |
| GLM-5.2 | 2/5 | #1 | $0.07 | 63s | Great results when it worked. |
| GPT-5.6 Sol | 5/5 | #1 | $0.84 | 61s | Tries to go overboard on styling like GPT-5.5. That helped in the raycaster, but here it was annoying, some styling was off instead of clean and consistent. |
| GPT-5.6 Terra | 4/5 | #1 | $0.28 | 20s | Good results when it worked. |
| GPT-5.6 Luna | 5/5 | #1 | $0.11 | 16s | Good, consistent results, felt similar to Grok's. |
| Muse Spark 1.1 | 5/5 | #5 | $0.33 | 94s | Pretty good overall with no glaring issues, on par with Grok 4.5. Occasionally out-of-order or oddly-placed buttons cost it some style points. |
Overall, this was clearly Claude's best work: both Opus and Fable nailed all five, and Fable's was our favorite on style. GPT-5.6 Sol tries to go overboard with styles and render the calculator in 3D similar to Fable. However, it doesn't nail the styles, which leads to a worse overall experience. The simpler GPT models seemed to do a better job just because the experience worked out of the box.
Task 4: Conway's Game of Life
Grid canvas, Play/Pause/Step/Randomize/Clear, click to toggle cells, animated generations.
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
We did not run a separate five-attempt scoring pass on Life, so here it is just cost and time plus the general impression below.
| Model | Cost (5 runs)↕ | Avg time↕ |
|---|---|---|
| Grok 4.5 | $0.14 | 38s |
| Claude Opus 4.8 | $0.48 | 35s |
| GPT-5.5 | $1.06 | 79s |
| Qwen 3.7 Plus | $0.04 | 11s |
| Kimi K2.6 | $0.93 | 53s |
| Claude Fable 5 | $1.27 | 48s |
| DeepSeek V4 Pro | $0.25 | 304s |
| GLM-5.2 | $0.10 | 121s |
| GPT-5.6 Sol | $0.99 | 62s |
| GPT-5.6 Terra | $0.36 | 25s |
| GPT-5.6 Luna | $0.13 | 18s |
| Muse Spark 1.1 | $0.32 | 98s |
Grok 4.5 did well here, but the bigger takeaway is that this task was simple enough for the OSS models to do extremely well on. There is probably enough open example code for Game of Life out there that they were able to do a much better job at a way lower cost. Qwen 3.7 Plus and GLM-5.2 are clearly the go-to for something like this, but I would not rely on them generally, the other tasks show they still struggle with genuinely novel or more complex work.
The receipts: raw speed and cost (short answers)
Overall results.
| Model | Median latency↕ | Throughput↕ | Cost / reply↕ |
|---|---|---|---|
| GPT-5.6 Luna | 1.0s | 97 tok/s | $0.001 |
| GPT-5.6 Terra | 1.5s | 62 tok/s | $0.001 |
| GPT-5.6 Sol | 1.8s | 45 tok/s | $0.003 |
| Qwen 3.7 Plus | 2.1s | 204 tok/s | $0.001 |
| Claude Opus 4.8 | 2.5s | 44 tok/s | $0.004 |
| GPT-5.5 | 3.0s | 45 tok/s | $0.003 |
| Grok 4.5 | 3.0s | 112 tok/s | $0.003 |
| Muse Spark 1.1 | 3.1s | 125 tok/s | $0.002 |
| Kimi K2.6 | 4.5s | 83 tok/s | $0.01 |
| Claude Fable 5 | 6.6s | 30 tok/s | $0.01 |
| GLM-5.2 | 7.0s | 58 tok/s | $0.001 |
| DeepSeek V4 Pro | 9.3s | 37 tok/s | $0.001 |
Several open-weights buffered their whole reply in a burst and hit the 400-token cap, so their tok/s is a ceiling, not a true decode rate. The GPT-5.6 tiers are the fastest models on short prompts (Luna answers in about a second), Qwen is absurdly cheap and fast, and DeepSeek and GLM were really slow.
Bonus: draw a horse riding an astronaut (SVG, best of 5)
One-shot SVG, no libraries. We show each model's best-of-five (we prefer a strictly-valid SVG, then the most detailed).
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
![]() |
![]() |
![]() |
![]() |
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
![]() |
![]() |
![]() |
![]() |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
![]() |
![]() |
![]() |
![]() |
Personally, Claude Fable does a great job with the SVG rendering. It's funny most of the time as well and came up with good quality results. The GPT-5.6 models were surprisingly lackluster here since none of them included clean renderings of the horse or the astronaut. Grok 4.5 did pretty well here as well.
Bonus 2: Elon and Bezos watch a Blue Origin landing (SVG, best of 5)
A harder scene by request: two recognizable tech-billionaire caricatures watching a Blue Origin booster land on a pad in the open ocean. This one leans on composition and likeness. Again, Claude Fable sweeps the field: it produces great detail with a clean render, a shiny spot on Bezos' forehead, and smoke around the landing pad. GPT again produces pretty cartoony results; if that's something you prefer then maybe you can try improving the one-shot results it produces, since there are always some minor errors with the generations. The OSS models, GLM-5.2 and Qwen 3.7, also did really well for this task.
| Grok 4.5 | Claude Opus 4.8 | GPT-5.5 | Qwen 3.7 Plus |
|---|---|---|---|
![]() |
![]() |
![]() |
![]() |
| Kimi K2.6 | Claude Fable 5 | DeepSeek V4 Pro | GLM-5.2 |
| --- | --- | --- | --- |
![]() |
![]() |
![]() |
![]() |
| GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | Muse Spark 1.1 |
| --- | --- | --- | --- |
![]() |
![]() |
![]() |
![]() |
TL;DR
- The frontier still wins the hard tasks, and it is not particularly close on the complex ones. GPT-5.6 Sol and Claude Fable 5 were the standouts: Sol excelled at the raycaster, Fable at the Rubik's cube. Everything else did decently, but the best results are still at the frontier.
- There is a clear gap between SOTA and open-weights on genuinely novel or complex work, and you can see it in the raycaster and cube. But on a simple, well-trodden task like Game of Life, the OSS models hold their own, there is enough example code out there that Qwen 3.7 and GLM-5.2 nail it at a fraction of the cost. Use them for that class of problem; just do not lean on them generally, because the other tasks show they still struggle with the hard stuff.
- Grok 4.5 is genuinely "Opus level" on some tasks. If I cared about cost for my business, I would reach for it as a secondary execution model without hesitation. See Grok 4.5 vs Opus 4.8 and Grok 4.5 vs GPT-5.5.
- Muse Spark 1.1 pleasantly surprised me. I was not really expecting much, and it felt a step below Grok 4.5 but generally better than the open-weights. It is a real debut, though not something I would reach for just yet.
The takeaway holds even after launch day: the newest, most expensive flagship is not the automatic winner. Want to run these prompts yourself? Every model here is on one TryAI account, pay-as-you-go. Browse the models and start your own build-off.
Play with every attempt
Every raw build we generated, twelve models and five attempts per task. Click any number to open that exact attempt and poke at it yourself. A red asterisk (*) marks the attempt we liked best for that model (Game of Life had no per-attempt scoring, so it has none).
Raycaster maze
| Model | Attempts |
|---|---|
| Grok 4.5 | 1 · 2 · 3 · 4* · 5 |
| GPT-5.6 Sol | 1 · 2 · 3 · 4 · 5* |
| GPT-5.6 Terra | 1* · 2 · 3 · 4 · 5 |
| GPT-5.6 Luna | 1 · 2 · 3 · 4 · 5* |
| Muse Spark 1.1 | 1* · 2 · 3 · 4 · 5 |
| GPT-5.5 | 1* · 2 · 3 · 4 · 5 |
| Claude Opus 4.8 | 1 · 2* · 3 · 4 · 5 |
| Claude Fable 5 | 1* · 2 · 3 · 4 · 5 |
| Qwen 3.7 Plus | 1 · 2 · 3 · 4* · 5 |
| DeepSeek V4 Pro | 1 · 2 · 3 · 4* · 5 |
| Kimi K2.6 | 1 · 2* · 3 · 4 · 5 |
| GLM-5.2 | 1 · 2 · 3 · 4 · 5 |
Rubik's Cube
| Model | Attempts |
|---|---|
| Grok 4.5 | 1 · 2 · 3 · 4* · 5 |
| GPT-5.6 Sol | 1 · 2 · 3 · 4 · 5* |
| GPT-5.6 Terra | 1 · 2 · 3 · 4* · 5 |
| GPT-5.6 Luna | 1 · 2 · 3 · 4 · 5 |
| Muse Spark 1.1 | 1* · 2 · 3 · 4 · 5 |
| GPT-5.5 | 1 · 2 · 3 · 4* · 5 |
| Claude Opus 4.8 | 1* · 2 · 3 · 4 · 5 |
| Claude Fable 5 | 1 · 2* · 3 · 4 · 5 |
| Qwen 3.7 Plus | 1 · 2 · 3 · 4 · 5* |
| DeepSeek V4 Pro | 1 · 2 · 3* · 4 · 5 |
| Kimi K2.6 | 1 · 2 · 3 · 4* · 5 |
| GLM-5.2 | 1 · 2 · 3 · 4 · 5 |
Calculator
| Model | Attempts |
|---|---|
| Grok 4.5 | 1 · 2 · 3 · 4 · 5* |
| GPT-5.6 Sol | 1* · 2 · 3 · 4 · 5 |
| GPT-5.6 Terra | 1* · 2 · 3 · 4 · 5 |
| GPT-5.6 Luna | 1* · 2 · 3 · 4 · 5 |
| Muse Spark 1.1 | 1 · 2 · 3 · 4 · 5* |
| GPT-5.5 | 1* · 2 · 3 · 4 · 5 |
| Claude Opus 4.8 | 1* · 2 · 3 · 4 · 5 |
| Claude Fable 5 | 1* · 2 · 3 · 4 · 5 |
| Qwen 3.7 Plus | 1 · 2 · 3 · 4* · 5 |
| DeepSeek V4 Pro | 1 · 2 · 3* · 4 · 5 |
| Kimi K2.6 | 1 · 2 · 3 · 4 · 5* |
| GLM-5.2 | 1* · 2 · 3 · 4 · 5 |
Game of Life
| Model | Attempts |
|---|---|
| Grok 4.5 | 1 · 2 · 3 · 4 · 5 |
| GPT-5.6 Sol | 1 · 2 · 3 · 4 · 5 |
| GPT-5.6 Terra | 1 · 2 · 3 · 4 · 5 |
| GPT-5.6 Luna | 1 · 2 · 3 · 4 · 5 |
| Muse Spark 1.1 | 1 · 2 · 3 · 4 · 5 |
| GPT-5.5 | 1 · 2 · 3 · 4 · 5 |
| Claude Opus 4.8 | 1 · 2 · 3 · 4 · 5 |
| Claude Fable 5 | 1 · 2 · 3 · 4 · 5 |
| Qwen 3.7 Plus | 1 · 2 · 3 · 4 · 5 |
| DeepSeek V4 Pro | 1 · 2 · 3 · 4 · 5 |
| Kimi K2.6 | 1 · 2 · 3 · 4 · 5 |
| GLM-5.2 | 1 · 2 · 3 · 4 · 5 |
Try it yourself
Every model mentioned here is available on TryAI with one account, pay-as-you-go, no subscription.























