LLM 推理有多赚钱？以 Kimi K3 为例算一笔账

[视频/嵌入内容：视频]

引言 __

LLM 推理到底有多赚钱？

这是位于 AI 狂热核心的一个关键问题。

本文将介绍 token 定价的基础知识，建立一些直觉，再将其应用于最近发布的 Kimi K3，以回答盈利能力这一问题。

LLM 定价速览 __

要生成 token，我们需要 GPU 和模型权重。我们将权重加载到一张或多张 GPU 上。大型前沿模型无法装进单张 GPU：Kimi K3 至少需要 16 张 B200 或 8 张 B300。

GPU 可以按小时租用。目前，我们可以前往 Lambda 或 Runpod，以约 7 美元/小时的价格租用一张 B300（运行 Kimi 至少需要其中 8 张）。

Token 按百万计费。输出 token 是模型预测生成的内容，输入 token 则是提示词。两者都需要处理：一次前向传播可以同时处理所有输入 token，而每个输出 token 都需要单独进行一次前向传播（不过投机解码会改变这一情况）。输入 token 被处理后会生成一种内部状态（KV cache），该状态会被复用，因此无需重新计算。缓存输入 token 最便宜：不需要 GPU 计算，只需从缓存中加载即可，缓存可以位于 GPU VRAM、RAM，甚至 SSD 等更冷的存储中。

以下是 Kimi K3 在 OpenRouter 上的定价：

每 100 万输入 token 的价格为 3 美元，输出 token 为 15 美元，缓存 token 为 0.30 美元。

这些价格从何而来？下面是一个简单模型。

GPU 生成 token。从边际角度看，一定量的 GPU 时间可以生成一定量的 token：

cost per M tokens = hourly_GPU_price / tokens_per_gpu_hour * 1,000,000

这就是全部的生意。现在核心问题是：什么决定了 tokens_per_gpu_hour？

b 与 N_GPU 的微妙关系，以及 Pareto 前沿 __

有两个至关重要的调节旋钮：

• 批处理大小（b）： 在同一次前向传播中，通过相同权重同时处理的请求数量
• GPU 数量（N_GPU）： 用多少张芯片来分摊一个模型

token_latency = fn(b, N_GPU)

token_latency 是批处理大小和 GPU 数量的函数。token_latency 指一次前向传播所需的秒数，也就是为批次中每条序列各生成一个新 token、即生成 b 个 token 所需的时间。

随后，我们可以计算一组 GPU 每小时能产出多少 token：

tokens_per_gpu_hour = (b / token_latency) / N_GPU * 3600

之所以要除以 N_GPU，是因为该延迟是同时使用 N_GPU 张芯片换来的，因此每张芯片只应承担其对应份额的输出。

还有一个至关重要的因素：

token_latency 同时决定了速度。1/token_latency 就是每位用户体验到的 token/秒数：延迟越低，速度越快。通常，在 GPU 数量固定时，降低 b（批处理大小）会提升速度。因此，我们是在用吞吐量（总 token 数）换取速度。而单位时间内生成的 token 越少，token 的成本就越高。

每一组 (b, N_GPU) 都会映射到一组 (token_latency, cost)，因为它们共同决定了速度/吞吐量和价格。

b 和 N_GPU 之间存在一种可以称为“微妙关系”的互动：

• 更大的 N_GPU 意味着更多的总计算能力和内存带宽 -> 可以支持更大的批次。
• 更大的 N_GPU 也意味着更多的总内存，因此能容纳更多 KV cache -> 可以支持更大的批次。
• 但与此同时，更大的 N_GPU 也意味着更高的网络开销，因为协调信息必须在所有 GPU 之间传递 -> 延迟更高。
• 更大的 b 则意味着更多数据需要通过网络传输 -> 延迟更高。

Pareto 前沿是由 (b, N_GPU) 及其对应的 (token_latency, cost) 组成的集合；在这一集合中，无法改善其中一个指标而不让另一个指标变差：要么更慢，要么更贵。

成本/速度前沿很可能是判断盈利能力的关键图表。幸运的是，vLLM 公告博客中就有这样一张图：

前沿上的每一个点，都对应在该速度下的最优价格。结合上面的 OpenRouter 定价来看，Fireworks 提供了两个版本的 Kimi：常规版和快速版。快速版的价格为每百万 token 22 美元，而非 15 美元，但能提供更高的 tok/s（更低的 token_latency）。它们很可能运行了更小的批次，或在相同批处理大小下使用了更多 GPU，以更高成本换取速度。

Kimi K3 推理有多赚钱 __

Kimi K3 是一个约 2.8 万亿参数的模型。量化为 MXFP4 后，其权重约为 1.4TB：单张 GPU 无法容纳。一张 B300 拥有 288GB 内存，8 张合计可提供 2.3TB，足以容纳权重并为 KV cache 留出余量。

在成本/速度前沿的 Pareto 最优点上，该配置可为单个用户提供约 30 tok/s 的速度，同时每张 GPU 的总吞吐量约为 1,500 tok/s（这正是批处理发挥作用的结果——没有任何单个用户会看到 1,500 tok/s，但每张 GPU 在所有用户合计下每秒能服务这么多 token）。

tokens_per_gpu_hour = 1,500 * 3,600 = 5,400,000 tokens/hour/GPU

B300 的租赁价格约为 7.39 美元/小时：

cost per M tokens = $7.39 / 5,400,000 * 1,000,000 ≈ $1.37 per million tokens

好得令人难以置信 __

每百万 token 1.37 美元，与标出的 15 美元价格相差甚远。推理服务商是否正在一路笑着走进银行？或许会心一笑，但还不至于开怀大笑。有两大因素会削弱这份喜悦：

• 利用率不可能达到 100%。 你需要为峰值需求配置资源，而不是为平均需求配置资源。需求具有突发性：编程模型在夜间、周末和节假日的活跃度较低。服务商规模越大，越能通过多个需求峰值来平滑需求。但假设利用率为 50%，成本就会立刻翻倍。
• **这只是边际成本，**即生成额外 token 的成本。此外还有薪资、其他 IT 成本，以及最重要的训练和微调成本。模型是一种会折旧的资产。当前开放权重模型无处不在，但总得有人承担训练成本，而这些成本必须在某个环节被计入。

vLLM 的文章提到，结合 DSpark 的投机解码可带来约 3 倍加速，这一提升非常可观，而且这还只是一个优化方向。KV cache 卸载、预填充/解码分离，以及其他技术都正在积极开发中。任何推理服务商都可能拥有某种秘密优势，使其经济效益远高于这份粗略估算所显示的水平。

VIDEO

Intro __

How profitable is LLM inference?

This is a central question sitting at the heart of the AI mania.

In this post, we’ll cover token pricing basics, build up some intuition, then apply it to the recently released Kimi K3 to answer the profitability question.

LLM Pricing TLDR __

To produce tokens, we need GPUs and model weights. We load the weights onto one or many GPUs. Large frontier models won’t fit on a single GPU: Kimi K3 needs 16 B200s or 8 B300s at minimum.

GPUs can be rented by the hour. Right now we can head to Lambda or Runpod and rent a B300 for ~$7/hour (at least 8 of those to run Kimi).

Tokens are billed by the million. Output tokens are what the model predicts, input tokens are the prompt. Both need processing: one forward pass can process all input tokens at once, while each output token needs its own pass (spec decoding changes this, though). Once input tokens are processed, they generate an internal state (KV cache) that gets reused, so it’s never recomputed. Cached input tokens are cheapest: no GPU compute, just a load from cache (GPU VRAM, RAM, or even colder storage like SSD).

Here’s Kimi K3 pricing on OpenRouter:

1M input tokens cost $3, output is $15, cached tokens are $0.30.

Where do these prices come from? Here’s a simple model.

GPUs produce tokens. At the margin, some amount of GPU time produces some amount of tokens:

`

1

|

cost per M tokens = hourly_GPU_price / tokens_per_gpu_hour * 1,000,000

---|---
`

That’s the whole business. Now the central question is: what determinestokens_per_gpu_hour?

The b and N_GPU Situationship and Pareto Frontier __

There are two crucial knobs:

• batch size (b): how many requests you process together through the same weights in the same forward pass
• number of GPUs (N_GPU): how many chips we spread one model across

`

1

|

token_latency = fn(b, N_GPU)

---|---
`

token_latency is a function of the batch size and number of GPUs. token_latency is how long, in seconds, one forward pass takes, the time to produce one new token for every sequence in the batch, i.e. b tokens.

We can then determine how many tokens a fleet of GPUs cranks out per hour:

`

1

|

tokens_per_gpu_hour = (b / token_latency) / N_GPU * 3600

---|---
`

We divide by N_GPU because that latency was bought with N_GPU chips at once, so each chip is only accountable for its share of the output.

One more crucial thing:

token_latency also determines speed. 1/token_latency is how many tokens/second each user experiences: lower latency, faster things are. Usually, lowering b (batch size) at a fixed number of GPUs increases speed. So we trade throughput (total tokens) for speed. And fewer tokens per unit time means more expensive tokens.

Each (b, N_GPU) pair maps to a (token_latency, cost) pair, since together they set both speed/throughput and price.

b and N_GPU are in what you could call a situationship:

• Bigger N_GPU means more aggregate compute and memory bandwidth -> bigger batch.
• Bigger N_GPU also means more aggregate memory, so more room for KV cache -> bigger batch.
• At the same time, bigger N_GPU means more network overhead, since coordination has to travel across all the GPUs -> higher latency.
• And bigger b means more data that has to move across the network -> higher latency.

The Pareto frontier is the set of (b, N_GPU) and their corresponding(token_latency, cost) pairs where you can’t move one without making the other worse: slower or more expensive.

The cost/speed frontier is probably the key graph for determining profitability. Luckily, the vLLM announcement blog has one:

Each point on the frontier gives the optimal price for that speed. Looking at the OpenRouter pricing above, Fireworks offers two versions of Kimi: regular and fast, with fast costing $22 per million tokens instead of $15 but at higher tok/s (lower token_latency). They’re likely running a smaller batch, or more GPUs for the same batch size, buying speed at a higher cost.

How profitable is Kimi K3 inference __

Kimi K3 is a ~2.8 trillion parameter model. Quantized to MXFP4, the weights come out to about 1.4TB: too big for one GPU. A B300 has 288GB of memory, 8 of them gives 2.3TB, enough for the weights plus KV cache headroom.

At the Pareto-optimal point on the cost/speed frontier, this setup gets roughly 30 tok/s for a single user, and about 1,500 tok/s of aggregate throughput per GPU (that’s batching at work — no single user sees 1,500 tok/s, but each GPU serves that many tokens/sec across everyone combined).

`

1

|

tokens_per_gpu_hour = 1,500 * 3,600 = 5,400,000 tokens/hour/GPU

---|---
`

B300s rent for about $7.39/hour:

`

1

|

cost per M tokens = $7.39 / 5,400,000 * 1,000,000 ≈ $1.37 per million tokens

---|---
`

Too good to be true __

$1.37 per million tokens is very far from the advertised $15. Are inference providers laughing their way to the bank? Maybe a chuckle, not a laugh. Two big things dampen the joy:

• Utilization won’t be 100%. You provision for peak demand, not average. Demand is bursty: coding models see less activity nights, weekends, holidays. The larger the provider, the more demand smooths out across many peaks. But assume 50% utilization and cost doubles immediately.
• This is only marginal cost the cost of producing extra tokens. There’s also salaries, other IT costs, and most importantly, training and fine-tuning. A model is a depreciating asset. Open weights are everywhere right now, but someone shoulders the training cost, and that has to be accounted for somewhere.

The vLLM post mentions speculative decoding with DSpark giving a ~3x speedup which is tremendous and that’s just one optimization vector. KV cache offloading, prefill/decode disaggregation, and others are actively being worked on. Any inference provider might be sitting on a secret edge that makes the economics far more lucrative than this back-of-the-envelope math suggests.