At a certain point, speed >> smartness
英文标题：At a certain point, speed >> smartness
It feels like a zig-zag: you don't want a model that's too dumb to do anything agentic. But once a model is good enough to be agentic, you don't want it to run so slow that iterating takes hours. For me the sweet spot is something like ~500 tps prefill, ~25tps decode. If a model isn't fast enough to
打开原文相关 3 · 机会 0
中文
It feels like a zig-zag: you don't want a model that's too dumb to do anything agentic. But once a model is good enough to be agentic, you don't want it to run so slow that iterating takes hours. For me the sweet spot is something like ~500 tps prefill, ~25tps decode. If a model isn't fast enough to
中文全文翻译正在补齐，可先查看下方英文原文。
English / 原文
It feels like a zig-zag: you don't want a model that's too dumb to do anything agentic. But once a model is good enough to be agentic, you don't want it to run so slow that iterating takes hours. For me the sweet spot is something like ~500 tps prefill, ~25tps decode. If a model isn't fast enough to