Deepseek V4 Flash ~105 t/s on two Nvidia 4090d 48G (ada) in vLLM
英文标题：Deepseek V4 Flash ~105 t/s on two Nvidia 4090d 48G (ada) in vLLM
TLDR: I (with the help of AI) re-implemented every Blackwell-only kernel (DeepGEMM, FlashInfer sparse-MLA, block-scaled FP8) in Triton, because they simply don't exist for sm89. The performance is 2-3x more for parallel agentic workflows. Benchmark llama-server vs vLLM I was inspired by the po
打开原文相关 9 · 机会 1
中文
TLDR: I (with the help of AI) re-implemented every Blackwell-only kernel (DeepGEMM, FlashInfer sparse-MLA, block-scaled FP8) in Triton, because they simply don't exist for sm89. The performance is 2-3x more for parallel agentic workflows. Benchmark llama-server vs vLLM I was inspired by the po
中文全文翻译正在补齐，可先查看下方英文原文。
English / 原文
TLDR: I (with the help of AI) re-implemented every Blackwell-only kernel (DeepGEMM, FlashInfer sparse-MLA, block-scaled FP8) in Triton, because they simply don't exist for sm89. The performance is 2-3x more for parallel agentic workflows. Benchmark llama-server vs vLLM I was inspired by the po