(NInfer Fork) I wanted to have a 1M context Qwen-3.8 27B, tp2, dual 5090s
英文标题：(NInfer Fork) I wanted to have a 1M context Qwen-3.8 27B, tp2, dual 5090s
Hey! I forked NInfer (a from-scratch C++20/CUDA inference engine for Qwen models) and added two things: tensor-parallel across two GPUs, and YaRN ×4 rope scaling. Together they let Qwen3.8-27B NVFP4 run a 1,048,576-token context on two consumer 5090s — 27.4 GB per card, no NVLink. Numbers (sin
打开原文相关 2 · 机会 0
中文
Hey! I forked NInfer (a from-scratch C++20/CUDA inference engine for Qwen models) and added two things: tensor-parallel across two GPUs, and YaRN ×4 rope scaling. Together they let Qwen3.8-27B NVFP4 run a 1,048,576-token context on two consumer 5090s — 27.4 GB per card, no NVLink. Numbers (sin
中文全文翻译正在补齐，可先查看下方英文原文。
English / 原文
Hey! I forked NInfer (a from-scratch C++20/CUDA inference engine for Qwen models) and added two things: tensor-parallel across two GPUs, and YaRN ×4 rope scaling. Together they let Qwen3.8-27B NVFP4 run a 1,048,576-token context on two consumer 5090s — 27.4 GB per card, no NVLink. Numbers (sin