community Reddit LocalLLaMA 2026-07-25T22:20 正文 thin 0 words

4x 3090, 96gb vram what Model to drive Hermes?

英文标题:4x 3090, 96gb vram what Model to drive Hermes?

3 year lurker, now i finally got my server up and running. dont know which model to choose. llama.cpp or vllm, what makes more sense? mainly single user with maybe 2-3 more additional users in family, if everything checks out. hermes is gonna be used as "ai playground" to manifest ideas on tailscale
打开原文相关 3 · 机会 1

中文

3 year lurker, now i finally got my server up and running. dont know which model to choose. llama.cpp or vllm, what makes more sense? mainly single user with maybe 2-3 more additional users in family, if everything checks out. hermes is gonna be used as "ai playground" to manifest ideas on tailscale

中文全文翻译正在补齐,可先查看下方英文原文。

English / 原文

3 year lurker, now i finally got my server up and running. dont know which model to choose. llama.cpp or vllm, what makes more sense? mainly single user with maybe 2-3 more additional users in family, if everything checks out. hermes is gonna be used as "ai playground" to manifest ideas on tailscale