How are y’all stomaching the “AI Boom” prices?

I am in the middle of considering an upgrade to my Home Server. I want to get a decent GPU for LocalAI. I mean, I have an RTX 4060 TI 16GB, I know that is much more than most people have, but even though it has a lot of cram the bus width is really slowing it down - I get ~23 tok/s on Gemma 4 26b A4

中文全文翻译正在补齐，可先查看下方英文原文。

I am in the middle of considering an upgrade to my Home Server. I want to get a decent GPU for LocalAI. I mean, I have an RTX 4060 TI 16GB, I know that is much more than most people have, but even though it has a lot of cram the bus width is really slowing it down - I get ~23 tok/s on Gemma 4 26b A4