Building an open-source control plane for self-hosted vLLM, what would you want in it?
英文标题:Building an open-source control plane for self-hosted vLLM, what would you want in it?
Every time I self-host a model I rebuild the same stuff: start the container, set up a route, check why it died overnight, remember to shut the GPU off before it burns money. So I'm building a panel that handles it. Start/stop models, OpenAI-compatible endpoint, health checks with auto-restart, basi
打开原文相关 4 · 机会 2
中文
Every time I self-host a model I rebuild the same stuff: start the container, set up a route, check why it died overnight, remember to shut the GPU off before it burns money. So I'm building a panel that handles it. Start/stop models, OpenAI-compatible endpoint, health checks with auto-restart, basi
中文全文翻译正在补齐,可先查看下方英文原文。
English / 原文
Every time I self-host a model I rebuild the same stuff: start the container, set up a route, check why it died overnight, remember to shut the GPU off before it burns money. So I'm building a panel that handles it. Start/stop models, OpenAI-compatible endpoint, health checks with auto-restart, basi