Osmantic ODS：将 PC、Mac 或 Linux 主机变成私有 AI 服务器

Osmantic Deployment System

将你的 PC、Mac 或 Linux 主机变成一台私有 AI 服务器。

AI 服务器和家庭实验室的搭建正迅速成为一个已基本解决的问题。每个人都应该拥有这样的体验。

ODS 会安装并连接起本地运行 AI 所需的一切，因此你无需手动拼装 Ollama、Open WebUI、n8n、ComfyUI 和隐私工具：

• 本地模型推理 —— 在你自己的硬件上运行开放模型
• ChatGPT 风格 Web UI —— 可从任何浏览器与模型对话
• 控制仪表盘 —— 在一处管理模型、服务、设置、GPU 状态和扩展
• 语音、Agent 和工作流 —— 构建能够聆听、说话、调用工具并完成任务的自动化流程
• RAG 与搜索 —— 连接本地文档、私有搜索和检索工作流
• 图像生成 —— 运行本地图像工具，无需将提示词发送给托管 API
• 隐私与运维 —— 将服务认证、密钥、可观测性和诊断能力整合在一个本地技术栈中

无需云端。无需订阅。除非你自行选择，否则提示词和数据都会留在你的设备上。需要时可选用云端和混合 API 模式。

**版本发布验证：**运行层面的变更会在发布级设备群和发行版实验室中接受检查：零前置条件引导、全新安装、产品流程、完整模型能力、生命周期恢复，以及最终的 User Green 门禁。有关一次绿色运行所证明内容，请参阅 Release Validation。

**仓库布局：**仓库根目录包含公开 README、安装程序、安全策略、GitHub 工作流及项目协调文档。ods/ 目录则是产品运行时所在位置，包括服务、安装程序阶段、Compose 覆盖层、仪表盘、CLI、测试和运维文档。

稳定使用：v2.6.0 是当前稳定版本。main 分支更新较快，应将其用于积极开发和验证候选版本。对于 Fork、设备镜像、实验室或类生产安装，请固定到带标签的发布版本或已审计提交，并保留自己的验证记录。稳定补丁修复会先进入 release/2.6.x，再向前合并。参阅 Release Channels、Installer Trust 和 Forkability。

选择你的系统，复制相应代码块，并在普通终端中运行。ODS 将安装整套技术栈，为你的硬件选择模型，启动服务，并提供本地 Web UI。

Linux 或 macOS

curl -fsSL https://install.osmantic.com/ods.sh | bash

Windows PowerShell

$ProgressPreference = "SilentlyContinue"
$odsSrc = Join-Path $env:TEMP ("ods-install-" + [guid]::NewGuid().ToString("N"))
$odsZip = Join-Path $odsSrc "ods-main.zip"
New-Item -ItemType Directory -Path $odsSrc | Out-Null
Invoke-WebRequest "https://github.com/Osmantic/ODS/archive/refs/heads/main.zip" -OutFile $odsZip
Expand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force
cd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1

前置条件：必须已安装并正在运行 Docker。在 Windows 上，请使用启用了 WSL2 后端的 Docker Desktop，并在普通、非管理员权限的 PowerShell 窗口中运行上述代码块。

托管的 Linux/macOS 端点会代理仓库 main 分支中当前的引导程序。经过审查的合并会在边缘缓存刷新后自动到达该端点。ODS_REF 用于选择兼容的仓库检出版本。参阅 Installer Trust，以检查脚本，或手动安装稳定版本或已审计提交。

Windows 用户不应在 PowerShell 中运行 curl ... | bash 命令。上方的 PowerShell 代码块会下载源代码 ZIP，并运行与基于克隆工作流相同的 Windows 安装程序。更多细节参阅 Windows Quickstart。

安装后，打开 http://localhost:3000 并开始对话。

之后可使用与平台对应的命令卸载：

cd ~/ods
./ods-uninstall.sh --force


$installDir = "$env:USERPROFILE\ods"
cd $installDir
.\ods.ps1 uninstall --force

Windows 恢复说明：如果运行时目录不完整且缺少 .\ods.ps1，请在源代码检出目录中使用 ./ods\installers\windows\ods.ps1 uninstall --force 运行相同命令。该操作会先移除标记为 ODS Compose 项目的 Docker 资源，再删除运行时目录。

**API 端点：**Linux Docker 安装默认在 http://localhost:11434（OLLAMA_PORT）暴露 llama-server，而容器使用 llama-server:8080。macOS 原生 Metal 路径和 Windows 原生/Lemonade 路径默认使用 http://localhost:8080，除非另行覆盖。Open WebUI 始终位于 http://localhost:3000。

**没有 GPU？**ODS 同样支持云端模式——使用 OpenAI/Anthropic/Together API 而非本地推理来驱动相同的完整技术栈：

**端口冲突？**每个端口都可通过环境变量配置。完整列表请参阅 .env.example，或在安装时覆盖：

WEBUI_PORT=9090 ./install.sh

**初次接触？**请阅读 Friendly Guide，或收听音频版本——其中完整介绍了 ODS 是什么、如何工作，以及如何将其改造成适合自己的系统。无需技术背景。

---

问题	回答
它是什么？	面向自有硬件的本地 AI 服务器技术栈，提供一键式 Linux/macOS 安装程序和适用于 Windows 的 PowerShell 安装程序。
适合谁使用？	希望在家庭、实验室或工作站中使用私有 AI，又不想手动连接十几个服务的人。
我能获得什么？	本地推理、Open WebUI 聊天、控制仪表盘、语音、Agent、工作流、RAG、搜索、图像生成、隐私工具、可观测性和开发者工具。
支持什么平台？	Linux、搭配 WSL2/Docker Desktop 的 Windows，以及 Apple Silicon macOS。
必须使用云端吗？	不需要。本地模式是默认选项；云端和混合 API 模式为可选项。
如果你了解……	ODS 增加了……
---	---
Ollama / llama.cpp	围绕其构建的服务器技术栈：聊天、仪表盘、语音、RAG、工作流、Agent、隐私和服务管理。
Open WebUI	围绕 Open WebUI 的完整安装程序和控制平面，以及预先连接好的本地服务。
AnythingLLM	超越 RAG 的更广泛本地 AI 设备能力：推理、聊天、语音、工作流、图像生成和运维。
n8n 自托管 AI 入门套件	将工作流自动化作为更大型私有 AI 服务器的一部分。

---

当前平台支持情况

平台	状态
Linux（NVIDIA + AMD + Intel Arc）	已支持 —— 今天即可安装并运行
Windows（NVIDIA + AMD）	已支持 —— 今天即可安装并运行
macOS（Apple Silicon）	已支持 —— 今天即可安装并运行

**已测试的 Linux 发行版：**Ubuntu 24.04/22.04、Debian 12、Linux Mint 21.3、Fedora 41+、Rocky Linux 9、Arch Linux、Manjaro、CachyOS 和 openSUSE Tumbleweed。其他使用 apt、dnf、pacman 或 zypper 的发行版也应可运行——如果你的发行版无法运行，请提交 Issue。

**版本发布验证：**运行层面的变更会通过发布级门禁，该门禁覆盖零前置条件引导、干净安装、产品行为、完整模型能力、生命周期恢复和 User Green。请参阅 Release Validation 和 Validation Matrix。

**Windows：**需要使用 WSL2 后端的 Docker Desktop。NVIDIA GPU 使用 Docker GPU 直通；AMD Strix Halo 则通过 Windows 安装程序和支持矩阵中记录的平台专用加速路径运行。

**macOS：**需要 Apple Silicon（M1+）和 Docker Desktop。llama-server 以原生方式运行并使用 Metal GPU 加速；其他所有服务均在 Docker 中运行。

有关受支持的平台声明，请参阅 Support Matrix；有关用于验证这些声明的分层测试范围，请参阅 Validation Matrix。

---

少数几家公司控制着全球绝大多数 AI 流量——以及随之而来的你的数据、成本和可用性。你发送给中心化服务商的每一次查询，都是你并不拥有的商业情报；它运行在你无法控制的基础设施上，定价条款也无法由你协商。

如果 AI 正成为关键基础设施，它就不应被租用。自托管本地 AI 应当是一项主权人权，而不是一种职业选择。

因为运行自己的 AI 不应该要求你拥有计算机科学学位，再花一个周末调试 CUDA 驱动。目前，搭建本地 AI 意味着把十几个项目拼接起来，从零编写 Docker 配置，并祈祷所有组件能彼此通信。多数人最终会放弃，重新回到为 OpenAI 付费的模式。

我们构建 ODS，就是为了让你不必这样做。

• 一条命令 —— 检测 GPU、选择正确模型、生成凭据并启动所有服务
• 2 分钟内开始聊天 —— 引导模式可立即提供可用模型，同时在后台下载完整模型
• 预先连接好的完整服务栈 —— 聊天、Agent、语音、工作流、搜索、RAG、图像生成、隐私工具、可观测性和开发者工具。开箱即用，所有组件可彼此通信
• 完全可改造 —— 每项服务都是一个扩展。放入一个文件夹，运行 ods enable，即可完成

手动安装（Linux）

git clone https://github.com/Osmantic/ODS.git
cd ODS/ods
./install.sh

Windows（PowerShell）

需要启用 WSL2 后端的 Docker Desktop。请先安装 Docker Desktop，并确保在开始前它已运行。

打开普通的 PowerShell 会话并运行：

$ProgressPreference = "SilentlyContinue"
$odsSrc = Join-Path $env:TEMP ("ods-install-" + [guid]::NewGuid().ToString("N"))
$odsZip = Join-Path $odsSrc "ods-main.zip"
New-Item -ItemType Directory -Path $odsSrc | Out-Null
Invoke-WebRequest "https://github.com/Osmantic/ODS/archive/refs/heads/main.zip" -OutFile $odsZip
Expand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force
cd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1

Set-ExecutionPolicy 命令允许安装脚本在当前会话中运行。它不会更改你的系统级策略。不建议以管理员身份运行安装程序，因为 .opencode、data/ 和 .env 等用户级路径可能会以管理员拥有的权限创建。

安装程序会检测 GPU、选择正确模型、生成凭据、启动所有服务，并在桌面创建 Dashboard 快捷方式。可在运行时目录中使用 .\ods.ps1 status 进行管理；使用 .\ods.ps1 uninstall --force 卸载。

macOS（Apple Silicon）

需要 Apple Silicon（M1+）和 Docker Desktop。请先安装 Docker Desktop，并确保在开始前它已运行。

git clone https://github.com/Osmantic/ODS.git
cd ODS/ods
./install.sh

安装程序会检测你的芯片，根据统一内存选择正确模型，以原生方式启动采用 Metal 加速的 llama-server，并在 Docker 中启动其他所有服务。使用 ./ods-macos.sh status 进行管理。

详情请参阅 macOS Quickstart。

---

• Open WebUI —— 功能完整的聊天界面，支持对话历史、网页搜索、文档上传和 30 多种语言

• llama-server —— 支持连续批处理的高性能 LLM 推理服务，会根据 GPU 自动选择；Linux Docker 主机 API 默认使用 localhost:11434，原生 macOS/Windows 路径使用 localhost:8080，容器 API 则运行在 8080

• LiteLLM —— 支持本地、云端和混合模式的 API 网关

• TEI Embeddings —— 面向 RAG 和搜索工作流的文本嵌入服务

• Whisper —— 语音转文字

• Kokoro —— 文字转语音

• Hermes Agent —— 默认的本地优先自主/浏览器 Agent，具有记忆、技能和受魔法链接门控保护的代理

• OpenClaw —— 已弃用的旧版自主 Agent，在迁移窗口期间仍可选择启用

• n8n —— 具备 400 多项集成的工作流自动化

Osmantic Deployment System

Turn your PC, Mac, or Linux box into a private AI server.

AI server and homelab setup is rapidly becoming a solved problem. It should feel that way for everyone.

ODS installs and wires together everything you need to run AI locally, so you do not have to assemble Ollama, Open WebUI, n8n, ComfyUI, and privacy tools by hand:

• Local model inference — run open models on your own hardware
• ChatGPT-style web UI — talk to your models from any browser
• Control dashboard — manage models, services, setup, GPU status, and extensions from one place
• Voice, agents, and workflows — build automations that can listen, speak, call tools, and get work done
• RAG and search — connect local documents, private search, and retrieval workflows
• Image generation — run local image tools without sending prompts to a hosted API
• Privacy and ops — keep service auth, secrets, observability, and diagnostics in one local stack

No cloud required. No subscriptions required. Your prompts and data stay on your machine unless you choose otherwise. Cloud and hybrid API modes are optional when you want them.

Release validation: Operational changes are checked with a release-grade fleet and distro lab: zero-prereq bootstrap, fresh installs, product flows, full-model capabilities, lifecycle recovery, and the final User Green gate. See Release Validation for what a green run proves.

Repo layout: the repository root holds the public README, installers, security policy, GitHub workflows, and project coordination docs. The ods/ directory is the product runtime: services, installer phases, compose overlays, dashboard, CLI, tests, and operator docs.

Stable consumption: v2.6.0 is the current stable release. main moves quickly; use it for active development and validation candidates. For forks, appliances, labs, or production-like installs, pin a tagged release or audited commit and keep your own validation receipt. Stable patch fixes land on release/2.6.x before being merged forward. See Release Channels, Installer Trust, and Forkability.

Choose your system, copy the block, run it in a normal terminal. ODS installs the stack, picks a model for your hardware, starts the services, and gives you the local web UI.

Linux or macOS

curl -fsSL https://install.osmantic.com/ods.sh | bash

Windows PowerShell

$ProgressPreference = "SilentlyContinue"
$odsSrc = Join-Path $env:TEMP ("ods-install-" + [guid]::NewGuid().ToString("N"))
$odsZip = Join-Path $odsSrc "ods-main.zip"
New-Item -ItemType Directory -Path $odsSrc | Out-Null
Invoke-WebRequest "https://github.com/Osmantic/ODS/archive/refs/heads/main.zip" -OutFile $odsZip
Expand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force
cd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1

Prerequisites: Docker must be installed and running. On Windows, use Docker Desktop with the WSL2 backend enabled and run the block in a normal, non-Administrator PowerShell window.

The hosted Linux/macOS endpoint proxies the current bootstrap from repository main. Reviewed merges reach it automatically after edge-cache refresh. ODS_REF selects a compatible repository checkout. See Installer Trust to inspect the script or install a stable release or audited commit manually.

Windows users should not run the curl ... | bash command from PowerShell. The PowerShell block above downloads the source ZIP and runs the same Windows installer used by the clone-based workflow. For more detail, see the Windows Quickstart.

After install, open http://localhost:3000 and start chatting.

Uninstall later with the matching platform command:

cd ~/ods
./ods-uninstall.sh --force


$installDir = "$env:USERPROFILE\ods"
cd $installDir
.\ods.ps1 uninstall --force

Windows recovery note: if the runtime folder is partial and .\ods.ps1 is missing, run the same command from a source checkout as .\ods\installers\windows\ods.ps1 uninstall --force. It removes Docker resources labelled as the ODS compose project before removing the runtime directory.

API endpoint: Linux Docker installs expose llama-server on http://localhost:11434 by default (OLLAMA_PORT) while containers use llama-server:8080. macOS native Metal and Windows native/Lemonade paths use http://localhost:8080 unless overridden. Open WebUI stays on http://localhost:3000.

No GPU? ODS also runs in cloud mode — same full stack, powered by OpenAI/Anthropic/Together APIs instead of local inference:

Port conflicts? Every port is configurable via environment variables. See .env.example for the full list, or override at install time:

WEBUI_PORT=9090 ./install.sh

New here? Read the Friendly Guide or listen to the audio version — a complete walkthrough of what ODS is, how it works, and how to make it your own. No technical background needed.

---

Question	Answer
What is it?	A local AI server stack for your own hardware, with a one-command Linux/macOS installer and a PowerShell installer for Windows.
Who is it for?	People who want private AI at home, in a lab, or on a workstation without hand-wiring a dozen services.
What do I get?	Local inference, Open WebUI chat, a control dashboard, voice, agents, workflows, RAG, search, image generation, privacy tools, observability, and developer tools.
What does it run on?	Linux, Windows with WSL2/Docker Desktop, and macOS Apple Silicon.
Is cloud required?	No. Local mode is the default; cloud and hybrid API modes are optional.
If you know...	ODS adds...
---	---
Ollama / llama.cpp	The surrounding server stack: chat, dashboard, voice, RAG, workflows, agents, privacy, and service management.
Open WebUI	A full installer and control plane around Open WebUI, plus pre-wired local services.
AnythingLLM	Broader local AI appliance behavior beyond RAG: inference, chat, voice, workflows, image generation, and ops.
n8n self-hosted AI starter kits	Workflow automation as one part of a larger private AI server.

---

Current Platform Support

Platform	Status
Linux (NVIDIA + AMD + Intel Arc)	Supported — install and run today
Windows (NVIDIA + AMD)	Supported — install and run today
macOS (Apple Silicon)	Supported — install and run today

Tested Linux distros: Ubuntu 24.04/22.04, Debian 12, Linux Mint 21.3, Fedora 41+, Rocky Linux 9, Arch Linux, Manjaro, CachyOS, and openSUSE Tumbleweed. Other distros using apt, dnf, pacman, or zypper should also work — open an issue if yours doesn't.

Release validation: Operational changes run through a release-grade gate that covers zero-prereq bootstrap, clean installs, product behavior, full-model capabilities, lifecycle recovery, and User Green. See Release Validation and the Validation Matrix.

Windows: Requires Docker Desktop with WSL2 backend. NVIDIA GPUs use Docker GPU passthrough; AMD Strix Halo runs through the platform-specific accelerated path documented in the Windows installer and support matrix.

macOS: Requires Apple Silicon (M1+) and Docker Desktop. llama-server runs natively with Metal GPU acceleration; all other services run in Docker.

See the Support Matrix for supported platform claims and the Validation Matrix for the layered test surface used to test those claims.

---

A handful of companies control the vast majority of global AI traffic — and with it, your data, your costs, and your uptime. Every query you send to a centralized provider is business intelligence you don’t own, running on infrastructure you don’t control, priced on terms you can’t negotiate.

If AI is becoming critical infrastructure, it shouldn’t be rented. Self-hosting local AI should be a sovereign human right, not a career choice.

Because running your own AI shouldn't require a CS degree and a weekend of debugging CUDA drivers. Right now, setting up local AI means stitching together a dozen projects, writing Docker configs from scratch, and praying everything talks to each other. Most people give up and go back to paying OpenAI.

We built ODS so you don't have to.

• One command — detects your GPU, picks the right model, generates credentials, launches everything
• Chatting in under 2 minutes — bootstrap mode gives you a working model instantly while your full model downloads in the background
• Full service stack, pre-wired — chat, agents, voice, workflows, search, RAG, image generation, privacy tools, observability, and developer tools. All talking to each other out of the box
• Fully moddable — every service is an extension. Drop in a folder, run ods enable, done

Manual install (Linux)

git clone https://github.com/Osmantic/ODS.git
cd ODS/ods
./install.sh

Windows (PowerShell)

Requires Docker Desktop with WSL2 backend enabled. Install Docker Desktop first and make sure it is running before you start.

Open a normal PowerShell session and run:

$ProgressPreference = "SilentlyContinue"
$odsSrc = Join-Path $env:TEMP ("ods-install-" + [guid]::NewGuid().ToString("N"))
$odsZip = Join-Path $odsSrc "ods-main.zip"
New-Item -ItemType Directory -Path $odsSrc | Out-Null
Invoke-WebRequest "https://github.com/Osmantic/ODS/archive/refs/heads/main.zip" -OutFile $odsZip
Expand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force
cd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1

The Set-ExecutionPolicy command allows the installer script to run in the current session. It does not change your system-wide policy. Running as Administrator is not recommended for the installer because user-level paths such as .opencode, data/, and .env can be created with admin-owned permissions.

The installer detects your GPU, picks the right model, generates credentials, starts all services, and creates a Desktop shortcut to the Dashboard. Manage from the runtime directory with .\ods.ps1 status; uninstall with .\ods.ps1 uninstall --force.

macOS (Apple Silicon)

Requires Apple Silicon (M1+) and Docker Desktop. Install Docker Desktop first and make sure it is running before you start.

git clone https://github.com/Osmantic/ODS.git
cd ODS/ods
./install.sh

The installer detects your chip, picks the right model for your unified memory, launches llama-server natively with Metal acceleration, and starts all other services in Docker. Manage with ./ods-macos.sh status.

See the macOS Quickstart for details.

---

• Open WebUI — full-featured chat interface with conversation history, web search, document upload, and 30+ languages

• llama-server — high-performance LLM inference with continuous batching, auto-selected for your GPU; Linux Docker host API defaults to localhost:11434, native macOS/Windows paths use localhost:8080, and container API runs on 8080

• LiteLLM — API gateway supporting local/cloud/hybrid modes

• TEI Embeddings — text embedding service for RAG and search workflows

• Whisper — speech-to-text

• Kokoro — text-to-speech

• Hermes Agent — default local-first autonomous/browser agent with memory, skills, and a magic-link-gated proxy

• OpenClaw — deprecated legacy autonomous agent, still opt-in during the migration window

• n8n — workflow automation with 400+ integrations (Slack, email, databases, APIs)

• APE — Agent Policy Engine for auditing and governing autonomous tool calls

• OpenCode — browser-based AI coding assistant wired to the local stack

• Memory Shepherd — host/systemd helper for agent memory lifecycle management

• Qdrant — vector database for retrieval-augmented generation (RAG)

• SearXNG — self-hosted web search (no tracking)

• Perplexica — deep research engine

• Brave Search — optional paid Brave Search API integration

• ComfyUI — node-based image generation

• Privacy Shield — PII scrubbing proxy for API calls

• Dashboard — real-time GPU metrics, service health, model management

• Dashboard API — service health, setup, status, metrics, and management API behind the dashboard

• Token Spy — token usage monitor for local and proxied LLM traffic

• Langfuse — optional LLM observability and tracing

---

The installer detects your GPU and first assigns a deterministic hardware tier. Linux and macOS then run the versioned catalog selector (ods/scripts/select-model.py), while Windows uses the PowerShell catalog selector in ods/installers/windows/lib/tier-map.ps1; both read ods/config/model-library.json to choose the best installable GGUF for the detected memory envelope. The final choice is written to .env as LLM_MODEL, GGUF_FILE, MAX_CONTEXT, and MODEL_RECOMMENDATION_*.

MODEL_PROFILE=qwen is the default non-Gemma catalog profile, so the effective pick can be Qwen, Phi, or DeepSeek depending on what fits best. MODEL_PROFILE=gemma4 forces Gemma 4 where available, and MODEL_PROFILE=auto uses Gemma 4 on NVIDIA, Apple Silicon, and Intel Arc tiers. Override tier selection with ./install.sh --tier 3; override the model family with MODEL_PROFILE=gemma4 ./install.sh or MODEL_PROFILE=auto ./install.sh.

When Hermes is enabled, which is the default agent path, installers keep the first-run bootstrap model at a 64K context floor and promote the full local model context to 128K where the selected model supports it. That avoids Hermes's hard 64K minimum while preserving the under-2-minute first chat experience. The examples below are current catalog-selector outputs for common hardware envelopes; exact installs can differ with detected VRAM/RAM, host architecture, existing downloads, or explicit profile overrides. Throughput still needs a local benchmark after first launch.

Tier / envelope	Current default catalog pick	Context	Example hardware
0 / 8 GB CPU fallback	Qwen3.5 2B (Q4_K_M)	8K	Low-RAM CPU-only
1 / 8 GB discrete VRAM	Qwen3.5 9B (Q4_K_M)	32K	RTX 4060, RTX 3060 12GB
2 / 12 GB discrete VRAM	Phi-4 14B (Q4_K_M)	16K	RTX 4070-class cards
3 / 24 GB discrete VRAM	Qwen3.5 27B (Q4_K_M)	32K	RTX 4090, A6000
4 / 48 GB discrete VRAM	DeepSeek R1 Distill Llama 70B (Q4_K_M)	32K	A6000 Ada, L40S
NV_ULTRA / 90+ GB amd64 discrete VRAM	Qwen3 Coder Next (Q4_K_M)	128K	Multi-GPU A100/H100
NV_ULTRA / 90+ GB arm64 unified memory	Qwen3.6 35B-A3B (UD-Q4_K_M)	128K	DGX Spark / GB10-class hosts

AMD Strix Halo (Unified Memory)

Tier / envelope	Current default catalog pick	Context	Hardware
SH_COMPACT / 64 GB unified RAM	Qwen3.6 35B-A3B (UD-Q4_K_M)	128K	Ryzen AI MAX+ 395 (64GB)
SH_LARGE / 96 GB unified RAM	DeepSeek R1 Distill Llama 70B (Q4_K_M)	32K	Ryzen AI MAX+ 395 (96GB)
SH_LARGE / 124 GB unified RAM	Qwen3.6 35B-A3B (UD-Q4_K_M)	128K	Ryzen AI MAX+ 395 (128GB class)

The selector routes unified-memory hosts away from Qwen3 Coder Next when that model would otherwise be selected, because current repo policy documents correctness issues on those backends.

Apple Silicon (Unified Memory, Metal)

Tier / envelope	Current default catalog pick	Context	Example hardware
0 / 8 GB unified RAM	Phi-4 Mini (Q4_K_M)	128K	M1/M2 base (8GB)
1 / 16 GB unified RAM	Qwen3.5 9B (Q4_K_M)	32K	M4 Mac Mini (16GB)
2 / 32 GB unified RAM	Phi-4 14B (Q4_K_M)	16K	M4 Pro Mac Mini, M3 Max MacBook Pro
3 / 48 GB unified RAM	Qwen3.5 27B (Q4_K_M)	32K	M4 Pro (48GB), M2 Max (48GB)
4 / 64+ GB unified RAM	Qwen3.6 35B-A3B (UD-Q4_K_M)	128K	M2 Ultra Mac Studio, M4 Max (64GB+)
Tier / envelope	Current default catalog pick	Context	Example hardware
---	---	---	---
ARC_LITE / 6 GB discrete VRAM	Phi-4 Mini (Q4_K_M)	128K	Arc A380
ARC_LITE / 8 GB discrete VRAM	Qwen3.5 9B (Q4_K_M)	32K	Arc A750
ARC / 16 GB discrete VRAM	Phi-4 14B (Q4_K_M)	16K	Arc A770 16GB, newer Arc GPUs

Gemma 4 profile tiers remain in the installer tier maps: E2B on entry hardware, E4B on midrange hardware, 26B-A4B on pro hardware, and 31B on large/ultra hardware.

---

No waiting for large downloads. ODS uses bootstrap mode by default:

1. Downloads a tiny 1.5B model in under a minute
2. You start chatting immediately
3. The full model downloads in the background
4. Hot-swap to the full model when it's ready — zero downtime

The bootstrap model starts with a 64K context window so Hermes can work during the first session. After the background download finishes, ODS swaps to the full model and restores the Hermes/full-model context target.

Skip bootstrap: ./install.sh --no-bootstrap

---

The installer picks a model for your hardware, but you can switch anytime:

ods model current              # What's running now?
ods model list                 # Show all available tiers
ods model swap T3              # Switch to a different tier

If the new model isn't downloaded yet, pre-fetch it first:

./scripts/pre-download.sh --tier 3    # Download before switching
ods model swap T3                    # Then swap (restarts llama-server)

Already have a GGUF you want to use? Drop the single .gguf file in data/models/, then open Dashboard -> Models and load the local entry. For older installs or headless maintenance, update GGUF_FILE and LLM_MODEL in .env, then restart with the CLI:

Or restart the container directly from the installed ods directory:

docker compose restart llama-server

Rollback is automatic — if a new model fails to load, ODS reverts to your previous model.

---

ODS is designed to be modded. Every service is an extension — a folder with a manifest.yaml and a compose.yaml. The dashboard, CLI, health checks, and compose stack all discover extensions automatically.

extensions/services/
  my-service/
    manifest.yaml      # Metadata: name, port, health endpoint, GPU backends
    compose.yaml       # Docker Compose fragment (auto-merged into the stack)



ods enable my-service     # Enable it
ods disable my-service    # Disable it
ods list                  # See everything

The installer itself is modular — 19 library modules, a shared service registry, and 13 ordered phases. Want to add a hardware tier, swap a default model, or skip a phase? Start with the installer architecture map so you update the Linux, macOS, Windows, upgrade, and host-agent writers together.

Full extension guide | Installer architecture

---

The ods CLI manages your entire stack:

ods status                # Health checks + GPU status
ods list                  # All services and their state
ods logs llm              # Tail logs (aliases: llm, stt, tts)
ods restart [service]     # Restart one or all services
ods start / stop          # Start or stop the stack

ods mode cloud            # Switch to cloud APIs via LiteLLM
ods mode local            # Switch back to local inference
ods mode hybrid           # Local primary, cloud fallback

ods model swap T3         # Switch to a different hardware tier
ods enable n8n            # Enable an extension
ods disable whisper       # Disable one

ods config show           # View .env (secrets masked)
ods preset save gaming    # Snapshot current config
ods preset load gaming    # Restore it

---

Other tools get you part of the way. ODS gets you the whole way.

| ODS | Ollama + Open WebUI | LocalAI
---|---|---|---
Scope | Full AI stack — inference to agents to workflows | LLM + chat | LLM only
One-command install | Everything, auto-configured | LLM + chat only | LLM only
Hardware auto-detect + model selection | NVIDIA + AMD Strix Halo + Apple Silicon + Intel Arc + CPU/cloud fallback | No | No
AMD APU unified memory support | Platform-specific accelerated backend, selected by installer | Partial (Vulkan) | No
Autonomous AI agents | Hermes Agent default; OpenClaw legacy opt-in | No | No
Workflow automation | n8n (400+ integrations) | No | No
Voice (STT + TTS) | Whisper + Kokoro | No | No
Image generation | ComfyUI | No | No
RAG pipeline | Qdrant + embeddings | No | No
Extension system | Manifest-based, hot-pluggable | No | No
Multi-GPU | Yes (NVIDIA) | Partial | Partial

---

|
---|---
Quickstart | Step-by-step install guide with troubleshooting
Docs Index | Maintained map for operators, contributors, and reviewers
Build On ODS | Forking, custom editions, extension templates, and downstream validation
Forkability | How to fork, audit, customize, and independently operate ODS
Maintainer Runbook | Release, rollback, validation, and operator continuity guidance for maintainers and forks
High-Risk Change Map | Which changes require focused checks, fleet validation, or release-grade gates
Headless Setup | QR onboarding, first-boot setup, AP mode, mDNS, and local agent access
Support Matrix | Current platform and GPU support status
Release Validation | User Green gates and the release-grade fleet/distro validation policy
2.6.0 Release Notes | Current stable release notes, validation receipt, and known validation boundaries
Validation Matrix | Sanitized CI, distro lab, and real-hardware fleet release-readiness evidence
Validation Reproducibility | How forks and operators can reproduce the validation story on their own hardware
Offline And Mirroring | Pinning, mirroring, and preserving release artifacts for independent operation
Installer Trust | Inspect-first install paths, ref pinning, and current provenance limits
Model Management | Curated and Hugging Face GGUF discovery, verified imports, switching, and recovery
Hardware Guide | What to buy, tier recommendations
FAQ | Common questions and configuration
Extensions | How to add custom services
Installer Architecture | Modular installer deep dive
Installer Phase Contracts | Phase ownership, idempotency, failure modes, and validation expectations
Compose Resolver Contracts | Rules for compose layers, extensions, backends, ports, and mode overlays
Changelog | Version history and release notes
Contributing | How to contribute

---

Contributors And Recognition

ODS is built by a growing group of contributors across installers, GPU support, dashboard, security, extensions, docs, and release validation. The README keeps the product overview focused; the long-form credits, upstream acknowledgements, and contributor history live in CONTRIBUTORS.md.

ODS has been recognized by the local AI and developer community, including AMD Featured Developer recognition, selection as a May 2026 AMD Lemonade Developer Challenge winner, and a feature at (Co)nnect: Philly's AI Ecosystem Summit at Pennovation Works.

---

Apache 2.0 — Use it, modify it, ship it. See LICENSE.

---

Built byOsmantic and the growing resistance that refuses to rent what should be owned.