AI News
开源项目 GitHub Topic mcp 2026-07-06T12:02 正文 ok 1524 words

LocalAI：在任何硬件上运行任何模型的开源 AI 引擎

英文标题：📦 mudler/LocalAI

LocalAI 是一个开源 AI 引擎，支持 LLM、视觉、语音、图像、视频等多种模态，无需 GPU 即可运行。其核心采用可组合架构，按需拉取 llama.cpp、vLLM、whisper.cpp 等后端，实现轻量部署。该工具提供与 OpenAI、Anthropic、ElevenLabs 等 API 的兼容性，内置智能体、多用户管理与隐私保护，对本地开发与隐私优先的场景极具价值。
打开原文相关 5.6 · 机会 7
media893.3 KB media789 B media590 B media595 B media1.1 KB media2.5 KB media5.1 KB media303.9 KB media1.5 KB media51.9 KB media309.6 KB media62.8 KB media396.8 KB media112.9 KB

中文

LocalAI star 数 LocalAI 许可证

关注 LocalAI_API 加入 LocalAI Discord 社区

mudler%2FLocalAI | Trendshift

Deutsch | Español | français | 日本語 | 한국어 | Português | Русский | 中文

LocalAI 是开源的 AI 引擎。在任何硬件上运行任何模型——LLM、视觉、语音、图像、视频。无需 GPU。

小而精的核心，而非捆绑包。 每个后端在各自的镜像中封装了同类最佳的引擎（llama.cpp、vLLM、whisper.cpp、stable-diffusion、MLX……），仅在模型需要时才拉取。你不用的东西绝不会被安装。

  • 可组合式设计：后端彼此独立，按需拉取，因此你只安装模型所需的部分
  • 开放且可扩展：加载任意模型，或基于开放接口用任何语言构建自己的后端
  • 直接替换的 API 兼容：所有后端均兼容 OpenAI、Anthropic 和 ElevenLabs API
  • 任意模型，任意模态：LLM、视觉、语音、图像和视频统一到一个 API 后
  • 任意硬件：NVIDIA、AMD、Intel、Apple Silicon、Vulkan 或纯 CPU
  • 多用户就绪：API 密钥认证、用户配额、基于角色的访问控制
  • 内置 AI 智能体：具备工具使用、RAG、MCP 和技能的自主智能体
  • 隐私优先：你的数据永远不会离开你的基础设施

一个小型的 LocalAI 核心，其后端（llama.cpp、vLLM、MLX、whisper.cpp、stable-diffusion、kokoro、parakeet.cpp……）以独立按需镜像的形式接入

由 Ettore Di Giacinto 创建，并由 LocalAI 团队维护。

📖 文档 | 💬 Discord | 💻 快速开始 | 🖼️ 模型 | ❓ 常见问题

模型适配画布模式 点击查看更多！ 用户配额 智能体 用量

微调与量化

量化微调 交流 下载 LocalAI for macOS

注意： 该 DMG 未经 Apple 签名。安装后，请运行：sudo xattr -d com.apple.quarantine /Applications/LocalAI.app。详见 #6268。

容器 (Docker, podman, ...)

之前运行过 LocalAI？使用 docker start -i local-ai 重新启动现有容器。

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest


# CUDA 13
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13

# CUDA 12
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

# NVIDIA Jetson ARM64 (CUDA 12，适用于 AGX Orin 及类似设备)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64

# NVIDIA Jetson ARM64 (CUDA 13，适用于 DGX Spark)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13


docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas


docker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel


docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan


# 从模型库中（使用 `local-ai models list` 或访问 https://models.localai.io 查看可用模型）
local-ai run llama-3.2-1b-instruct:q4_k_m
# 从 Huggingface
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# 从 Ollama OCI 注册表
local-ai run ollama://gemma:2b
# 从 YAML 配置
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
# 从标准 OCI 注册表（例如 Docker Hub）
local-ai run oci://localai/phi-2:latest

若要从终端测试正在运行的 LocalAI 服务器，可从另一个 shell 打开交互式聊天会话。在提示符中，/models 列出已安装的模型，/model <name> 在它们之间切换。

# 终端 1
local-ai run llama-3.2-1b-instruct:q4_k_m

# 终端 2
local-ai chat --model llama-3.2-1b-instruct:q4_k_m

自动后端检测：LocalAI 会自动检测你的 GPU 能力并下载合适的后端。高级选项请参考 GPU 加速。

更多细节请参阅 入门指南。

更早的新闻和完整发布说明请参见 GitHub Releases 和 新闻页面。

支持的后端与加速

LocalAI 支持 60+ 后端，包括

English / 原文

LocalAI stars LocalAI License

Follow LocalAI_API Join LocalAI Discord Community

mudler%2FLocalAI | Trendshift

Deutsch | Español | français | 日本語 | 한국어 | Português | Русский | 中文

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

A small core, not a bundle. Each backend wraps a best-in-class engine (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX...) in its own image, pulled only when a model needs it. You install nothing you don't use.

  • Composable by design : backends are separate and pulled on demand, so you install only what your model needs
  • Open and extensible : load any model, or build your own backend in any language against an open interface
  • Drop-in API compatibility : OpenAI, Anthropic, and ElevenLabs APIs across every backend
  • Any model, any modality : LLMs, vision, voice, image, and video behind one API
  • Any hardware : NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only
  • Multi-user ready : API key auth, user quotas, role-based access
  • Built-in AI agents : autonomous agents with tool use, RAG, MCP, and skills
  • Privacy-first : your data never leaves your infrastructure

A small LocalAI core with backends (llama.cpp, vLLM, MLX, whisper.cpp, stable-diffusion, kokoro, parakeet.cpp...) plugged in as separate on-demand images

Created by Ettore Di Giacinto and maintained by the LocalAI team.

📖 Documentation | 💬 Discord | 💻 Quickstart | 🖼️ Models | ❓FAQ

model-fit-canvas-mode.mp4 Click to see more! usersquota-1775167475876.mp4 agents.mp4 usage.mp4

Fine-tuning and Quantization

quantize-fine-tune.mp4 talk.mp4 Download LocalAI for macOS

Note: The DMG is not signed by Apple. After installing, run: sudo xattr -d com.apple.quarantine /Applications/LocalAI.app. See #6268 for details.

Containers (Docker, podman, ...)

Already ran LocalAI before? Use docker start -i local-ai to restart an existing container.

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest


# CUDA 13
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13

# CUDA 12
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

# NVIDIA Jetson ARM64 (CUDA 12, for AGX Orin and similar)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64

# NVIDIA Jetson ARM64 (CUDA 13, for DGX Spark)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13


docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas


docker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel


docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan


# From the model gallery (see available models with `local-ai models list` or at https://models.localai.io)
local-ai run llama-3.2-1b-instruct:q4_k_m
# From Huggingface
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# From the Ollama OCI registry
local-ai run ollama://gemma:2b
# From a YAML config
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
# From a standard OCI registry (e.g., Docker Hub)
local-ai run oci://localai/phi-2:latest

To test a running LocalAI server from the terminal, open an interactive chat session from another shell. Inside the prompt, /models lists installed models and /model <name> switches between them.

# Terminal 1
local-ai run llama-3.2-1b-instruct:q4_k_m

# Terminal 2
local-ai chat --model llama-3.2-1b-instruct:q4_k_m

Automatic Backend Detection : LocalAI automatically detects your GPU capabilities and downloads the appropriate backend. For advanced options, see GPU Acceleration.

For more details, see the Getting Started guide.

For older news and full release notes, see GitHub Releases and the News page.

Supported Backends & Acceleration

LocalAI supports 60+ backends including llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is available for NVIDIA (CUDA 12/13), AMD (ROCm), Intel (oneAPI/SYCL), Apple Silicon (Metal), Vulkan , and NVIDIA Jetson (L4T). All backends can be installed on-the-fly from the Backend Gallery.

See the full Backend & Model Compatibility Table and GPU Acceleration guide.

Most backends wrap a best-in-class upstream engine. A handful of them are native C/C++/GGML engines (no Python at inference) developed and maintained by the LocalAI project itself:

Backend What it does
parakeet.cpp C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription
ced.cpp C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition
voxtral.c Voxtral Realtime 4B speech-to-text in pure C
vibevoice.cpp Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization
rf-detr.cpp Native RF-DETR object detection and instance segmentation
locate-anything.cpp Open-vocabulary object detection and visual grounding (LocateAnything-3B)
depth-anything.cpp Depth Anything 3 monocular metric depth + camera pose estimation
privacy-filter.cpp Standalone GGML PII/NER token-classification engine powering LocalAI's PII redaction tier
LocalVQE Joint acoustic echo cancellation, noise suppression, and dereverberation
local-store Local-first vector database for embeddings (shipped in-tree)

We also maintain apex-quant, a per-tensor, per-layer quantization recipe for Mixture-of-Experts models that exploits their structural sparsity to produce GGUFs matching or beating Q8_0 quality - and they run out of the box on stock llama.cpp.

LocalAI is maintained by a small team of humans, together with the wider community of contributors.

A huge thank you to everyone who contributes code, reviews PRs, files issues, and helps users in Discord — LocalAI is a community-driven project and wouldn't exist without you. See the full contributors list.

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: The free, Open source OpenAI alternative},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/go-skynet/LocalAI}},

Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project covering CI expenses, and our Sponsor list:

Past sponsors


Individual sponsors

A special thanks to individual sponsors, a full list is on GitHub and buymeacoffee. Special shout out to drikster80 for being generous. Thank you everyone!

LocalAI Star history Chart

LocalAI is a community-driven project created by Ettore Di Giacinto and maintained by the LocalAI team.

MIT - Author Ettore Di Giacinto mudler@localai.io

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

This is a community project, a special thanks to our contributors!