借助 LiteRT 和 Gemma 掌握 Raspberry Pi 边缘 AI

设想一下：在 Raspberry Pi 这样的单一紧凑设备上，构建一台能够实时看见、听见并对周围环境作出反应，同时完全离线运行的全自主机器人。边缘 AI 正在实现这种自主能力。它让开发者能够构建高度安全、自包含的系统，例如智能机器人和本地 AI Agent：无需依赖云端，具备超低延迟，并能确保数据完全隐私。

借助 Google AI Edge 的 LiteRT——一款高性能、经过生产环境验证的端侧推理运行时——我们让在 Raspberry Pi 上运行边缘 AI 变得轻而易举。LiteRT 可让你开箱即用地在多种平台上无缝部署从传统机器学习模型到先进 LLM 的各类模型。通过在 CPU 和 GPU 上提供优化执行与高效内存使用，LiteRT 能够最大限度释放 Raspberry Pi 的完整计算潜力。

[视频/嵌入内容：浏览器不支持播放；Reachy Mini 在 Raspberry Pi 5 上由 Gemma 和 LiteRT 驱动，通过回答和动作作出反应。]

当 LiteRT 与 Google 的轻量级开放模型家族 Gemma 搭配使用时，这种软硬件协同的优势最为明显。为展示其可能性，我们将演示 Raspberry Pi 5 上的 Gemma 和 LiteRT 如何驱动 Reachy Mini 机器人，完全在本地实时感知环境并作出反应。继续阅读，开始部署你自己的方案。

探索 Gemma 的 Agent 能力

Gemma 模型非常适合构建可直接在 Raspberry Pi 上进行推理并执行复杂、多步骤工作流的自主 Agent、智能摄像头和社交机器人。为适应不同的硬件约束，Gemma 模型家族提供了多种高效选项：

• Gemma 3 270M：一款超高效、紧凑的基础模型，专为任务特定的后训练微调而设计，可在资源受限环境中实现情感分析、实体提取等高速、低延迟功能。
• EmbeddingGemma 300M：一款文本嵌入模型，可在设备端生成高质量嵌入，适用于检索增强生成（RAG）、语义搜索和分类。
• Gemma 3 1B：一款轻量级多语言纯文本模型，在紧凑体积与强大的生成能力之间取得平衡，适用于摘要、内容创作等广泛的端侧任务。
• Gemma 4 E2B：专为移动端和资源紧张的边缘环境打造，具有按层内存映射嵌入功能，适用于持续监测、快速文本/图像/音频推理，以及内存节省至关重要的边缘语音处理。
• Gemma 4 E4B：在性能与体积之间的理想平衡点。该模型在保持紧凑的同时，提供明显更强的推理能力和前沿级边缘性能；对于不会压垮 Raspberry Pi 资源的复杂多步骤规划而言，它是理想选择。

Gemma 在 Raspberry Pi CPU 上的性能

通过构建在 LiteRT 之上的专用编排层 LiteRT-LM，开发者可以开箱即用地无缝部署 Gemma。在底层，LiteRT 和 XNNPACK 提供了复杂的 CPU 加速能力，确保 Gemma 模型家族针对资源效率和 Raspberry Pi 端的低延迟执行进行了高度优化。

在 Raspberry Pi 5 上，LiteRT-LM 为 Gemma 4 E2B 提供了稳定的性能：预填充达到 99 tokens/sec，解码达到 9 tokens/sec，同时峰值内存占用仅为 1432 MB。这将 Gemma 高响应、通用的智能能力带到了 Raspberry Pi 上。

得益于 Gemma 4 E2B 的高效 tokenizer——它以更少的 token 容纳更多文本，平均每个 token 约为 4.2 个字符——LiteRT-LM 在 Reachy Mini 语音演示中实现了约 27.3 字符/秒的出色端到端生成速度，约合每分钟 300 个单词（wpm）。这一吞吐量使 Gemma 4 E2B 非常适合实时语音和翻译任务，其文本输出速度约为正常人类语速（约 150 wpm）的两倍。

从 LiteRT Hugging Face Community 探索更多可直接在 Raspberry Pi 上运行的现成开放模型。

使用 LiteRT 在 Raspberry Pi GPU 上执行推理

在 Raspberry Pi 5 上，四核 ARM Cortex-A76 CPU 是一台原始算力强大的计算引擎，可提供约 153.6 GFLOPS（FP32）和最高约 2.0 TOPS（INT8）。相比之下，集成的 Broadcom VideoCore VII GPU 主频为 800 MHz，峰值性能约为 76.8 GFLOPS（FP32）和约 0.24 TOPS（INT8）。

尽管 CPU 拥有显著的算力优势，GPU 引入了异构并行执行这一对实时边缘应用至关重要的范式。开发者不必让 CPU 饱和运行，而可以将任务分配到两种处理器上，以优化整体系统和散热效率。例如，将持续运行的视觉或音频模型卸载至 VideoCore VII GPU，可以为整体系统监控、流水线编排或计算密集型 LLM 推理保留高优先级 CPU 周期。

因此，我们通过 ML Drift 启用了 LiteRT 的 WebGPU（Vulkan）后端，以支持 Raspberry Pi 上的 GPU 推理。该集成让你可以直接从 LiteRT Hugging Face Community 运行大量计算机视觉、音频和嵌入模型，并可无缝支持热门的 MediaPipe 模型、Ultralytics YOLO 模型、Moonshine 等更多模型。

[视频/嵌入内容：浏览器不支持播放；Ultralytics YOLO26n 模型通过 LiteRT 在 Raspberry Pi 5 上进行实时目标检测。可从 YOLO 指南获取示例代码开始使用。]

下表展示了通过 LiteRT 运行经典计算机视觉和音频模型时的 CPU 与 GPU 延迟：

深入解析：由 LiteRT 驱动的 Reachy Mini 流水线

Reachy Mini 流水线有力展示了完全运行在 Raspberry Pi 5 上的低延迟、实时边缘 AI 推理。借助 LiteRT，该系统将高负载视觉和语言工作负载拆分为在 CPU 与 GPU 之间并发运行的双处理架构。

以下是该并行架构在底层如何工作，以确保流畅交互：

• 目标检测（GPU 上的 Ultralytics YOLO）：摄像头帧被流式传输至 Raspberry Pi，Ultralytics YOLO 检测在 GPU 上持续运行，避免资源争用并释放 CPU。
• 语音识别（CPU 上的 Moonshine）：当用户说话时，ASR 组件会直接在 CPU 上将音频转录为文本。
• 推理与动作（CPU 上的 Gemma 4 E2B）：Gemma 4 E2B 模型处理生成的转录文本和最新视觉元数据，以生成低延迟流式响应，例如语音回复和机器人物理动作。
• 文本转语音（CPU 上的 TTS）：TTS 组件以流式方式将生成的文本合成为音频。系统会将合成的语音流式传回 Reachy Mini 机器人。

在 LiteRT Samples GitHub repo 中查看 Reachy 演示的完整源代码。

在 Raspberry Pi 上使用 LiteRT 进行 Agent 编程

LiteRT 提供覆盖完整开发周期的综合工具套件：转换、量化、基准测试和推理。若要快速、顺畅地完成设置，最直接的方法是使用 LiteRT CLI 工具。LiteRT CLI 无需开发者或编程 Agent 手动管理多个独立库，而是将核心边缘工作流整合为一套统一的命令集。

[视频/嵌入内容：浏览器不支持播放；使用 LiteRT CLI 简化开发周期：转换、量化、基准测试和推理。]

现在，你可以将 LiteRT CLI skill 及其他高级 LiteRT skills添加到你的 AI 编程 Agent 中，例如 Google Antigravity。这让 Agent 能代表你自主编排和执行复杂的多阶段机器学习工作流。例如，你可以轻松在 Raspberry Pi 上完全离线地构建自己的语音翻译器，如下方展示的 Gemma Translator。

在 Gemma Translator GitHub repo 中探索完整的实现细节。

对于资源受限的 IoT 设备，尽可能降低存储与内存开销至关重要。若不进行专门优化，通用 AI 运行时通常会捆绑沉重的桌面端或服务器端依赖。相比之下，LiteRT 专为设备端部署设计，保持了极其精简且模块化的发行包。

下表比较了在 Raspberry Pi（ARM64 Linux）上运行 LLM 推理所需的下载体积：

运行你的第一个模型

只需几条简单命令，你就可以在 Raspberry Pi 5 上安装 LiteRT CLI 并运行第一个模型。

1. 安装 LiteRT CLI

要开始使用，请通过 pip 安装 LiteRT CLI（最好在虚拟环境中进行）：

shell pip install litert-cli

Shell

2. 运行模型

直接从 LiteRT Hugging Face Community 下载并运行任意兼容模型。以下代码片段演示如何在 Raspberry Pi 5 上执行 Gemma 4 E2B（例如 gemma-4-E2B-it-litert-lm）。

通过提供你的 Hugging Face 认证 token 来运行模型：

shell export HUGGING_FACE_HUB_TOKEN=<your_hugging_face_token_here> litert lm run
--from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm
gemma-4-E2B-it.litertlm
--attachment=image.jpg
--prompt="You are Reachy Mini. Identify the main object in front of you, "
"state its location (Left/Right/Center), and suggest head action in "
"10 words or less."

Shell

下一步计划

我们很高兴地宣布，LiteRT 集成和 Gemma 模型即将支持 Hailo AI accelerators！此次更新将让你通过当前熟悉的同一套 LiteRT 工作流，将模型推理无缝卸载至 Raspberry Pi AI HAT+ 和 AI HAT+ 2，并获得巨大的硬件加速优势。

探索我们的资源，开启你的 LiteRT 之旅：

我们重视你的意见。请通过我们的 GitHub Issue Tracker 创建 issue，分享你的想法、反馈或功能请求。也欢迎通过 @googlegemma 分享你出色的 Raspberry Pi + LiteRT + Gemma 项目。我们迫不及待想看到你的作品！

致谢

Google：Changming Sun、Chintan Parikh、Cormac Brick、Dillon Sharlet、Daisuke Majima、Erin Walsh、Frank Barchard、Glenn Cameron、Ian Ballantyne、Jingjiang Li、Jun Jiang、Kimish Patel、Lu Wang、Matthias Grundmann、Rodney Witcher、Sachin Kotwani、Sasha Denisov、Scott Loftin、Shuangfeng Li、Somdatta Banerjee、Terry (Woncheol) Heo、Volodymyr Kysenko、Weiyi Wang、Yi-Chun Kuo、Yu-hui Chen，以及 gtech 团队

Raspberry Pi 与 Hailo：Ashley Whittaker、Eldad Rubinstein、José María Casanova (Igalia)、Naushir Patuck 和 Sarah Cunningham

Ultralytics：Francesco Mattioli、Onuralp Sezer、Lakshantha Dissanayake

Moonshine AI：Pete Warden

Imagine building a fully autonomous robot that can see, hear, and react to its environment in real time, completely offline on a single compact device like Raspberry Pi. Edge AI unlocks this exact autonomy. It enables developers to build highly secure and self-contained systems like intelligent robotics and local AI agents with zero cloud dependencies, ultra-low latency, and total data privacy.

We’ve made running edge AI on the Raspberry Pi a breeze with Google AI Edge’s LiteRT, high-performance production-proven on-device inference runtime. LiteRT allows you to seamlessly deploy everything from classical ML models to state-of-the-art LLMs right out of the box across platforms. By delivering optimized execution and hyper-efficient memory usage on both CPU and GPU, LiteRT maximizes your Raspberry Pi's full computing potential.

Sorry, your browser doesn't support playback for this video

Reachy Mini reacts with responses and movement, powered by Gemma and LiteRT on Raspberry Pi 5.

This hardware-software synergy shines brightest when paired with Gemma, Google’s family of lightweight open models. To show you what’s possible, we'll demonstrate how Gemma and LiteRT on a Raspberry Pi 5 can power the Reachy Mini robot to perceive and react to its environment entirely locally in real time. Read on to get started with your own deployment.

Discover the Agentic Capability of Gemma

Gemma models are well suited for building autonomous agents, smart cameras, and social robotics that can reason and execute complex, multi-step workflows directly on your Raspberry Pi. To accommodate different hardware constraints, the Gemma family of models provides several highly efficient options:

• Gemma 3 270M: A hyper-efficient and compact base model designed for task-specific post fine-tuning, enabling high-speed, low-latency features like sentiment analysis or entity extraction in resource-constrained environments.
• EmbeddingGemma 300M: A text embedding model that produces high-quality embeddings on-device, great for Retrieval Augmented Generation (RAG), semantic search, and classification.
• Gemma 3 1B: A lightweight and multilingual text-only model that balances compact size with strong generative capabilities, making it ideal for a wide range of on-device tasks, such as summarization and content creation.
• Gemma 4 E2B: Tailored specifically for mobile and tight edge environments, it features memory-mapped per-layer embeddings, and is ideal for continuous monitoring, fast text/image/audio inference, and edge-based speech processing where saving RAM is absolutely critical.
• Gemma 4 E4B: The sweet spot for performance and size. This model delivers noticeably stronger reasoning capabilities and frontier-level edge performance while remaining compact. It is the perfect choice for complex multi-step planning without overwhelming the Pi's resources.

Gemma Performance on Raspberry Pi CPU

Through LiteRT-LM, a specialized orchestration layer on top of LiteRT, developers can seamlessly deploy Gemma right out of the box. Under the hood, sophisticated CPU acceleration via LiteRT and XNNPACK ensures the Gemma family of models is highly optimized for resource efficiency and low-latency execution directly on the Raspberry Pi.

On a Raspberry Pi 5, LiteRT-LM delivers a robust performance for Gemma 4 E2B, achieving 99 tokens/sec for prefill and 9 tokens/sec for decode, all while maintaining a remarkably low peak memory footprint of just 1432 MB. This brings Gemma’s highly responsive, general-purpose intelligence to Raspberry Pi.

Thanks to Gemma 4 E2B's highly efficient tokenizer, which packs more text into fewer tokens (averaging ~4.2 characters per token), LiteRT-LM achieves an impressive end-to-end generation speed of ~27.3 characters per sec, roughly 300 words per minute (wpm), in the Reachy Mini voice demo. This throughput makes Gemma 4 E2B excellent for real-time speech and translation tasks , delivering text at twice the speed of normal human speech (~150 wpm).

Explore more ready-to-use open models to run on Raspberry Pi from the LiteRT Hugging Face Community.

Execute on Raspberry Pi GPU with LiteRT

On the Raspberry Pi 5, the quad-core ARM Cortex-A76 CPU is a raw computing powerhouse, delivering ~153.6 GFLOPS (FP32) and up to ~2.0 TOPS (INT8). In comparison, the integrated Broadcom VideoCore VII GPU is clocked at 800 MHz and offers a peak of ~76.8 GFLOPS (FP32) and ~0.24 TOPS (INT8).

While the CPU possesses a massive capacity advantage, the GPU introduces heterogeneous parallel execution , a paradigm critical for real-time edge applications. Rather than saturating the CPU, developers can delegate tasks across both processors to optimize overall system and thermal efficiency. For example, by offloading continuous vision or audio models to the VideoCore VII GPU, it preserves high-priority CPU cycles for overall system monitoring, pipeline orchestration, or computationally demanding LLM inference.

As such, we have enabled GPU inference on the Raspberry Pi withLiteRT’s WebGPU (Vulkan) backend via ML Drift. This integration allows you to run a wide range of computer vision, audio, and embedding models directly from the LiteRT Hugging Face Community, including seamless support for popular MediaPipe models, Ultralytics YOLO models, Moonshine, and much more.

Sorry, your browser doesn't support playback for this video

Real-time object detection with the Ultralytics YOLO26n model running on Raspberry Pi 5 with LiteRT. Get started with the sample code from YOLO guide.

The table below demonstrates the CPU and GPU latency of running classic computer vision and audio models via LiteRT:

Deep Dive: Reachy Mini Pipeline Powered by LiteRT

The Reachy Mini pipeline is a powerful showcase of low-latency, real-time edge AI inference running entirely on the Raspberry Pi 5. By leveraging LiteRT, the system splits intensive vision and language workloads into a concurrent, dual-processing architecture across CPU and GPU.

Here is how the parallel architecture works under the hood to ensure seamless interactions:

• Object Detection (Ultralytics YOLO on GPU) : Camera frames are streamed to the Pi, where a Ultralytics YOLO detection runs continuously on the GPU, avoiding resource contention and frees up the CPU.
• Speech Recognition (Moonshine on CPU) : When the user speaks, the ASR component transcribes the audio into text directly on the CPU.
• Reasoning & Action (Gemma 4 E2B on CPU): The Gemma 4 E2B model processes the resulting transcript alongside the latest visual metadata to generate low-latency, streaming responses, such as speech replies and physical robotic gestures.
• Text-to-Speech (TTS on CPU) : The TTS component synthesizes the generated text into audio in streaming. The system streams the synthesized voice back to the Reachy Mini robot.

See the full source code of the Reachy demo in the LiteRT Samples Github repo.

Agentic Coding with LiteRT on Raspberry Pi

LiteRT provides a comprehensive suite of tools that covers the full development cycle: conversion, quantization, benchmark, and inference. For a fast, frictionless setup, the most straightforward approach is using the LiteRT CLI tool. Rather than requiring developers or coding agents to manually manage multiple independent libraries, the LiteRT CLI aggregates core edge workflows into a single, unified command set.

Sorry, your browser doesn't support playback for this video

Streamline your development cycle with LiteRT CLI: convert, quantize, benchmark, and inference.

You can now supercharge your development cycle by adding the LiteRT CLI skill and other advanced LiteRT skills into your AI coding agent, such as Google Antigravity. This empowers agents to autonomously orchestrate and execute complex, multi-stage machine learning workflows on your behalf. For example, you can easily build your own voice translator completely offline on a Raspberry Pi, like the Gemma Translator shown below.

Explore the complete implementation details in the Gemma Translator GitHub repo.

For resource-constrained IoT devices, minimizing storage and memory overhead is critical. Without special optimization, generic AI runtimes often bundle heavy desktop or server dependencies. In contrast, LiteRT is engineered specifically for on-device deployment, maintaining an exceptionally lean and modular distribution.

The table below compares the download footprint required to run LLM inference on a Raspberry Pi (ARM64 Linux).

Running Your First Model

You can install the LiteRT CLI and run your first model on a Raspberry Pi 5 with just a few simple commands.

1.Install LiteRT CLI

To get started, install the LiteRT CLI via pip (ideally within a virtual environment):

pip install litert-cli

Shell

2. Run the model

Download and run any compatible model directly from the LiteRT Hugging Face Community. The code snippet below demonstrates how to execute Gemma 4 E2B (e.g. gemma-4-E2B-it-litert-lm) on the Raspberry Pi 5.

Run the model by providing your Hugging Face authentication token:

export HUGGING_FACE_HUB_TOKEN=<your_hugging_face_token_here>
litert lm run \
  --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm \
  gemma-4-E2B-it.litertlm \
  --attachment=image.jpg \
  --prompt="You are Reachy Mini. Identify the main object in front of you, "\
    "state its location (Left/Right/Center), and suggest head action in "\
    "10 words or less."

Shell

What’s Next

We are excited to share that LiteRT integration and Gemma models are coming soon to Hailo AI accelerators! This update will allow you to seamlessly offload model inference to the Raspberry Pi AI HAT+ and AI HAT+ 2, delivering massive hardware acceleration benefits through the exact same, familiar LiteRT workflows you use today.

Explore our resources and start your journey with LiteRT:

We value your input. Please share your thoughts, feedback, or feature requests by opening an issue on our GitHub Issue Tracker. Share your cool Raspberry Pi + LiteRT + Gemma projects with @googlegemma. We can't wait to see what you build!

Acknowledgements

Google : Changming Sun, Chintan Parikh, Cormac Brick, Dillon Sharlet, Daisuke Majima, Erin Walsh, Frank Barchard, Glenn Cameron, Ian Ballantyne, Jingjiang Li, Jun Jiang, Kimish Patel, Lu Wang, Matthias Grundmann, Rodney Witcher, Sachin Kotwani, Sasha Denisov, Scott Loftin, Shuangfeng Li, Somdatta Banerjee, Terry (Woncheol) Heo, Volodymyr Kysenko, Weiyi Wang, Yi-Chun Kuo, Yu-hui Chen, and gtech team

Raspberry Pi & Hailo: Ashley Whittaker, Eldad Rubinstein, José María Casanova (Igalia), Naushir Patuck, and Sarah Cunningham

Ultralytics : Francesco Mattioli, Onuralp Sezer, Lakshantha Dissanayake

Moonshine AI : Pete Warden