🔥 huggingface/speech-to-speech：用开源模型构建本地语音智能体

一个低延迟、完全模块化的语音智能体流水线：VAD -> STT -> LLM -> TTS，通过兼容 OpenAI Realtime 的 WebSocket API 暴露。每一个组件都可以替换。LLM 插槽支持 OpenAI 兼容协议，因此你可以将它指向托管供应商、HF Inference Providers，或者指向本地硬件上的 vLLM 或 llama.cpp 服务器，构建一个完全本地、完全开源的栈。

该流水线已在生产环境中作为数千台 Reachy Mini 机器人的对话后端运行。

pip install speech-to-speech
export OPENAI_API_KEY=...
speech-to-speech

这将在 ws://localhost:8765/v1/realtime 启动一个兼容 OpenAI Realtime 的服务器，使用 Parakeet TDT 进行本地 STT、一个兼容 OpenAI 的 LLM，以及 Qwen3-TTS 进行本地语音输出。

从一份源码检出中，可以通过第二个终端与之对话：

python scripts/listen_and_play_realtime.py --host 127.0.0.1 --port 8765

如果你想将 LLM 保留在本地机器上？用 llama.cpp 提供 Gemma 4：

llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full

然后将兼容 OpenAI 的 LLM 后端指向它：

speech-to-speech \
    --model_name "ggml-org/gemma-4-E4B-it-GGUF" \
    --responses_api_base_url "http://127.0.0.1:8080/v1" \
    --responses_api_api_key ""

任何兼容 OpenAI Realtime 的客户端都可以连接。关于协议和 LLM 后端选项，请参见 Realtime API 与 LLM 后端。

流水线是由四个组件组成的级联，每个组件运行在自己的线程中，并通过队列连接：

1. 语音活动检测 (VAD)： Silero VAD v5 检测语音边界和轮流发言。
2. 语音转文本 (STT)：转录用户的话轮，并可选地提供实时部分转录。
3. 语言模型 (LLM)：生成回复，以流式方式输出文本和工具调用。
4. 文本转语音 (TTS)：合成音频并流式传回客户端。

每个阶段都有多个可互换的后端，通过 CLI 参数选择。代码设计力求易于修改，侧重于通过 Transformers 和 Hugging Face Hub 可用的模型。

需要 Python 3.10+。

pip install speech-to-speech

默认安装涵盖标准的实时路径：

• Parakeet TDT 用于 STT
• 兼容 OpenAI 的 API 用于语言模型
• Qwen3-TTS 用于语音输出，在非 macOS 平台上默认使用 GGML 后端，在 Apple Silicon 上使用 mlx-audio
• 本地音频和实时服务器模式

macOS 和非 macOS 的依赖项通过 pyproject.toml 中的平台标记自动解析。

在 Linux 上，Qwen3-TTS GGML 后端来自 faster-qwen3-tts[ggml]。其在 PyPI 上的默认 qwentts-cpp-python wheel 目标为 CUDA 12.8。如果您的机器没有该 wheel 所需的 CUDA 12 运行时，请在安装 speech-to-speech 前从 Hugging Face wheelhouse 安装匹配的 wheel：

# CUDA 13.x
pip install "qwentts-cpp-python==0.3.0+cu130" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130

# CUDA 12.4
pip install "qwentts-cpp-python==0.3.0+cu124" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu124

# 仅 CPU 回退
pip install "qwentts-cpp-python==0.3.0+cpu" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cpu

pip install speech-to-speech

若要使用之前的 CUDA-graphs 实现而非 GGML，请传递 --qwen3_tts_backend torch。

额外的后端通过 pip extras 安装：

pip install "speech-to-speech[kokoro]"          # Kokoro-82M TTS 在非 macOS 上
pip install "speech-to-speech[pocket]"          # Pocket TTS
pip install "speech-to-speech[chattts]"         # ChatTTS
pip install "speech-to-speech[facebook-mms]"    # MMS TTS
pip install "speech-to-speech[faster-whisper]"  # Faster Whisper STT
pip install "speech-to-speech[whisper-mlx]"     # Lightning Whisper MLX STT 在 macOS 上
pip install "speech-to-speech[paraformer]"      # Paraformer STT through FunASR
pip install "speech-to-speech[mlx-lm]"          # mlx-vlm 支持 macOS 上的视觉模型

已弃用的实现（包括 MeloTTS）位于 archive/，不再连接到 CLI。

关于 DeepFilterNet 的说明： DeepFilterNet 用于 VAD 中的可选音频增强，需要 numpy<2，并与需要 numpy>=2 的 Pocket TTS 冲突。请仅在不使用 Pocket TTS 的环境中手动安装它。

git clone https://github.com/huggingface/speech-to-speech.git
cd speech-to-speech
uv sync

这将以可编辑模式安装该包，并使 speech-to-speech CLI 可用。

组件	后端	平台	安装
VAD	Silero VAD v5	全部	内置
STT	Parakeet TDT （默认）	通过 nano-parakeet 的 CUDA / CPU，通过 MLX 的 Apple Silicon	内置
STT	Whisper through Transformers	CUDA / CPU	内置
STT	Faster Whisper	CUDA / CPU	faster-whisper
STT	Lightning Whisper MLX	Apple Silicon	whisper-mlx
STT	MLX Audio Whisper	Apple Silicon	macOS 内置
STT	Paraformer	CUDA / CPU	paraformer
LLM	兼容 OpenAI 的 API （responses-api, chat-completions）	托管供应商或自托管服务器	内置
LLM	Transformers	CUDA / CPU	内置
LLM	mlx-lm	Apple Silicon	macOS 内置
TTS	Qwen3-TTS （默认）	Linux 上的 GGML / CUDA，macOS 上的 mlx-audio	内置
TTS	Kokoro-82M	CUDA / CPU, Apple Silicon	非 macOS 上用 kokoro；macOS 内置
TTS	Pocket TTS	CPU / CUDA	pocket
TTS	ChatTTS	CUDA / CPU	chattts
TTS	MMS TTS	CUDA / CPU	facebook-mms

通过 --stt、--llm_backend 和 --tts 选择实现。运行 speech-to-speech -h 获取确切值和后端特定标志。

模式	传输方式	何时使用
realtime （默认）	WebSocket, /v1/realtime 上的 OpenAI Realtime 协议	当你基于标准语音 API 构建应用或设备时。
local	你的机器的麦克风和扬声器	当你希望直接与流水线对话，无需客户端时。
websocket	WebSocket 上的原始 PCM	当你想要一个最小的自定义客户端，而不使用 Realtime 协议时。
socket	TCP 上的原始 PCM	模型运行在远程服务器上，配合简单的麦克风/播放客户端。

export OPENAI_API_KEY=...
speech-to-speech

这相当于：

speech-to-speech \
    --thresh 0.6 \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
    --qwen3_tts_speaker Aiden \
    --qwen3_tts_language auto \
    --qwen3_tts_backend ggml \
    --qwen3_tts_non_streaming_mode True \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name gpt-5.4-mini \
    --chat_size 30 \
    --responses_api_stream \
    --enable_live_transcription \
    --mode realtime

默认模型是通过 OpenAI Responses API 的 gpt-5.4-mini。使用 --model_name 重写它，并使用 --responses_api_base_url 设置另一个兼容 OpenAI 的提供商或服务器。

speech-to-speech --local_mac_optimal_settings

可选指定一个特定的 LLM：

speech-to-speech \
    --local_mac_optimal_settings \
    --model_name mlx-community/Qwen3-4B-Instruct-2507-bf16

该设置会：

• 添加 --device mps 来为所有模型使用 MPS。
• 将 Parakeet TDT 设为 STT。
• 将 MLX LM 设为 LLM 后端。
• 将 Qwen3-TTS 设为 TTS，默认使用 mlx-audio 配合 6bit MLX 变体。
• 设置 --mode local。

--tts pocket 和 --tts kokoro 在 macOS 上同样有效。

要在本地比较 MLX 量化变体：

python scripts/benchmark_tts.py \
    --handlers qwen3 \
    --iterations 3 \
    --qwen3_mlx_quantizations bf16 4bit 6bit 8bit

1. 以 WebSocket 模式运行流水线：

   speech-to-speech --mode websocket --ws_host 0.0.0.0 --ws_port 8765
   

2. 从你的客户端连接到 ws://<server-ip>:8765。发送原始音频字节作为 16 kHz、int16、单声道 PCM，并接收生成的音频字节。

TCP socket 模式有意保持最小。它流式传输原始 PCM 音频，但不提供完整的 Realtime API 功能集，包括中断处理、实时转录事件或工具调用事件。

1. 在服务器上运行流水线：

   speech-to-speech --mode socket --recv_host 0.0.0.0 --send_host 0.0.0.0
   

2. 在本地运行客户端以处理麦克风输入和播放：

   python scripts/listen_and_play.py --host <服务器的 IP 地址>
   

安装 NVIDIA Container Toolkit，然后：

该 compose 文件启动一个带有 Gemma 4 的 llama.cpp 服务器，启动 TCP socket 服务器，并暴露端口 8080、12345 和 12346。

Realtime 模式通过 WebSocket 使用 OpenAI Realtime 协议流式传输音频，支持实时转录和低延迟轮流发言。服务器公开 /v1/realtime，任何兼容 OpenAI Realtime 的客户端都可以连接：

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8765/v1",
    websocket_base_url="ws://localhost:8765/v1",
    api_key="not-needed",
)

with client.realtime.connect(model="local") as conn:
    conn.send(
        {
            "type": "session.update",
            "session": {
                "type": "realtime",
                "instructions": "你是一个乐于助人的助手。",
                "audio": {
                    "input": {
                        "turn_detection": {
                            "type": "server_vad",
                            "interrupt_response": True,
                        }
                    }
                },
            },
        }
    )

    for event in conn:
        print(event.type)

该服务器实现了核心 Realtime 事件集：入站 input_audio_buffer.append、session.update、conversation.item.create、response.create 和 response.cancel；出站 语音开始/停止、流式转录、音频增量、工具调用和 response.done。完整的事件参考、架构和设计细节参见 Realtime Engine README。

LLM 是流水线中计算最密集、延迟最高的组件。通过一个大模型的单次前向传播可能会主导端到端响应时间，因此为你的硬件和延迟预算选择合适的后端非常重要。流水线支持：

• 本地推理：CUDA / CPU 上的 transformers 和 Apple Silicon 上的 mlx-lm。
• 自托管服务器：responses-api 和 chat-completions 可以指向本地的 vLLM 或 llama.cpp 服务器。
• 供应商 API：相同的后端可与 OpenAI、HF Inference Providers、OpenRouter 以及其他兼容 OpenAI 的供应商配合使用。

有两个 API 后端可用，共享相同的 --responses_api_* 连接标志：

• --llm_backend responses-api （默认）目标为 /v1/responses。
• --llm_backend chat-completions 目标为 /v1/chat/completions。

以下示例将用于本地 STT 的 Parakeet TDT 和用于本地 TTS 的 Qwen3-TTS，与不同的 LLM 后端配对。

可与任何实现 OpenAI Responses API 的供应商或服务器配合使用。将 --responses_api_base_url 指向端点，并相应设置 --model_name：

供应商 / 服务器	--responses_api_base_url	--responses_api_api_key
OpenAI	省略，使用 OpenAI 默认值	$OPENAI_API_KEY
HF Inference Providers	https://router.huggingface.co/v1	$HF_TOKEN
OpenRouter	https://openrouter.ai/api/v1	$OPENROUTER_API_KEY
vLLM	http://localhost:8000/v1	省略或任意字符串
llama.cpp	http://127.0.0.1:8080/v1	空字符串

# OpenAI
speech-to-speech \
    --mode local \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name "gpt-4o-mini" \
    --responses_api_api_key "$OPENAI_API_KEY" \
    --responses_api_stream \
    --enable_live_transcription


# HF Inference Providers: Qwen3.5-9B via Together
speech-to-speech \
    --mode local \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantiz

A low-latency, fully modular voice-agent pipeline: VAD - > STT -> LLM -> TTS, exposed through an OpenAI Realtime-compatible WebSocket API. Every component is swappable. The LLM slot speaks OpenAI-compatible protocols, so you can point it at a hosted provider, at HF Inference Providers, or at a vLLM or llama.cpp server on your own hardware for a fully local, fully open stack.

This pipeline runs in production as the conversation backend for thousands of Reachy Mini robots.

pip install speech-to-speech
export OPENAI_API_KEY=...
speech-to-speech

This starts an OpenAI Realtime-compatible server at ws://localhost:8765/v1/realtime using Parakeet TDT for local STT, an OpenAI-compatible LLM, and Qwen3-TTS for local speech output.

From a source checkout, talk to it from a second terminal:

python scripts/listen_and_play_realtime.py --host 127.0.0.1 --port 8765

Prefer to keep the LLM on your own machine? Serve Gemma 4 with llama.cpp:

llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full

Then point the OpenAI-compatible LLM backend at it:

speech-to-speech \
    --model_name "ggml-org/gemma-4-E4B-it-GGUF" \
    --responses_api_base_url "http://127.0.0.1:8080/v1" \
    --responses_api_api_key ""

Any OpenAI Realtime-compatible client can connect. See Realtime API for the protocol and LLM backends for provider and local-server options.

The pipeline is a cascade of four components, each running in its own thread and connected by queues:

1. Voice Activity Detection (VAD) : Silero VAD v5 detects speech boundaries and turn-taking.
2. Speech to Text (STT) : transcribes the user's turn, with optional live partial transcripts.
3. Language Model (LLM) : generates the response, streaming text and tool calls.
4. Text to Speech (TTS) : synthesizes audio and streams it back to the client.

Every stage has multiple interchangeable backends, selected via CLI flags. The code is designed for easy modification, with a focus on models available through Transformers and the Hugging Face Hub.

Requires Python 3.10+.

pip install speech-to-speech

The default install covers the standard realtime path:

• Parakeet TDT for STT
• OpenAI-compatible API for the language model
• Qwen3-TTS for speech output, using the GGML backend by default on non-macOS platforms and mlx-audio on Apple Silicon
• local audio and realtime server modes

macOS and non-macOS dependencies are resolved automatically via platform markers in pyproject.toml.

On Linux, the Qwen3-TTS GGML backend comes from faster-qwen3-tts[ggml]. Its default qwentts-cpp-python wheel on PyPI targets CUDA 12.8. If your machine does not have the CUDA 12 runtime that wheel expects, install the matching wheel from the Hugging Face wheelhouse before installing speech-to-speech:

# CUDA 13.x
pip install "qwentts-cpp-python==0.3.0+cu130" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130

# CUDA 12.4
pip install "qwentts-cpp-python==0.3.0+cu124" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu124

# CPU-only fallback
pip install "qwentts-cpp-python==0.3.0+cpu" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cpu

pip install speech-to-speech

To use the previous CUDA-graphs implementation instead of GGML, pass --qwen3_tts_backend torch.

Extra backends are installed with pip extras:

pip install "speech-to-speech[kokoro]"          # Kokoro-82M TTS on non-macOS
pip install "speech-to-speech[pocket]"          # Pocket TTS
pip install "speech-to-speech[chattts]"         # ChatTTS
pip install "speech-to-speech[facebook-mms]"    # MMS TTS
pip install "speech-to-speech[faster-whisper]"  # Faster Whisper STT
pip install "speech-to-speech[whisper-mlx]"     # Lightning Whisper MLX STT on macOS
pip install "speech-to-speech[paraformer]"      # Paraformer STT through FunASR
pip install "speech-to-speech[mlx-lm]"          # mlx-vlm support for vision models on macOS

Deprecated implementations, including MeloTTS, live in archive/ and are no longer wired into the CLI.

Note on DeepFilterNet: DeepFilterNet, used for optional audio enhancement in VAD, requires numpy<2 and conflicts with Pocket TTS, which requires numpy>=2. Install it manually only in environments where you are not using Pocket TTS.

git clone https://github.com/huggingface/speech-to-speech.git
cd speech-to-speech
uv sync

This installs the package in editable mode and makes the speech-to-speech CLI available.

Component	Backend	Platforms	Install
VAD	Silero VAD v5	all	built-in
STT	Parakeet TDT (default)	CUDA / CPU through nano-parakeet, Apple Silicon through MLX	built-in
STT	Whisper through Transformers	CUDA / CPU	built-in
STT	Faster Whisper	CUDA / CPU	faster-whisper
STT	Lightning Whisper MLX	Apple Silicon	whisper-mlx
STT	MLX Audio Whisper	Apple Silicon	built-in on macOS
STT	Paraformer	CUDA / CPU	paraformer
LLM	OpenAI-compatible API (responses-api, chat-completions)	hosted providers or self-hosted servers	built-in
LLM	Transformers	CUDA / CPU	built-in
LLM	mlx-lm	Apple Silicon	built-in on macOS
TTS	Qwen3-TTS (default)	GGML / CUDA on Linux, mlx-audio on macOS	built-in
TTS	Kokoro-82M	CUDA / CPU, Apple Silicon	kokoro on non-macOS; built-in on macOS
TTS	Pocket TTS	CPU / CUDA	pocket
TTS	ChatTTS	CUDA / CPU	chattts
TTS	MMS TTS	CUDA / CPU	facebook-mms

Select implementations with --stt, --llm_backend, and --tts. Run speech-to-speech -h for exact values and backend-specific flags.

Mode	Transport	Use it when
realtime (default)	WebSocket, OpenAI Realtime protocol at /v1/realtime	You are building an app or device against a standard voice API.
local	Your machine's microphone and speakers	You want to talk to the pipeline directly, no client needed.
websocket	Raw PCM over WebSocket	You want a minimal custom client without the Realtime protocol.
socket	Raw PCM over TCP	Models run on a remote server, with a simple microphone/playback client.

export OPENAI_API_KEY=...
speech-to-speech

This is equivalent to:

speech-to-speech \
    --thresh 0.6 \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
    --qwen3_tts_speaker Aiden \
    --qwen3_tts_language auto \
    --qwen3_tts_backend ggml \
    --qwen3_tts_non_streaming_mode True \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name gpt-5.4-mini \
    --chat_size 30 \
    --responses_api_stream \
    --enable_live_transcription \
    --mode realtime

The default model is gpt-5.4-mini through the OpenAI Responses API. Override it with --model_name, and set --responses_api_base_url for another OpenAI-compatible provider or server.

speech-to-speech --local_mac_optimal_settings

Optionally with a specific LLM:

speech-to-speech \
    --local_mac_optimal_settings \
    --model_name mlx-community/Qwen3-4B-Instruct-2507-bf16

This setting:

• Adds --device mps to use MPS for all models.
• Sets Parakeet TDT for STT.
• Sets MLX LM as the LLM backend.
• Sets Qwen3-TTS for TTS, using mlx-audio with the 6bit MLX variant by default.
• Sets --mode local.

--tts pocket and --tts kokoro are also valid on macOS.

To compare the MLX quantization variants locally:

python scripts/benchmark_tts.py \
    --handlers qwen3 \
    --iterations 3 \
    --qwen3_mlx_quantizations bf16 4bit 6bit 8bit

1. Run the pipeline in WebSocket mode:

   speech-to-speech --mode websocket --ws_host 0.0.0.0 --ws_port 8765
   

2. Connect from your client at ws://<server-ip>:8765. Send raw audio bytes as 16 kHz, int16, mono PCM and receive generated audio bytes back.

TCP socket mode is intentionally minimal. It streams raw PCM audio, but does not provide the full Realtime API feature set, including interruption handling, live transcript events, or tool-call events.

1. Run the pipeline on the server:

   speech-to-speech --mode socket --recv_host 0.0.0.0 --send_host 0.0.0.0
   

2. Run the client locally to handle microphone input and playback:

   python scripts/listen_and_play.py --host <IP address of your server>
   

Install the NVIDIA Container Toolkit, then:

The compose file starts a llama.cpp server with Gemma 4, starts the TCP socket server, and exposes ports 8080, 12345, and 12346.

Realtime mode streams audio over a WebSocket using the OpenAI Realtime protocol, with live transcription and low-latency turn-taking. The server exposes /v1/realtime, and any OpenAI Realtime-compatible client can connect:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8765/v1",
    websocket_base_url="ws://localhost:8765/v1",
    api_key="not-needed",
)

with client.realtime.connect(model="local") as conn:
    conn.send(
        {
            "type": "session.update",
            "session": {
                "type": "realtime",
                "instructions": "You are a helpful assistant.",
                "audio": {
                    "input": {
                        "turn_detection": {
                            "type": "server_vad",
                            "interrupt_response": True,
                        }
                    }
                },
            },
        }
    )

    for event in conn:
        print(event.type)

The server implements the core Realtime event set: input_audio_buffer.append, session.update, conversation.item.create, response.create, and response.cancel inbound; speech start/stop, streaming transcription, audio deltas, tool calls, and response.done outbound. The full event reference, architecture, and design details live in the Realtime Engine README.

The LLM is the most compute-intensive and highest-latency component in the pipeline. A single forward pass through a large model can dominate end-to-end response time, so choosing the right backend for your hardware and latency budget matters. The pipeline supports:

• Local inference : transformers on CUDA / CPU and mlx-lm on Apple Silicon.
• Self-hosted servers : responses-api and chat-completions can point at a local vLLM or llama.cpp server.
• Provider APIs : the same backends work with OpenAI, HF Inference Providers, OpenRouter, and other OpenAI-compatible providers.

Two API backends are available, sharing the same --responses_api_* connection flags:

• --llm_backend responses-api (default) targets /v1/responses.
• --llm_backend chat-completions targets /v1/chat/completions.

The examples below pair Parakeet TDT for local STT and Qwen3-TTS for local TTS with different LLM backends.

Works with any provider or server that implements the OpenAI Responses API. Point --responses_api_base_url at the endpoint and set --model_name accordingly:

Provider / server	--responses_api_base_url	--responses_api_api_key
OpenAI	omit, uses OpenAI default	$OPENAI_API_KEY
HF Inference Providers	https://router.huggingface.co/v1	$HF_TOKEN
OpenRouter	https://openrouter.ai/api/v1	$OPENROUTER_API_KEY
vLLM	http://localhost:8000/v1	omit or any string
llama.cpp	http://127.0.0.1:8080/v1	empty string

# OpenAI
speech-to-speech \
    --mode local \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name "gpt-4o-mini" \
    --responses_api_api_key "$OPENAI_API_KEY" \
    --responses_api_stream \
    --enable_live_transcription


# HF Inference Providers: Qwen3.5-9B via Together
speech-to-speech \
    --mode local \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name "Qwen/Qwen3.5-9B:together" \
    --responses_api_base_url "https://router.huggingface.co/v1" \
    --responses_api_api_key "$HF_TOKEN" \
    --responses_api_stream \
    --enable_live_transcription


# HF Inference Providers: GPT-oss-20B via Groq
speech-to-speech \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name "openai/gpt-oss-20b:groq" \
    --responses_api_base_url "https://router.huggingface.co/v1" \
    --responses_api_api_key "$HF_TOKEN" \
    --responses_api_stream \
    --enable_live_transcription

Identical configuration to responses-api, reusing the same --responses_api_* connection flags, but talks to /v1/chat/completions instead of /v1/responses. Prefer it when:

• the provider ignores chat_template_kwargs.enable_thinking on the Responses path and needs a reasoning_effort knob to suppress reasoning, or
• the server's Responses streaming tool-call path is unreliable, while its Chat Completions tool-call streaming is solid. This is useful for some vLLM builds; see #312.

Add --responses_api_reasoning_effort none to disable reasoning on providers where the chat-template flag has no effect:

# vLLM serving a Qwen model with tool calling
speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend chat-completions \
    --tts qwen3 \
    --model_name "Qwen/Qwen3-4B-Instruct-2507" \
    --responses_api_base_url "http://localhost:8000/v1" \
    --responses_api_stream


# Gemma 4 31B via the HF router on Cerebras, with reasoning disabled for low voice latency
speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend chat-completions \
    --tts qwen3 \
    --model_name "google/gemma-4-31B-it:cerebras" \
    --responses_api_base_url "https://router.huggingface.co/v1" \
    --responses_api_api_key "$HF_TOKEN" \
    --responses_api_reasoning_effort none \
    --responses_api_stream

Run the LLM in a separate llama.cpp process for the lowest-friction fully local setup, as shown in the Reachy Mini local conversation guide:

# Terminal 1: llama.cpp serving Gemma 4
llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full


# Terminal 2: speech-to-speech using that local LLM server
speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --model_name "ggml-org/gemma-4-E4B-it-GGUF" \
    --responses_api_base_url "http://127.0.0.1:8080/v1" \
    --responses_api_api_key "" \
    --responses_api_stream \
    --enable_live_transcription

You can use --mode local instead of --mode realtime when you want to talk through the machine running the server directly. In-process local backends are still available with --llm_backend mlx-lm on Apple Silicon or --llm_backend transformers on CUDA / CPU.

Language coverage depends on the STT and TTS backends you pick, not on the pipeline itself:

Component	Backend	Languages
STT	Parakeet TDT (default)	25 European languages
STT	Whisper / Whisper MLX / Faster Whisper	Broad multilingual coverage, depending on the selected Whisper checkpoint
STT	Paraformer	Depends on the selected FunASR checkpoint; the default is Chinese-oriented
TTS	Qwen3-TTS (default)	Multilingual, with --qwen3_tts_language auto by default
TTS	Kokoro	Multiple language/voice mappings, depending on backend availability
TTS	ChatTTS	English and Chinese
TTS	MMS TTS	Broad multilingual coverage through MMS checkpoints

Make sure the STT, LLM, and TTS you pair all cover your target language(s). Two usage patterns:

• Single language : set --language to the target language code. The default is en.
• Language switching : set --language auto. The STT detects the language of each spoken prompt and forwards it to the LLM. Optionally add --enable_lang_prompt to append a "Please reply to my message in ..." instruction. It defaults to False; large LLMs usually infer the language from context, but the explicit instruction can help smaller models.

Automatic language detection:

speech-to-speech \
    --stt parakeet-tdt \
    --language auto \
    --llm_backend mlx-lm \
    --model_name "mlx-community/Qwen3-4B-Instruct-2507-bf16"

A single non-English language, Chinese in this example:

speech-to-speech \
    --stt whisper-mlx \
    --stt_model_name large-v3 \
    --language zh \
    --llm_backend mlx-lm \
    --model_name mlx-community/Qwen3-4B-Instruct-2507-bf16

Both commands also work on top of --local_mac_optimal_settings; explicit --stt flags override the defaults it sets.

Pocket TTS from Kyutai Labs provides streaming TTS with voice cloning:

speech-to-speech \
    --tts pocket \
    --pocket_tts_voice jean \
    --pocket_tts_device cpu

Available voice presets: alba, marius, javert, jean, fantine, cosette, eponine, azelma. Custom voice files and Hugging Face paths also work.

References for all CLI arguments live in the arguments classes and in speech-to-speech -h.

See ModuleArguments. It allows setting:

• a common --device, if every part should run on the same device
• --mode: realtime (default), local, socket, or websocket
• STT implementation (--stt)
• LLM backend (--llm_backend: transformers, mlx-lm, responses-api, or chat-completions)
• TTS implementation (--tts)
• logging level
• realtime pipeline pool size (--num_pipelines)

See VADHandlerArguments. Notable options:

• --thresh: threshold value to trigger voice activity detection.
• --min_speech_ms: minimum duration of detected voice activity to be considered speech.
• --min_speech_continuation_ms: sustain-bar hysteresis threshold for speech that continues a reopenable soft-ended, uncommitted turn within the reopen window. The default and recommended pairing is --min_speech_ms 384 --min_speech_continuation_ms 192.
• --min_silence_ms: minimum length of silence intervals for segmenting speech. Default is 64 ms.
• --short_segment_merge_ms: optional merge window for stitching adjacent VAD segments that are each shorter than --min_speech_ms.
• --unanswered_reopen_ms: sanity cap on how long a soft-ended speculative turn that has not yet received any assistant output stays reopenable.

STT, LLM, and TTS Parameters

model_name, torch_dtype, and device are exposed for each STT, LLM, and TTS implementation. STT and TTS parameters use the handler prefix, for example --stt_model_name or --qwen3_tts_device. LLM model selection and chat settings are shared across backends via unprefixed flags, for example --model_name and --chat_size; backend-specific flags use the responses_api_ prefix for the responses-api and chat-completions backends and the llm_ prefix for local backends.

For example:

# Local transformers/mlx-lm backend
--model_name google/gemma-2b-it

# OpenAI-compatible backend
--llm_backend responses-api --model_name deepseek-chat --responses_api_base_url https://api.deepseek.com

Other generation parameters can be set using the handler prefix plus _gen_, for example --stt_gen_max_new_tokens 128 or --llm_gen_temperature 0.7. Parameters not yet exposed can be added to the relevant arguments class.

Issues and PRs are welcome. Good starting points are the open issues. For larger changes, open an issue first to discuss the approach.

For local development:

uv sync
pytest
ruff check

If you use this pipeline, please also cite the component models you run. The defaults are:

@misc{SileroVAD,
  author = {Silero Team},
  title = {Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/snakers4/silero-vad}},
  email = {hello@silero.ai}
}


@misc{parakeet-tdt,
  author = {NVIDIA},
  title = {Parakeet TDT 0.6B v3},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3}}
}


@misc{qwen3-tts,
  author = {Qwen Team},
  title = {Qwen3-TTS},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice}}
}

Citations for optional backends such as Kokoro, Pocket TTS, ChatTTS, Whisper variants, Paraformer, and MMS live in the respective component READMEs.