API Gateway 支持流式传输：无需修改规范，解锁面向 AI 与实时的 API 能力

在过去十年的大部分时间里，一次 API 调用看起来就像是一笔事务。客户端发送请求、等待，并一次性接收到完整的响应。这种模式正是绝大多数 API 管理工具（包括 API Gateway）最初设计所针对的场景，并且至今仍代表着生产环境中极大部分的流量。

但这已不再涵盖所有的流量类型。大模型在生成响应时是逐个 Token 输出的，而不是在最后一次性返回；执行多步任务的 Agent 在运行过程中需要实时汇报进度；仪表盘、协同编辑器、实时推理端点——这些场景都需要一个能够长期保持开启并实现双向数据传输的连接。

在此之前，开发者在 API Gateway 中构建此类体验时不得不面临一个痛苦的选择：要么将 API Gateway 置于 API 前端并放弃流式传输能力，要么让流式端点绕过网关，从而失去原本不可或缺的身份验证、API Key 强制校验和可观测性。最终，团队不得不维护一套割裂的架构——一条路径走受治理的常规 API，另一条路径走不受治理的流式流量。

我们正式宣布：API Gateway 的流式传输支持现已进入公开预览阶段（Public Preview）。 你的流式端点现在可以与所有其他服务一样，置于同一个网关之后，享有完全一致的安全与治理策略。

对于 AI 应用而言，流式传输还强化了 API Gateway 与 Gemini 平台的连接。支持模型路由（Model Routing）的网关现在可以在生成内容的同时，实时流式返回来自 Gemini 平台 Model Garden 中的 Gemini、Claude 和 OpenAI gpt-oss 模型的响应，托管在其他任何位置的流式后端也同样适用。与 Apigee 以及 Gemini 平台中的 Agent Gateway 一起，API Gateway 成为了 Google Cloud AI 网关能力的轻量级切入点。

支持的流式传输类型

API Gateway 现已全面支持现代 API 所依赖的各类流式传输模式：

• Server-Sent Events (SSE) —— 向浏览器或客户端 SDK 流式传输模型输出的事实标准，也是绝大多数兼容 OpenAI 的 Chat Completion API 底层采用的传输机制。
• 分块传输编码（Chunked transfer encoding）与 HTTP 响应流 —— 包括换行符分隔的 JSON（NDJSON）以及任何以渐进方式交付的响应体。响应会在后端生成时即时传递，而不是经过缓冲后在最后统一交付。
• WebSockets —— 用于聊天、协作、实时仪表盘、通知和交互式会话的全双工连接。
• gRPC 流式传输 —— 服务端流式与双向流式，使你能够为此前无法被代理的 gRPC 服务部署统一网关。

流连接最长可保持开启 1 小时。与任何长连接一样，客户端应实现重连逻辑以确保会话在流结束时能够恢复——正常规范的 WebSocket 或 SSE 客户端本来就应当具备该机制。

你的 API 契约无需任何更改

这是我们最感到欣慰的一点：完全没有新概念需要学习。

流式传输并不是一个全新的产品、新的资源类型，也不需要改换 API 的描述方式。无需添加任何新的扩展插件，无需流式专属的注解，也无需为流式操作维护单独的规范文档。描述现有 API 的同一份文档，就可以直接描述你的流式 API。

你此前所依赖的一切功能均可照常使用：

• API Key 强制校验
• JWT 验证（包括 Firebase、Auth0、Okta 以及 Google ID Token）
• 面向后端的服务账号（Service account）身份验证
• 与 Cloud Logging 和 Cloud Monitoring 的原生集成
• 路径模板化与请求路由

在公开预览期间，你只需在创建网关时显式启用：

gcloud api-gateway gateways create GATEWAY_ID
--api=API_ID
--api-config=CONFIG_ID
--location=REGION
--enable-streaming

Plain text

这一个单独的标记（flag）就是全部的区别。在正式商用（GA）时，计划将流式传输作为 API Gateway 的默认行为，无需显式 opt-in，并且与预览期一样，无需修改你的任何 API 配置。

可预测的确定性主机名

与流式传输一同推出的，还有网关的确定性默认主机名：

GATEWAY_ID-PROJECT_NUMBER.REGION.gateway.dev

(Example: streaming-test-12345.us-central1.gateway.dev)

Plain text

此前，默认的 gateway.dev 主机名中包含随机生成的哈希值，这意味着只有在网关创建完成后，你才能知道其具体 URL。这虽是一个小细节，却带来了极大的摩擦：在资源创建之前，你无法将端点地址写入客户端配置、Terraform 模块、CI 流水线、防火墙白名单或合作伙伴集成文档中；而且一旦拆除环境重新构建，底层 URL 就会发生变化。

现在，主机名纯粹由你预先选择的值决定。在网关创建之前，你就可以提前生成、模板化、记录并将其提交至版本控制系统；每当在同一个项目和区域中重建同一个网关时，你都会获得完全相同的 URL。对于任何践行基础设施即代码（IaC）或将配置从开发环境推向测试、生产环境的开发者而言，这彻底解决了一整类启动初始化难题。

为什么这对 AI 工作负载至关重要

对于 AI 应用而言，流式传输绝非可有可无的小众特性——它是行业标配的交互模式。

用户期望看到大模型在生成时逐字显示响应，而不是停顿 30 秒后一次性吐出；Agent 需要汇报执行的中间步骤；工具调用协议（Tool-calling protocols）以及远程 MCP 服务器都依赖长连接与渐进式交付。一个不支持流式传输的 API 管理层，会严重损害 AI 应用的用户体验。

有了流式传输支持，API Gateway 成为承载这些 Agent 流量的理想入口——而这一入口的价值与以往一脉相承，甚至更加迫切。AI 端点成本高昂、消耗配额，需要针对每个消费调用方进行鉴权与速率限制，并且需要具备完整的可观测性。这些正是 AI 网关旨在解决的治理难题。如果任由每个团队在各自的模型端点前独立搭建代理，企业最终将无法弄清“谁在调用什么”这一根本问题。

API Gateway 的模型路由（Model routing）功能允许你对外暴露一个统一的兼容 OpenAI 的端点，并根据请求载荷中的 model 字段将请求分发至 Gemini、Claude 或 OpenAI gpt-oss 模型。模型路由加上流式传输，意味着客户端可以获得熟悉的、基于标准的流式接口，同时由你在中央集中控制哪些模型可被哪些人、依据何种策略访问。

流式传输、模型路由以及 MCP 支持 都是 API Gateway 提供的轻量级 AI 网关功能矩阵的一部分。这些 AI 网关能力帮助团队对生态系统内的工具、模型和 Agent 流量进行监控、保护与控制，且它们以可组合套件的形式提供，方便你针对不同用例选择适合的网关。对于更广泛的 AI 策略与全生命周期管理，可使用 Apigee；为了管理和保护 Agent 连接，可使用 Gemini 平台中的 Agent Gateway。你还可以将这些网关组合使用：API Gateway 或 Apigee 可以置于你的 Agent 和 API 之前，也可以置于 Agent Gateway 之后，以便你的 Agent 能够访问模型、工具和 MCP 服务器。例如，你可以让 Agent 的出站流量经过 Agent Gateway 以实现严格的安全治理，随后再将请求传递给 API Gateway 进行模型路由。

公开预览期间注意事项

A few things to plan around while the feature is in Preview:

• 流式传输需在创建网关时启用。 现有的网关无法直接原地转换；如需体验，请使用相应标记新建网关。
• 少数 API Gateway 功能在初期预览版本中暂不可用，但会在预览期内陆续推出：
  • 通过 Serverless NEG 与外部 Application Load Balancer 配置自定义域名
  • 通过 Cloud Console 控制台启用流式传输
  • Terraform 支持
• 延迟指标反映的是流的持续时间。 流式连接的请求延迟衡量的是整个流的存活周期，而非首字节时间（TTFB），因此长连接会改变你的延迟图表形态。

快速开始

如果你已经在使用 API Gateway，最快的体验方式就是创建一个启用了流式传输的新网关，将其指向现有的 SSE 后端，即可观察事件渐进式抵达的效果。无需修改任何规范。

For most of the last decade, an API call looked like a transaction. A client sent a request, waited, and got a complete response back in one piece. That model is exactly what most API management tooling — including API Gateway — was built to handle, and it still describes a huge amount of production traffic.

But that no longer describes all traffic types. A model generating a response emits tokens as it produces them, not in one batch at the end. An agent working through a multi-step task reports progress as it goes. A dashboard, a collaborative editor, a live inference endpoint — these all need a connection that stays open and carries data in both directions.

Until now, developers building those experiences in API Gateway had to make an uncomfortable choice: put API Gateway in front of their APIs and give up streaming, or route streaming endpoints around the gateway and give up the authentication, API key enforcement, and observability that made the gateway essential in the first place. Teams ended up with a split estate — governed APIs on one path, ungoverned streaming traffic on another.

We're announcing that streaming support in API Gateway is available in Public Preview. Your streaming endpoints can now sit behind the same gateway, with the same policies, as everything else you serve.

For AI applications, streaming also strengthens API Gateway's connection to Gemini platform. A gateway that uses model routing now streams responses from Gemini, Claude and OpenAI gpt-oss models in model garden in Gemini platform as they're generated, and streaming backends you host anywhere else work the same way. Alongside Apigee and Agent Gateway in Gemini platform, API Gateway is the lightweight entry point to Google Cloud's AI gateway capabilities.

What you can stream

API Gateway now supports the full range of streaming patterns modern APIs rely on:

• Server-Sent Events (SSE) — the de facto standard for streaming model output to a browser or client SDK, and the transport behind most OpenAI-compatible chat completion APIs.
• Chunked transfer encoding and HTTP response streaming — including newline-delimited JSON (NDJSON) and any other progressively-delivered response body. Responses are passed through as your backend produces them rather than buffered and delivered at the end.
• WebSockets — full-duplex connections for chat, collaboration, live dashboards, notifications, and interactive sessions.
• gRPC streaming — server-side streaming and bidirectional streaming, so you can put a gateway in front of gRPC services that were previously impossible to proxy.

Streams can stay open for up to one hour. As with any long-lived connection, clients should implement reconnection logic so that a session survives the end of a stream — a well-behaved WebSocket or SSE client should be doing this anyway.

Your API contract doesn't change

This is the part we are most pleased with: there is nothing new to learn.

Streaming is not a different product, a different resource type, or a different way of describing your API. There are no new extensions to add, no streaming-specific annotations, no separate spec for streaming operations. The same document that describes your existing API describes your streaming API.

Everything you already rely on continues to work exactly as before:

• API key enforcement
• JWT validation, including Firebase, Auth0, Okta, and Google ID tokens
• Service account authentication to backends
• Cloud Logging and Cloud Monitoring integration
• Path templating and request routing

During Public Preview, you opt-in when creating the gateway:

gcloud api-gateway gateways create GATEWAY_ID \
  --api=API_ID \
  --api-config=CONFIG_ID \
  --location=REGION \
  --enable-streaming

Plain text

That single flag is the entire difference. At GA, it is planned that streaming will simply be how API Gateway behaves, with no opt-in required and, as during Preview, no changes to your API config.

A hostname you can predict

Alongside streaming, gateways get a deterministic default hostname:

GATEWAY_ID-PROJECT_NUMBER.REGION.gateway.dev

(Example: streaming-test-12345.us-central1.gateway.dev)

Plain text

Previously the default gateway.dev hostname included a generated hash, which meant you could not know your gateway's URL until after you had created it. That is a small thing that causes a lot of friction: you can't write the endpoint into a client config, a Terraform module, a CI pipeline, a firewall allowlist, or a partner integration doc until the resource exists — and if you tear the environment down and rebuild it, the URL changes underneath you.

Now the hostname is a pure function of values you already chose. You can generate it, template it, document it, and commit it to source control before the gateway exists, and you get the same URL every time you rebuild the same gateway in the same project and region. For anyone managing environments as code, or promoting a configuration from dev to staging to prod, this removes an entire category of bootstrapping problem.

Why this matters for AI workloads

Streaming isn't a niche feature for AI applications — it's the expected interaction pattern.

Users expect to see a model's response appear as it's generated, not after a thirty-second pause. Agents need to report intermediate steps. Tool-calling protocols and remote MCP servers depend on long-lived, incrementally-delivered connections. An API management layer that can't stream severely limits the user experience of AI applications.

With streaming, API Gateway becomes a viable front door for that agentic traffic — and the value of a front door is exactly the same as it's always been, only more urgent. AI endpoints are expensive, they consume quota, they need per-consumer authentication and rate limiting, and they need to be observable. These are the governance problems that an AI gateway is intended to solve. The alternative — every team standing up its own proxy in front of its own model endpoint — is how organizations end up unable to answer basic questions about who is calling what.

Model routing in API Gateway lets you expose a single OpenAI-compatible endpoint and dispatch requests to Gemini, Claude, or OpenAI gpt-oss models based on the model field in the request payload. Model routing plus streaming means clients get a familiar, standards-based streaming interface, while you retain centralized control over which models are reachable, by whom, and under what policy.

Streaming, model routing and MCP support are all features within API Gateway that provide a lightweight on-ramp to AI gateway capabilities within Google Cloud. These AI gateway capabilities allow teams to monitor, secure and control agentic traffic of tools, models and agents in their ecosystem, and they're delivered as a composable suite so you can choose the right gateway for each use case. For more expansive AI policies and full lifecycle management, use Apigee. To manage and secure agent connectivity, use Agent Gateway in Gemini platform. You can also combine these gateways: API Gateway or Apigee can sit in front of your agents and APIs, or behind Agent Gateway so your agents can reach models, tools, and MCP servers. For example, you can route your agent's egress through Agent Gateway for strict security governance, and then pass the request to API Gateway to handle model routing.

What to know during Public Preview

A few things to plan around while the feature is in Preview:

• Streaming is enabled at gateway creation time. Existing gateways can't be converted in place; create a new gateway with the flag to try it.
• A few API Gateway features are not available at the initial Preview launch, but will be coming during the Preview time period.
  • Custom domains via Serverless NEGs and an external Application Load Balancer
  • Streaming enablement via the Cloud console
  • Terraform
• Latency metrics reflect stream duration. Request latency for a streaming connection measures the life of the stream, not time-to-first-byte, so expect long-lived connections to change the shape of your latency graphs.

Get started

If you're already using API Gateway, the fastest way to see this working is to point a new streaming-enabled gateway at an existing backend that emits SSE and watch the events arrive incrementally. No spec changes needed.