在 Google Cloud 上构建受监管的自进化 AI Agent 架构

自进化 AI 的崛起

随着企业级云平台扩展至服务数百万并发用户，高度耦合的微服务往往会引发级联故障模式，这是任何静态 Runbook 或阈值告警都无法预见的情况。在黑色星期五闪购等超高速流量事件中，网站可靠性工程师（SRE）面临着严苛的现实：必须在数秒内从充斥噪声的分布式遥测数据中完成根因诊断，否则将面临每分钟损失数百万美元的级联宕机风险。

尽管生成式 AI Copilot 加速了代码编写和日志总结，但 AI Agent 在可靠修复生产线上实时故障方面仍面临巨大挑战。一位正在应对连接池耗尽的值班工程师，需要的不是提供排查建议或未经验证且可能无法解决根本问题 PR 的 AI Agent；他们需要的是一个受监管的、自进化的 AI 系统，该系统能够：

1. 诊断：实时诊断跨缓存层和数据库层的分布式根本原因；
2. 自修补：在运行时动态合成、验证可执行的缓解工具，并将其挂载到线上服务器中间件中；
3. 自我批评：计算每项操作的边际贡献信用，区分有效修复与有害的下意识反射行为；
4. 记忆：将经过密码学签名的运维不变量提炼至语义内存中，以降低相同故障再次发生的概率；
5. 架构重构：在获得人类批准的前提下，在内存中动态重构自身的多 Agent 层级拓扑，以承载未来的突发流量。

这种全新范式即为自进化 AI（Self-Evolving AI）。

在这项代号为 Project Chrysalis 的 Google Cloud 研究参考架构中，我们展示了三项基础研究突破如何融合成一个统一的、生产级的架构，并运行在 Gemini 平台与 Agent Development Kit (ADK) 之上。在模拟的黑五缓存击穿（Cache Stampede）故障中，该受监管架构将首次缓解时间（time-to-first-mitigation）从 15~45 分钟缩短至 1.8 秒，HTTP 5xx 错误率从 72.7%~80.0% 降至 0.0%，P99 延迟从 519.8 毫秒降至 18.2 毫秒，并在 BigQuery 中以约 0.5 毫秒的速度评估了 400 种反事实流量策略。

至关重要的是，该自进化架构并非作为一个不受约束的“黑盒”运行。为确保 Agent 始终安全、可靠、对齐，并防范失控行为，其每一项自主操作都受到 Google Cloud 纵深防御安全技术栈的密码学约束：利用 Model Armor 对传入的遥测数据进行净化，防止提示词注入（Prompt Injection）与数据泄露；借助 Google Cloud Key Management Service (Cloud KMS) 和二进制授权（Binary Authorization）中间件对合成的补丁进行密码学签名，阻断未经验证的代码挂载；使用 Open Policy Agent (OPA) Policy Controller 强制执行严格的 SLA 防护护栏，防止造成二次人为故障；并引入双人托管（Dual-Custody）人机协同（HITL）治理，确保在对 Agent 拓扑结构进行任何永久性更改前，必须获得 SRE 的显式审批。

---

1. 故障事件：黑五缓存击穿与 Agent 自进化的必要性

为了理解静态预案和建议型 Copilot 为何在压力下失效，请看一个典型的分布式级联故障过程：

1. 入口流量激增：限时促销活动引发对冷门库存 Key 的突发高并发结账请求。
2. 缓存逐出：Google Cloud Memorystore (Redis) 在严格的最近最少使用（LRU）内存限制下逐出冷门 Key，导致缓存命中率暴跌至零。
3. 连接池耗尽：未经缓存过滤的流量直接级联冲击 Google Cloud SQL。一旦 PostgreSQL 连接池饱和，进来的查询将排队超时，服务开始大面积报出 HTTP 500 错误。
4. 致命的“凌晨 3:00 恐慌反射”：在巨大的运维压力下，人类响应者和简单朴素的 LLM 往往会本能地尝试重启数据库连接池。在实时流量冲击下，断开连接会导致数据库内核锁死，长尾延迟激增近十倍，并导致错误率飙升直至完全宕机。

自进化 Agent 如何随时间推移进行自我适应

传统的 AI Agent 在部署后处于“冻结”状态，具有固定的系统提示词和预注册工具。当遭遇新型基础设施瓶颈时，自进化 AI 架构通过在运行时动态合成针对特定情境的缓解工具、评估每次干预的遥测影响，并为未来的故障固化经过验证的防护边界，从而扩展这些基线能力。

要实现真正的自主弹性，Agent 本身必须根据“何种措施奏效、何种措施失效”的经验证据进行自我进化：

• 自主自修补（工具合成）：当 Agent 诊断出数据库连接池耗尽是由未受限的入口流量激增引起，且意识到自身缺乏内置的流量整形工具时，它会动态编写自身的中间件补丁（内存令牌桶速率限制器），验证其抽象语法树（AST），使用 Cloud KMS 进行签名，并将其直接热挂载到 API 网关中。
• 因果自我批评（评估有效与失效操作）：在故障恢复稳定后，Agent 会评估故障期间采取的每项操作对应的遥测指标增量变化。它从数学上对“重启数据库”这一反射行为进行惩罚（因为它恶化了延迟），同时对合成的速率限制器予以奖励（因为它消除了服务器错误）。
• 认知记忆与组织架构自我重构：为降低再次发生相同故障的概率，Agent 将一条经签名的运维原则写入向量内存，明确在缓存击穿期间禁止重启数据库。随后，它在 RAM 中修改自身的多 Agent 拓扑结构，在主管 Agent 下正式设立一个永久性的流量整形专家 Agent（Traffic Shaping Specialist Agent），以便在未来的突发流量到达数据库之前就完成流量整形。

---

2. 自进化 AI 的三大研究支柱

自进化云端 Agent 是当前应用研究的前沿领域，重点在于通过在 Google Cloud 上融合三大研究支柱，解决 SRE 三难困境（SRE Trilemma）（即平衡探索安全性、结构适应性与多步骤信用分配）：

1. 无服务器反事实模拟（Dream-RSI）：
挑战：在线上生产环境中直接测试未经验证的限流参数，存在误伤正常营收流量或重新诱发故障的风险。
机制：生产请求 Trace 被导出到 Google BigQuery，模拟引擎在亚毫秒级的向量化内存扫描中重放数百种候选令牌桶配置，从而在离线状态下找出帕累托最优（Pareto-optimal）策略。

2. 三层协同进化（APEX）：
挑战：单层提示词微调只能提供脆弱的权宜之计，而不受约束的代码自我修改又会导致系统失稳。
机制：Agent 的进化被解耦划分为三个受控的时间尺度：

• 第 1 层（临时热补丁）：在数秒内合成并挂载内存中间件工具，以制止正在发生的故障。
• 第 2 层（原则提炼）：将经 Cloud KMS 签名的经验教训编码写入 Agent Platform Vector Search，以便在后续收到警报时自动拦截禁忌操作。
• 第 3 层（拓扑重构）：将永久性的专家子 Agent 动态编排进活跃的多 Agent 有向无环图（DAG）中。

3. 步骤级 Shapley 归因（AgentEvolver）：
挑战：在混乱的突发事件中，多个干预操作通常按先后顺序执行；粗粒度的最终结果奖励无法隔离出究竟是哪一步修复了服务、哪一步造成了连带损害。
机制：AgentEvolver 利用干预前后的遥测指标变化量（P99 延迟、HTTP 5xx 错误率与数据库连接池饱和度），计算每个执行跨度（span）的边际 Shapley 信用值，从而将有效补丁与错误操作精准区分开来。

---

3. 四阶段参考演练：自进化的实践过程

我们基于 Gemini 平台将该架构实现为一个交互式应用。上述架构图展现了涵盖生产目标服务、Agent Development Kit (ADK) 多 Agent 控制平面、自进化引擎以及纵深防御治理流水线的闭环工作流。

交互式演示的每个阶段都展示了 Agent 自进化的不同时期：

阶段 1：流量涌入故障与 SRE 基线

• 发生情况：在 5 个请求的热身基线之后，突发的 50 个高并发闪购请求涌向 15 个冷门库存 Key。
• 系统影响：Memorystore 命中率从 100% 暴跌至 0.0%，Cloud SQL 连接池达到 100% 饱和（2/2 个活动连接被占满锁死），P99 延迟激增至 312.8~519.8 毫秒，HTTP 500 错误率蹿升至 72.7%~80.0%（40 次结账失败）。

阶段 2：ADK 快速循环分类与受控的 APEX 第 1 层热补丁

• 发生情况：遥测告警唤醒基于 Gemini 4 Argon 构建的 Agent Development Kit (ADK) 主管路由 Agent（Supervisor Router Agent）；在此之前，Model Armor 会对传入的告警负载进行安全检查以拦截提示词注入。
• 自主自修补：在诊断出连接池耗尽后，APEX 第 1 层热补丁合成器生成了一个内存令牌桶速率限制器（容量 10 个令牌，补充速率 40.0 req/s）。在挂载至 API 网关中间件之前，该补丁需通过三道自动化治理关卡：
  1. AST 沙箱访问器（Sandbox Visitor）：验证是否存在被禁止的系统模块和动态执行下沉点（sinks）。
  2. OPA Policy Controller：确认令牌补充速率符合企业最低 SLA 规定（>= 10.0 req/s）。
  3. Cloud KMS 与二进制授权：对产物摘要进行签名（EC_SIGN_P256_SHA256），并在中间件边界验证密钥来源合规性。
• 系统影响：面对第二波 50 个并发请求，HTTP 500 错误率降至 0.0%，Cloud SQL 连接池饱和度从 100% 骤降至 0%，P99 延迟恢复至 18.2 毫秒，超出限额的瞬时突发流量被安全平稳地限流抛出 HTTP 429（40 个请求被限流，10 个请求由热缓存正常处理）。

阶段 3：AgentEvolver 归因与第 2 级 HITL 内存拓扑重构

• 发生情况：AgentEvolver 信用归因引擎对故障期间的执行跨度进行审计，对重启数据库的反射操作予以惩罚（在导致延迟激增至 3,800 毫秒后扣减信用），并对令牌桶中间件补丁予以高分奖励（+0.97 信用）。
• 原则提炼：APEX 第 2 层原则提炼器对不可变运维规则进行签名并索引至 Gemini 平台 Vector Search 中：“缓存逐出击穿场景：禁用数据库重启；应采取令牌桶脱机削峰。”
• 第 2 级双人托管审批：系统没有长期依赖临时热补丁，而是提出了第 2 级结构更新方案：组建常态化的流量整形专家 Agent。UI 界面暂停执行并弹出双人托管 SRE 审批对话框，要求当班 SRE 显式批准。
• 内存级 ADK 图拓扑重构：
  在生产级云原生运行时（Google Kubernetes Engine、Cloud Run）中，容器镜像是不可变的；在实时故障排查期间修改磁盘上的源码文件会破坏容器完整性，并需要进行极具破坏性的重启。因此，在获得密码学签名的人工授权令牌后，APEX 第 3 层云拓扑适配器会在内存中执行有向无环图（DAG）的动态更新：

1. 在 RAM 中实例化流量整形专家 Agent，配备专用的速率限制、队列深度检测和令牌补充工具；
2. 将该专家 Agent 直接作为主管路由 Agent活跃子 Agent 图的并行节点追加挂载，与诊断 Agent平级（保持诊断节点不变）；
3. 更新主管的路由决策逻辑，并在内存中重新绑定线上运行的 Google ADK Runner，实现零停机、零容器重启。

阶段 4：Google BigQuery 反事实重放与 Curiosity 混沌验证

• 发生情况：故障请求 Trace 导入 Google BigQuery，BigQuery 反事实重放引擎针对不同令牌容量和补充速率执行了 400 次反事实策略模拟。
• 亚毫秒级发现与混沌验证：向量化内存追踪重放在约 0.5 毫秒内完成了全部 400 次扫描，确定了帕累托最优策略（5 个令牌容量、5.0 req/s 速率、零 5xx 错误）。最后，Curiosity 混沌红队 Agent（Chaos Red-Team Agent） 在隔离的影子沙箱中对进化后的系统进行压力测试——彻底清空 100% 缓存并发起 200 个并发请求。新连入拓扑的流量整形专家 Agent承受住了冲击，保持 0 次 HTTP 500 错误且数据库最大负载仅为 13%，从而证明了该自进化架构的韧性。

---

4. 企业级云端治理：纵深防御

治理原则： AI Agent 可以为了保障系统健康而自主进化其运维策略，但其执行权限必须受到不可变企业策略、硬件密钥来源可信性以及明确的人工托管机制的密码学约束。

为了在生产环境中安全落地自进化 Agent，我们的架构实施了严格闭环的四重关卡纵深防御托管链。每一个合成的补丁、提炼的记忆与拓扑变更……

The Emergence of Self-Evolving AI

As enterprise cloud platforms scale to serve millions of concurrent users, tightly coupled microservices introduce cascading failure modes that no static runbook or threshold alert can anticipate. During high-velocity events like a Black Friday flash sale, Site Reliability Engineers (SREs) face an unforgiving reality: diagnose noisy distributed telemetry in seconds, or suffer cascading outages costing millions of dollars per minute.

While generative AI copilots have accelerated code authoring and log summarization, AI agents struggle to reliably remediate live production outages. An on-call engineer battling connection pool starvation does not need an AI agent offering troubleshooting tips or unverified pull requests that might not fix the root cause; they need a governed, self-evolving AI system that can:

1. Diagnose distributed root causes across cache and database tiers in real time,
2. Self-patch by synthesizing, verifying, and mounting executable mitigation tools into live server middleware on the fly,
3. Self-critique by computing the marginal credit of every action to separate effective fixes from harmful reflexes,
4. Remember by distilling cryptographically signed operational invariants into semantic memory to reduce the chance of identical failures recurring, and
5. Re-architect by dynamically rewiring its own multi-agent hierarchy in memory with human sign-off to absorb future surges.

This paradigm is Self-Evolving AI.

In this Google Cloud research reference architecture, codenamed Project Chrysalis , we demonstrate how three foundational research breakthroughs combine into a unified, production-grade architecture that can run on Gemini platform and the Agent Development Kit (ADK). Across simulated Black Friday cache-stampede outages, this governed architecture cut time-to-first-mitigation from 15 to 45 minutes down to 1.8 seconds, reduced HTTP 5xx error rates from 72.7% to 80.0% down to 0.0%, dropped P99 latency from 519.8 ms to 18.2 ms, and evaluated 400 counterfactual traffic policies in BigQuery in ~0.5 ms.

Crucially, this self-evolving architecture does not operate as an unconstrained "black box". To ensure the agent remains secure, reliable, aligned, and contained against rogue behavior, every autonomous action is cryptographically bounded by Google Cloud's defense-in-depth security stack: Model Armor to sanitize incoming telemetry against prompt injections and data leaks; Google Cloud Key Management Service (Cloud KMS) and Binary Authorization middleware to cryptographically sign synthesized patches and block unverified code from mounting; Open Policy Agent (OPA) Policy Controller to enforce strict SLA guardrails that prevent self-inflicted outages; and Dual-Custody Human-in-the-Loop (HITL) governance to require explicit SRE sign-off before any permanent changes are made to the agent hierarchy.

---

1. The Incident: The Black Friday Cache Stampede & Why Agents Must Self-Evolve

To see why static playbooks and advisory copilots fail under pressure, consider a classic distributed failure cascade:

1. The Ingress Surge: A flash promotion drives a sudden burst of concurrent checkout requests across cold inventory keys.
2. The Cache Eviction: Google Cloud Memorystore (Redis) evicts cold keys under strict Least Recently Used (LRU) memory limits, collapsing the cache hit ratio to zero.
3. Connection Pool Starvation: Uncached traffic cascades directly into Google Cloud SQL. Once the PostgreSQL connection pool saturates, incoming queries queue past timeout thresholds and the service begins failing with widespread HTTP 500 errors.
4. The Fatal "3:00 AM Panic Reflex": Under operational stress, both human responders and naive LLMs instinctively try to restart the database connection pool. In a live stampede, dropping connections locks the database kernel, spikes tail latency nearly tenfold, and drives error rates toward total outage.

How a Self-Evolving Agent Adapts Over Time

Conventional AI agents are frozen at deployment with fixed system prompts and pre-registered tools. When a novel infrastructure bottleneck strikes, a self-evolving AI architecture extends those baseline capabilities by synthesizing situation-specific mitigation tools at runtime, scoring the telemetry impact of every intervention, and codifying verified guardrails for future incidents.

For true autonomous resilience, the agent itself must evolve based on empirical evidence of what worked and what failed :

• Autonomous Self-Patching (Tool Synthesis): When the agent diagnoses that database starvation is driven by an unthrottled ingress surge and realizes it lacks a built-in traffic-shaping tool, it writes its own middleware patch on the fly (an in-memory token-bucket rate limiter), validates its Abstract Syntax Tree (AST), signs it with Cloud KMS, and hot-mounts it directly into the API Gateway.
• Causal Self-Critique (Evaluating What Worked vs. What Failed): After stabilizing the outage, the agent evaluates the telemetry delta of every action taken during the incident. It mathematically penalizes the database restart reflex because it worsened latency, while rewarding its synthesized rate limiter for eliminating server errors.
• Cognitive Memory & Organizational Self-Rewiring: To reduce the likelihood of a repeat outage, the agent writes a signed operational principle into vector memory that blocks database restarts during cache stampedes. It then modifies its own multi-agent topology in RAM , commissioning a permanent Traffic Shaping Specialist Agent under the supervisor agent so future surges are shaped before they reach the database.

---

2. The Three Research Pillars of Self-Evolving AI

Self-evolving cloud agents are an active area of applied research focused on resolving the SRE Trilemma (balancing exploration safety, structural adaptability, and multi-step credit assignment) by unifying three research pillars on Google Cloud:

1.Serverless Counterfactual Simulation (Dream-RSI):
Challenge: Testing unvetted rate-limiting parameters in live production risks throttling legitimate revenue or re-triggering outages.
Mechanism: Production request traces are exported to Google BigQuery, where the engine replays hundreds of candidate token-bucket configurations in sub-millisecond vectorized memory sweeps to identify Pareto-optimal policies offline.

2. Tri-Level Co-Evolution (APEX):
Challenge: Single-layer prompt tuning provides fragile band-aids, while unconstrained code self-modification risks system instability.
Mechanism: Agent evolution is separated into three governed timescales:

• Layer 1 (Ephemeral Hotpatching): Synthesizes and mounts in-memory middleware tools to halt active outages in seconds.
• Layer 2 (Principle Distillation): Encodes Cloud KMS-signed lessons into Agent Platform Vector Search so contraindicated actions are blocked on future alerts.
• Layer 3 (Topology Rewiring): Commissions permanent specialist sub-agents into the active multi-agent Directed Acyclic Graph (DAG).

3. Step-Level Shapley Attribution (AgentEvolver):
Challenge: During chaotic incidents, multiple interventions occur in sequence; coarse outcome rewards cannot isolate which step healed the service and which caused collateral damage.
Mechanism: AgentEvolver computes marginal Shapley credit across each execution span using pre- and post-intervention telemetry deltas (P99 latency, HTTP 5xx rate, and database pool saturation), isolating beneficial patches from erroneous ones.

---

3. The 4-Phase Reference Walkthrough: Self-Evolution in Action

We implemented this architecture as an interactive application on Gemini platform. The diagram above illustrates the closed-loop workflow across the Production Target Service , Agent Development Kit (ADK) Multi-Agent Plane , Self-Evolution Engine , and Defense-in-Depth Governance Pipeline.

Each phase of the interactive demonstration showcases a distinct stage of the agent's self-evolution:

Phase 1: Ingress Outage & SRE Baseline

• What Happens: Following a 5-request warm baseline, a 50-request concurrent flash-sale surge hits 15 cold inventory keys.
• System Impact: Memorystore hit ratio drops from 100% to 0.0% , the Cloud SQL connection pool hits 100% saturation (2/2 active connections locked), P99 latency spikes to 312.8 to 519.8 ms , and HTTP 500 errors jump to 72.7% to 80.0% (40 failed checkouts).

Phase 2: ADK Fast-Loop Triage & Governed APEX Layer 1 Hotpatch

• What Happens: Telemetry alerts wake the Agent Development Kit (ADK) Supervisor Router Agent powered by Gemini 4 Argon, after Model Armor inspects the incoming alert payload to block prompt injections.

• Autonomous Self-Patching: Diagnosing pool exhaustion, the APEX Layer 1 Hotpatch Synthesizer generates an in-memory token-bucket rate limiter (10-token capacity, 40.0 req/s replenishment). Before mounting into the API Gateway middleware, the patch passes three automated governance gates:

  1. AST Sandbox Visitor verifies that forbidden system modules and dynamic execution sinks are absent.
  2. OPA Policy Controller confirms the replenishment rate meets the enterprise minimum SLA (>= 10.0 req/s).
  3. Cloud KMS & Binary Authorization signs the artifact digest (EC_SIGN_P256_SHA256) and verifies key provenance at the middleware boundary.

• System Impact: Across a second 50-request surge, HTTP 500 errors drop to 0.0% , Cloud SQL pool saturation drops from 100% to 0% , P99 latency recovers to 18.2 ms , and excess burst traffic is cleanly shed via HTTP 429 (40 requests throttled, 10 served from warm cache).

Phase 3: AgentEvolver Attribution & Tier-2 HITL In-Memory Topology Rewiring

• What Happens: The AgentEvolver Credit Attribution Engine audits the incident execution spans, penalizing the database restart reflex ( after spiking latency to 3,800 ms) and rewarding the token-bucket middleware patch (+0.97 credit).
• Principle Distillation: The APEX Layer 2 Principle Distiller signs and indexes an immutable operational rule into Gemini Platform Vector Search: "Cache eviction stampede: database restart contraindicated; apply token-bucket load shedding."
• Tier-2 Dual-Custody Approval: Rather than relying on temporary hotpatches, the system proposes a Tier-2 structural update: commissioning a permanent Traffic Shaping Specialist Agent. Execution pauses in the UI and presents a Dual-Custody SRE Approval Modal requiring explicit sign-off from the on-call SRE.
• In-Memory ADK Graph Rewiring:
  In production cloud runtimes (Google Kubernetes Engine, Cloud Run), container images are immutable; rewriting source files on disk during a live outage violates container integrity and requires disruptive restarts. Instead, once issued a cryptographic human-approval token, the APEX Layer 3 Cloud Topology Adapter executes an in-memory Directed Acyclic Graph (DAG) update:

1. Instantiates the Traffic Shaping Specialist Agent in RAM with dedicated rate-limiting, queue-depth inspection, and token-replenishment tools.
2. Appends the specialist agent directly to the Supervisor Router Agent's active sub-agent graph as a parallel peer alongside the Diagnostic Agent (leaving the diagnostic node unmodified).
3. Updates the supervisor's routing instructions and re-binds the live Google ADK Runner in memory with zero downtime and zero container restarts.

Phase 4: Google BigQuery Counterfactual Replay & Curiosity Chaos Verification

• What Happens: Incident request traces flow into Google BigQuery, where the BigQuery Counterfactual Replay Engine executes 400 counterfactual policy simulations across token capacities and replenishment rates.
• Sub-Millisecond Discovery & Chaos Proof: Vectorized in-memory trace replay completes all 400 sweeps in ~0.5 milliseconds , identifying the Pareto-optimal policy (5-token capacity, 5.0 req/s rate, zero 5xx errors). Finally, the Curiosity Chaos Red-Team Agent stress-tests the evolved system in an isolated shadow sandbox by purging 100% of the cache and firing 200 concurrent requests. The newly wired Traffic Shaping Specialist Agent absorbs the surge with 0 HTTP 500 errors and 13% max DB load , certifying the architecture as resilient.

---

4. Enterprise Cloud Governance: Defense-in-Depth

The Governing Principle: An AI agent may autonomously evolve its operational tactics to defend system health, but its authority to execute is cryptographically bounded by immutable enterprise policies, hardware key provenance, and explicit human custody.

To deploy self-evolving agents safely in production, our architecture enforces an unbroken four-gate defense-in-depth chain of custody. Every synthesized patch, distilled memory, and topology update must clear four sequential verification boundaries:

1. Ingress & AST Sandbox Sanitization: Before any telemetry alert enters the agent reasoning loop, Model Armor screens payloads for adversarial prompt injections and redacts sensitive data. When the agent synthesizes mitigation code, an AST Sandbox Visitor parses the syntax tree before compilation to block unauthorized system calls and arbitrary execution sinks.
2. Cryptographic Provenance & Binary Authorization: Validated code artifacts and distilled principles are signed using Google Cloud KMS asymmetric keys. API Gateway middleware enforces a strict Binary Authorization check, refusing to mount any in-memory rate limiter unless its cryptographic signature verifies against the enterprise key ring.
3. Policy-as-Code & Dual-Custody HITL Escalation: An Open Policy Agent (OPA) Policy Controller enforces rigid SLA guardrails (such as requiring replenishment rates >= 10.0 req/s so an agent cannot accidentally self-inflict a denial of service). Higher-risk Tier-2 structural updates, such as rewiring the ADK agent graph, trigger a Dual-Custody Human-in-the-Loop Gate , pausing execution until an authorized SRE approves the change with a cryptographic token.
4. Tamper-Evident Auditability: Every alert sanitization, AST check, KMS signature, policy verdict, and HITL approval emits structured records to Google Cloud Logging, guaranteeing end-to-end compliance and forensic traceability.

---

5. Empirical Results in the demo simulation

Across automated evaluations of Black Friday cache-stampede workloads, the self-evolving architecture delivered decisive operational gains over both manual triage and static advisory copilots:

6. Explore the Reference Architecture on GitHub & Google Cloud

Explore the reference implementation and Google Cloud documentation below: