商业产业 Hacker News AI 2026-08-09T21:04 正文 ok 2718 words

Show HN:Pacific Slate——自托管、模型无关的多智能体 AI 助手

英文标题:🔶 Show HN: Pacific Slate: a self-hosted, model-agnostic multi-agent AI assistant

Pacific Slate 是一个运行在自有租用服务器上的自托管多智能体 AI 助手架构,旨在持续汇集、筛选并保留用户私有数据,而非作为商业 SaaS 产品销售。系统采用 Google ADK 构建一棵包含 1 个调度者和 7 个专业 Agent 的多智能体树,并依据任务类型在不同模型之间进行路由、成本控制与故障降级。其知识库约含 1.4 万份文档和 19.6 万个段落,结合混合检索、连续性图谱、语义记忆和分层上下文,同时要求 Agent 回查源文件而非盲信记忆。对开发者而言,该项目展示了如何通过 MCP、OpenAI 兼容接口、Redis 事件流和可替换模型层,构建以数据主权、可审计性、可靠性及成本上限为核心的个人 AI 工作环境。
打开原文相关 3 · 机会 1
media23.0 KB media577.6 KB media4.8 KB media7.2 KB

中文

Pacific Slate

我读 MBA 第一年的时候,一边申请实习,一边要应对海量信息,同时还得兼顾妻子和两个孩子;而且每周换一个 AI 程序时,我都得重新解释一遍自己的完整情况。我想要一个能够自行更新、过滤垃圾内容,并保有数据矿藏的系统。

我搭建这个网站,是为了向任何想做类似事情的人分享这套结构。它并不是一项可购买的服务或产品,而是你可以交给自己的 LLM、并作为实施指南使用的东西。

它运行在一台租用服务器上:这是一台托管各项服务、并保存我关心文件之加密备份的专用机器。我负责设计、选择组件并进行日常运行(技术细节位于页面底部的折叠章节中)。

它能做什么

  • 从我的信息源自行更新,筛除噪声,并在我私有拥有的数据库中维护数据。
  • 使用多智能体树,将工作路由到最适合该任务的模型。
  • 记住我的选择和偏好,并从过去的结果中学习(将经验教训应用于新任务或情境)。
  • 标注来源并维护其所有操作的日志,从而支持可追溯性与问责。
  • 查验源文件,而不是相信它所记得的内容。
  • 集成至我已经在使用的工具和服务中,因此它适应我的工作流,而不是迫使我改变工作流。

哪些内容会被上报:每次定时运行都会从已连接的信息源中收集候选条目,包括消息、邮件、日历、GitHub、Hacker News、Reddit、arXiv、RSS 订阅源、天气、市场和地震信息流。每个条目都会按相关性、重要性和新颖性评分,之后才可能进入简报;一次运行会分为两种结果。通过筛选的条目会在简报中被上报,说明发生了什么变化以及哪些事项需要关注。未通过筛选的条目会进入丢弃日志——被记录而非直接丢弃,以便检查过滤器。

我不再构建的东西

我很快学会了,相比新颖性,应优先考虑耐用性、实用性和生产力。这个系统不是产品,也不是研究项目。它是我每天使用的工具;当我不在场或没有时间维护它时,它也必须能够运行。我最初为访问它构建了移动端和桌面端客户端,但它们需要投入的工作量超出了我能舒适承担的范围。我删除了它们,现在通过 AI 开发中常见的工具(MCP、插件、hooks)访问它。让系统具备面向未来的能力一直很重要,因此设计刻意保持服务无关性,使其能够灵活适配当时最合适的方案。

请求如何被处理

由八个 Agent 完成工作:一个负责读取请求并进行路由,另外七个则是负责研究、代码、分析、审查及相关工作的专家。每个 Agent 都运行在为该工作选定的模型上,因此简短查询和长篇分析不会消耗相同成本,也无需等待相同的处理机制。

回答会携带其来源,因此每项声明都可追溯;同时也会附带调用成本,使支出能够在发生时保持可见,而非等到月底才看到。

界面

从公开演示中截取的 Pacific Slate 工作区。一项问题已被回答为深色画布上的可移动卡片,卡片标有生成内容的模型、成本和耗时;旁边还有一份简短简报及建议的后续操作。页眉显示:Pacific Slate、demo、sample data、no backend。

来自公开演示的工作区。回答以可在画布上移动的卡片形式呈现,而不是一条很长的对话线程;每张卡片都标有使用的模型、成本和耗时。

演示:pacslate.com/demo。 使用样本数据的真实界面。

  • 画布。 输入提示词,或观看它运行。打开
  • 监控。 一个仪表板。地震数据和新闻标题为实时数据,在浏览器中获取;市场、飞机以及任何个人数据均标记为样本数据。打开

什么是我的,什么是租用的

会持续积累的部分留在我的服务器上。模型则是租用的。

数据存放位置。我所连接的文档、笔记、代码和信息流,会被摄入、去重,并筛查凭据和个人数据,然后保存在我自己的服务器上:包括知识语料库、连续性图谱、工具以及路由和支出限制。这些材料在此累积,可导出、可删除,且不会用于训练。其下方通过一个标注的标准接口连接租用模型:请求在已加载上下文后发出,回答则连同其来源和成本返回。更换模型只是一次配置变更,以上部分均不会迁移。

模型是我唯一无法控制的部分,因此被视为可替换组件。替换为更好的模型只是一次配置变更。

所存储的内容可导出、可删除,且不会用于训练。发出的请求仅限于零数据保留端点;提供商被配置为不使用这些请求进行训练或公开它们(账户设置,2026 年 8 月)。如果任何内容都不应离开服务器,工作可以指向运行在服务器上的模型。

我持有持续积累的内容,我决定请求发送到哪里,而且其中任何内容都不会成为供应商的资产。

开源,以及我支付的费用

几乎所有组件都是开源的,由我在自己的服务器上组装和运行,而非作为服务租用。只要开源替代方案能够胜任,我就切换到它并保留数据。我仍需付费的部分是模型,以及一项托管记忆服务;后者始终可替换,且不是系统的记录系统。

构建过程

尽管计算机科学和软件工程一直令我着迷,但我此前没有亲自开发代码的背景。这个系统的开发得益于与目前大量在线工具和服务的协作。架构本身均为独立创建:包括存在哪些组件、它们如何连接、各自运行在什么环境中、某个部分失败时如何表现,以及各模块或平台被允许“花费”的额度(预算)。我以 AI 原生方式构建它:由我进行规格定义和审查,而编码 Agent 编写了大部分代码。

它自 2026 年初开始运行,我每天都在使用。技术章节中描述的大部分内容,来自运行系统的过程而非事先规划。例如,回退层之所以存在,是因为框架自身的回退设置后来被证实根本不起作用。另一个例子是,每个回答上的模型标签,是为了将行为回归追溯到日志中不可见的一次模型替换。生产环境中遇到的大多数问题并非 bug,而是设计缺口;只有通过实际运行系统,我才发现了它们。

由于我是在工作和读研期间构建 Pacific Slate,空闲时间(或者说缺少空闲时间)比其他任何因素都更深刻地塑造了其设计。它必须能在我不在的时间里发挥作用:定时运行、可自行合并的低风险依赖项和安全更新,以及降级后仍能维持工作状态、而非等待我处理的故障机制。


技术细节

本页及示例配置中的模型名称仅作说明;该设计与模型无关,模型阵容会轮换。统计数字截至 2026 年 8 月,并已根据运行中的系统进行核验。

1 · Agent 与记忆 · 设计,而非组装

这是一个经过设计的多智能体系统,而不是带插件的聊天模型。

一个构建于 Google 的 Agent Development Kit(ADK) 之上的多智能体树:根节点为一名调度者,另有七名专家(编码员、研究员、分析师、生产力 Agent、审查员、评估员,以及一个限定服务于编码员的研究子 Agent)。

  • 成本与适配度。 每种角色均映射至最适合其工作、且价格匹配的模型。
  • 独立审查。 评估员所使用的模型家族不同于它评分的输出所属 Agent。不同权重不保证带来不同偏见,但总比自我评分更好。审查员是一个独立实例,依据策略被置于写入路径之外。
  • 最小权限。 工具按角色通过显式允许列表限定范围。研究员无法访问基础设施。审查员按职责设定为只读。生产力 Agent 的凭据相互隔离,因此该处发生故障不会拖垮调度者。
  • 隔离。 限流或崩溃只会停留在一个专家 Agent 内部。
  • ADK 对子 Agent 实施单父节点约束,因此编码员的研究分支与独立研究员是不同实例,尽管二者共享模型和工具。
  • 专家 Agent 将发现发布到 Redis 事件流,其他 Agent 将其作为环境上下文读取。较长的异步任务会交给后台编排器,而不会占用同步路径。

模型独立性。 该底层能力通过两种方式暴露:一个 MCP 网关(通过 Model Context Protocol 提供工具和记忆,可挂载至任何支持 MCP 的客户端)和一个 OpenAI 兼容端点。模型发生变化时,记忆、数据、工具、路由和验证均不会迁移。

四层记忆。

层级 功能
知识语料库 约 14,000 份文档、约 196,000 个段落。混合检索:关键词加向量相似度
连续性图谱 随时间关联相关对话、决策和主题
语义记忆 用于跨会话回忆的持久事实,包含每晚执行的协调与去重任务
分层上下文 始终加载的核心内容保持精简;其余内容按需检索

两条规则约束着所有这些内容。首先,在任何模型运行之前,由一次无模型调用的确定性相关性处理决定加载什么内容。其次,被回忆起的记忆被视为线索而非事实:如果记忆显示某供应商在 7 月续约,Agent 会调取源文件并据此行动。

主动任务。 定时例程会从多个信息源拉取内容,并在候选条目进入简报前,按相关性、重要性和新颖性为其评分。被丢弃的条目会被记录而非静默删除,因此可以审计过滤器。在任何内容被存储之前,整合阶段会执行去重和规范化,并在摄入途中筛查凭据和个人数据。

运行原则。

  • 算法优先。 默认使用确定性方式进行数据访问和分类:正则表达式、索引、SQL、集合成员关系。仅将模型用于不可约简的部分。复杂度评分器、工具搜索和记忆相关性处理都遵循这一原则。
  • 先让故障可见,再让故障代价变低。 每次可靠性修复都将护栏与追踪记录配对,使同一类问题不可能在无人察觉的情况下再次发生。
  • 韧性优先于峰值能力。 两者冲突时,优先保障可用性。
  • 将成本视为设计约束。 真正的上限会迫使人们就模型、上下文规模,以及何时根本不该使用模型作出诚实选择。

参见:examples/model-routing.example.yaml 了解路由结构 · 在演示中查看每张卡片上的模型和成本标签。

2 · 集成 · 将模型、工具、数据和界面整合为一个系统

模型提供商、工具、数据、记忆和前端被整合为一个环境,并可通过标准接口访问。

请求路径:

  1. 画布是一个 Next.js 单页应用,它会打开流式连接并发送提示词。流式传输使用 基于 Server-Sent Events 的 AG-UI 协议,因此 token、工具调用和模型解析事件会在同一条流中到达。
  2. 后端是一个 Starlette 服务,为画布暴露 AG-UI 端点,并为其他客户端暴露 OpenAI 兼容端点。
  3. 调度者对请求进行分类,并直接回答,或将其转交给专家 Agent。
  4. 专家 Agent 加载上下文,并通过 MCP 调用工具。
  5. 弹性模型层 为该调用解析模型,应用成本感知降级和健康状态感知回退(第 3 节),流式输出补全结果,并计量成本。
  6. 回答作为标有模型和成本的卡片流式返回。持久事实会被写入记忆。

由此形成两个速度层级:单次模型调用的内联回答,以及专家 Agent 执行其自身工具循环的委派运行。

Pacific Slate 架构

系统图。点击放大。

工具层。 约二十余个工具服务器(记忆、检索、代码执行、git、信息流、基础设施)位于一个采用 token 身份验证的网关之后,使用基于可流式 HTTP 的 FastMCP,每台服务器均为独立进程。

  • 注册,而不是重连。 新能力意味着注册一台服务器。代价是额外一跳以及网关处的单一故障域,因此网关拥有自身的健康检查,整个服务器集群也会被轮询健康状态。
  • 搜索,而不是加载。 在每个 Agent 的上下文中携带所有工具 schema 成本高昂且会分散注意力,因此网关提供对工具目录的 BM25 搜索。MCP 规范仅列出工具;该搜索能力是额外添加的。
层级 选择
Agent 框架 Google ADK,多智能体树
后端 Starlette,基于 SSE 的 AG-UI 加 OpenAI 兼容端点
前端 Next.js,空间画布上的流式结果卡片
Agent 间总线 Redis 事件流
工具 约 24 个 MCP 服务器位于一个网关之后,BM25 工具搜索
调度者 具备原生工具调用能力的长上下文模型
编码员 代码专用模型,大上下文
研究员 快速长上下文模型,单次处理
分析师 大型混合专家推理模型
审查员、评估员 独立于代码编写者,使用不同模型家族进行评分
记忆 混合检索语料库、连续性图谱、语义记忆、分层上下文
基础设施 Docker、四个分段网络层、Cloudflare Tunnel、沙箱执行
可靠性 按角色划分的回退链、健康状态跟踪器、首 token 看门狗、优雅降级
成本 按请求计量,设定月度上限,fu

English / 原文

Pacific Slate

I was in the first year of an MBA, trying to manage a firehose of information while applying to internships and keeping up with my wife and two kids, and I kept re-explaining my whole situation to whatever AI program I was using that week. I wanted one that updated itself, filtered the slop, and kept the data mine.

I built this website to share the structure for anyone looking to do something similar. This isn’t meant to be a service or product to purchase, it’s something you can hand off to your own LLM and use as a guide.

It runs on a rented server, a dedicated machine that hosts the services and holds encrypted backups of the files I care about. I designed it, selected the parts, and run it day to day (technical details in the collapsed sections at the bottom).

What it does

  • Updates itself from my sources, sorts through the noise, and maintains the data in my privately owned database.
  • Uses a multi-agent tree to route work to the model best suited for it.
  • Remembers my choices and preferences, and learns from past outcomes (applying lessons learned to new tasks or situations).
  • Cites sources and maintains logs of everything it does, allowing for traceability and accountability.
  • Checks the source document rather than trusting what it remembers.
  • Integrates into the tools and services I already use, so it adjusts to my workflow rather than forcing me to change it.

What gets raised: each scheduled run collects candidate items from connected sources, among them messages, mail, calendar, GitHub, Hacker News, Reddit, arXiv, RSS feeds, weather, markets, and seismic feeds. Every item is scored for relevance, materiality, and novelty before it can enter a brief, and the run splits into two outcomes. Items that pass are raised in the brief, which says what changed and what needs attention. Items that do not pass go to a discard log, recorded rather than dropped, so the filter can be checked.

What I stopped building

I learned quickly to prioritize durability, utility, and productivity over novelty. The system is not a product, and it is not a research project. It is a tool I use every day, and it has to work when I’m not there or don’t have the time to maintain it. I had originally built mobile and desktop clients to reach it, but they were more work than I could comfortably invest. I deleted them and now reach it from the tools (MCP, plugins, hooks) commonplace throughout AI development. It’s been important to futureproof the system, and the design is intentionally service-agnostic so it can flex to whatever is best at the time.

How a request is handled

Eight agents do the work: one that reads the request and routes it, and seven specialists for research, code, analysis, review, and related work. Each runs on a model chosen for that job, so a short lookup and a long analysis do not draw the same cost or wait on the same machinery.

Answers carry their sources, so a claim can be traced, and the cost of the call, so spending stays visible as it happens rather than at the end of the month.

The interface

The Pacific Slate workspace, captured from the public demo. A question has been answered into movable cards on a dark canvas, labeled with the model that produced it, the cost, and the elapsed time, alongside a short brief and suggested follow-ups. The header reads: Pacific Slate, demo, sample data, no backend.

The workspace, from the public demo. Answers arrive as cards that can be moved around rather than as one long thread, each labeled with the model used, the cost, and the time taken.

Demo:pacslate.com/demo. The real interface on sample data.

  • Canvas. Enter a prompt or watch it run. Open
  • Monitor. A dashboard. Seismic data and headlines are live, fetched in the browser; markets, aircraft, and anything personal are labeled sample data. Open

What is mine and what is rented

The parts that accumulate stay on my server. The model is rented.

Where the data sits. What I connect, documents notes code and feeds, is ingested, de-duplicated, and screened for credentials and personal data, then held on my own server: the knowledge corpus, continuity graph, tools, and the routing and spending limits. That material accumulates there and is exportable, deletable, and not used for training. Below it, across a labeled standard interface, is the rented model: the request goes out with context already loaded, and the answer returns with its sources and cost. Replacing the model is a configuration change, and nothing above it moves.

The model is the one part I do not control, so it is treated as replaceable. Substituting a better one is a configuration change.

What is stored is exportable and deletable, and is not used for training. Requests that go out are restricted to zero-data-retention endpoints, at providers configured not to train on them or publish them (account settings, August 2026). Where nothing should leave the server, work can be pointed at a model running on it.

I hold what accumulates, I decide where requests go, and none of it becomes a vendor’s asset.

Open source, and what I pay for

Almost all of it is open source, assembled and run on my own server rather than rented as a service. Where an open-source equivalent did the job, I switched to it and kept the data. What I still pay for is the model, and one hosted memory service that stays swappable and is not the system of record.

Building it

While computer science and software engineering have always been intriguing, I didn’t have any background developing code myself. The development of this system has been possible through working in partnership with the numerous tools and services currently available online. The architecture itself has all been created independently: which components exist, how they connect, what each runs on, how it behaves when a part fails, and what it is allowed to ‘spend’ (the budget for each module or platform). I built it AI-natively, specifying and reviewing while coding agents wrote most of the code.

It has run since early 2026 and I use it daily. Most of what the technical sections describe came out of operating it rather than planning it. For example, the fallback layer exists because the framework’s own fallback setting turned out to do nothing. Another instance, the model label on every answer exists as a way to backtrace a behavioral regression to a model swap that was invisible in the logs. Most of the issues I hit in production were not bugs but design gaps, and it was only through operating the system that I discovered them.

Because I built Pacific Slate while working and going to grad school, my free time (or lack thereof) shaped the design more than anything else. It had to be useful in the hours I wasn’t there: scheduled runs, low-risk dependencies and security updates that merge on their own, and failures that degrade to a working state rather than waiting for me.


Technical detail

Model names on this page and in the example config are illustrative; the design is model-agnostic and the roster rotates. Counts are current as of August 2026 and were checked against the running system.

1 · Agents and memory · the design, not the assembly

A designed multi-agent system, not a chat model with plugins.

A multi-agent tree on Google’s Agent Development Kit (ADK) : one operator at the root plus seven specialists (coder, researcher, analyst, productivity, reviewer, evaluator, and a research sub-agent scoped to the coder).

  • Cost and fit. Each role maps to the model best suited and priced for its work.
  • Independent review. The evaluator runs on a different model family than the agents whose output it scores. Different weights are not guaranteed different biases, but it beats self-scoring. The reviewer is a separate instance kept off the write path by policy.
  • Least privilege. Tools are scoped per role by explicit allow-list. The researcher cannot reach infrastructure. The reviewer is read-only by charter. The productivity agent’s credentials are isolated, so a failure there cannot take down the operator.
  • Containment. A rate-limit or crash stays inside one specialist.
  • ADK enforces a single-parent constraint on sub-agents, which is why the coder’s research arm is a separate instance from the standalone researcher despite sharing a model and tools.
  • Specialists publish findings to a Redis event stream that peers read as ambient context. Longer asynchronous jobs hand off to a background orchestrator rather than occupying the synchronous path.

Model independence. The substrate is exposed two ways: an MCP gateway (tools and memory over the Model Context Protocol, mountable by any MCP-capable client) and an OpenAI-compatible endpoint. Memory, data, tools, routing, and verification do not move when the model does.

Memory, in four layers.

Layer Function
Knowledge corpus ~14,000 documents, ~196,000 passages. Hybrid retrieval: keyword plus vector similarity
Continuity graph Links related conversations, decisions, and topics over time
Semantic memory Durable facts for cross-session recall, with a nightly reconciliation and de-duplication job
Tiered context Always-loaded core kept small; the rest retrieved on demand

Two rules govern all of it. A deterministic relevance pass decides what to load before any model runs, with no model call. And recalled memory is treated as a lead rather than a fact: if memory says a vendor renews in July, the agent pulls the source document and acts on that.

Proactive jobs. Scheduled routines pull from many sources and score each candidate item for relevance, materiality, and novelty before it can enter a brief. Discarded items are logged rather than dropped silently, so the filter can be audited. A consolidation stage de-duplicates and normalizes before anything is stored, and screens for credentials and personal data on the way in.

Operating principles.

  • Algorithm first. Default data access and classification to something deterministic: regex, an index, SQL, set membership. Use a model only for the irreducible part. The complexity scorer, the tool search, and the memory relevance pass are all applications of this.
  • Make failure visible, then cheap. Each reliability fix pairs a guardrail with a trace, so the same class of problem cannot recur unseen.
  • Resilience over peak capability. Where the two conflict, uptime wins.
  • Cost as a design constraint. A real ceiling forces honest choices about model, context size, and when to use a model at all.

See: examples/model-routing.example.yaml for the routing structure · the model and cost label on each card in the demo.

2 · Integration · models, tools, data, and interface as one system

Model providers, tools, data, memory, and a frontend are integrated into one environment reachable over standard interfaces.

Request path:

  1. The canvas, a Next.js single-page app, opens a streaming connection and posts the prompt. Streaming uses the AG-UI protocol over Server-Sent Events , so tokens, tool calls, and model-resolution events arrive on one stream.
  2. The backend is a Starlette service exposing the AG-UI endpoint for the canvas and an OpenAI-compatible endpoint for other clients.
  3. The operator classifies the request and either answers inline or hands off to a specialist.
  4. The specialist loads context and calls tools over MCP.
  5. The resilient model layer resolves the model for that call, applies cost-aware downgrades and health-aware fallback (section 3), streams the completion, and meters cost.
  6. The answer streams back as a card labeled with model and cost. Durable facts are written to memory.

Two speed tiers result: inline answers in a single model call, and delegated runs where a specialist works its own tool loop.

Pacific Slate architecture

System diagram. Click to enlarge.

Tool layer. About two dozen tool-servers (memory, retrieval, code execution, git, feeds, infrastructure) behind one token-authenticated gateway, FastMCP over streamable HTTP, each server its own process.

  • Register, do not rewire. New capability means registering a server. The cost is an extra hop and a single failure domain at the gateway, which is why the gateway has its own health check and the fleet is health-polled.
  • Search, do not load. Carrying every tool schema in every agent’s context is expensive and dilutes attention, so the gateway exposes a BM25 search over the tool catalog. The MCP specification only lists tools; this search is an addition.
Layer Choice
Agent framework Google ADK, multi-agent tree
Backend Starlette, AG-UI over SSE plus OpenAI-compatible endpoint
Frontend Next.js, streaming result cards on a spatial canvas
Inter-agent bus Redis event streams
Tooling ~24 MCP servers behind one gateway, BM25 tool search
Operator long-context model with native tool calling
Coder code-specialized model, large context
Researcher fast long-context model, single pass
Analyst large mixture-of-experts reasoning model
Reviewer, evaluator independent of the code-writers, scored on a different model family
Memory hybrid-retrieval corpus, continuity graph, semantic memory, tiered context
Infrastructure Docker, four segmented network tiers, Cloudflare Tunnel, sandboxed execution
Reliability per-role fallback chains, health tracker, first-token watchdog, graceful degrade
Cost per-request metering to a monthly ceiling, full call and swap tracing

See: the diagram above · the demo, which is the same canvas and protocol running against scripted events.

3 · Reliability and cost · the part that took the most iteration

It stays up, stays within budget, and checks its own output, and each of those mechanisms is observable.

Ninety days of traces as of August 2026: 13,053 runs, 26,515 model calls, nine models serving traffic. The median model call completes in 3.1 seconds; the 90th percentile is 14 seconds.

Cost control, applied before the call. A deterministic scorer rates prompt complexity 1 to 5 from word count, keyword classes, code markers, role weight, and conversation depth, with no model involved. That score and current budget state select the tier:

Monthly budget used Behavior
Under 80% Every role on its assigned model
80 to 95% Trivial tasks downgraded
95 to 100% All but essential high-complexity work downgraded
At 100% Non-essential calls blocked

Spend is tracked per request against a hard monthly ceiling, with atomic writes so the ledger survives a restart.

Fallback. In the ADK version I built on, the model wrapper called the provider directly and ignored the underlying library’s fallback configuration, so that setting did nothing. I wrapped the model layer with one that implements it.

  • A health tracker records rate-limited models and applies an escalating cooldown that grows with repeat offenses. Later calls skip a cooling-down model rather than hitting the same wall.
  • A time-to-first-token watchdog bounds only the first response of an attempt. A model that stalls without emitting trips it and the call moves to the next model. Because nothing was yielded, there is no duplicate output. Once a stream starts it runs unbounded, with a longer outer timeout as backstop.
  • A fallback model that cannot accept the tool definitions is skipped as a compatibility miss, not reported as a failure.
  • If the chain is exhausted, the layer returns a message stating that specialist is unavailable rather than crashing the run group.

Chains terminate on different models per role, so no two roles share a last resort. Any workload can be pinned to a model on the server as a final option.

A failure worth recording. Agents intermittently produced worse output, with more hedging and occasional refusals, and nothing in the error logs. The cause was the cost router: under budget pressure it substituted a cheaper model, which returned success and behaved differently. The fix was to surface the substitution. Every swap now appears on the result card and in tracing, so the first question on a behavioral regression is which model served the call.

See: examples/model_fallback.py for the fallback pattern, with the reasoning in the comments · examples/model-routing.example.yaml for budget structure.

4 · Operations and limits · deployment, and what this is not

This runs as versioned, monitored infrastructure with a gated deployment path.

  • Segmented. Docker services across four network tiers (public-facing, internal, outbound, isolated sandbox) plus purpose-built networks, so the blast radius of one service is bounded. Ingress is through a Cloudflare Tunnel; the origin is not directly exposed.
  • Sandboxed execution. Agent-run code executes in an isolated sandbox with an ephemeral workspace on its own network tier.
  • Gated deployment. A push triggers pull, rebuild, health check, and a build-provenance check that the running build matches what was deployed , so a deploy that reported success without swapping the image is caught. A drift job reconciles configuration.
  • Maintenance. A watchdog flags dependency and security updates. Low-risk ones merge through CI; the rest escalate.
  • Observability. Spend attributed per request and per model. Every call and every model swap traced.

Limits.

  • Single operator, by design. The reliability and privilege patterns are real mechanisms operating at personal scale. No claim is made about enterprise load, multi-tenancy, or an adversarial threat model.
  • One router in the path. Hosted models are reached through a single gateway, so a router outage is a shared dependency. Pinning to the local model mitigates it; it does not remove it.
  • The demo is not connected. Real interface, sample data, no link to the running system.
  • Semantic memory is a hosted service. The one cloud dependency in an otherwise local stack. Swappable, and not the system of record.
  • Retrieval tuning is ongoing. Hybrid retrieval covers more than keyword alone, but ranking is not a solved problem here.
  • Not a high-security design. Serious about data control, not a threat model against a determined adversary.

Rebuilding it. Point a coding agent at the architecture and rebuild the parts you want on what you already have. A working floor:

Item Cost
Coding assistant ~$20/mo
Small server ~$20/mo
Metered model calls, budget roster, personal load $10-40/mo

So roughly $50-80/mo, scaling up only where depth is wanted. My own build runs heavier because I develop it daily.

Artifacts.