面向开放世界的递归式 Agent Harness 优化系统（Recursive Agent Harnessing for an Open World）

所谓 Harness，是指围绕模型构建、使其能够作为 Agent 采取行动的整套系统，包括 Prompt、工具、记忆、技能以及编排等。而 Harnessing 则是持续优化该系统的全过程。

Harness² 是一套即插即用的框架，专为开放世界中的**递归式 Agent Harness 优化（recursive agent harnessing）**而设计。在开放世界中，Agent 往往会遭遇在 Harness 设计阶段从未见过的全新任务。该框架在保持模型权重固定的前提下，联结了两个层级的递归优化机制：

• 任务级递归（Task-level recursion）：通过对比式的**提议–探测–组合（Propose–Probe–Compose）**步骤，针对当前任务微调执行 Harness：提议修改方案、在同一任务上探测其效果，并在无需真实标注反馈（ground-truth feedback）的情况下，根据执行过程中的行为对比组合出精炼的 Harness。
• Harness 级递归（Harnessing-level recursion）：利用采纳的 Harness 以及从每个任务中获得的循证经验，更新一个持久化的改进 Harness（improvement harness），以指导后续的持续调优。

在涵盖 3 个专业基准评测和 4 种模型–Harness 配置的测试中，仅需单步 Harness² 处理，GLM-5 和 Gemini 3.5 Flash 相比各自的基础 Harness 均有明显提升：在 WorkBuddy-Bench 上最高提升 11.5 分，在 JobBench 上最高提升 4.5 分，在 LAB 的部分通过率上最高提升 11.8 个百分点。增加任务级递归的预算（budget）还能带来进一步的性能提升。

当前版本支持并行递归与序列递归、OpenCode 与 Codex 执行器，并适配 JobBench、LAB 以及 WorkBuddy-Bench。

视频/嵌入内容：演示视频 harness2-video.mp4

Harness² 暴露了围绕模型构建的八大可编辑核心组件：系统 Prompt（system prompt）、护栏（guardrails）、记忆（memory）、工具引导（tool guidance）、子 Agent（subagents）、技能（skills）、脚本（scripts）以及插件（plugins）。它们共同掌管上下文构建、工具接口定义、任务委托机制以及工具循环内部的行为控制。

推荐使用 Linux、Python 3.12 以及 uv。在代码仓库根目录下执行：

bash uv venv --python 3.12 .venv source .venv/bin/activate uv pip install -e '.[render]' harness2-demo --output runs/demo --k 2

本地 Demo 将使用确定性编辑器和真实的 Python 执行环境运行两种递归模式，无需配置任何模型凭证。

如需运行接入模型的基准测试，请安装 Node.js 20+、Google Cloud CLI 以及基准测试依赖项。LAB 需要 Pandoc 和 Poppler；WorkBuddy-Bench 则需要 Docker。随后配置您的模型信息：

bash cp .env.example .env

在 .env 文件中填入您的 Vertex AI 项目和模型 ID

gcloud auth application-default login

有关模型配置、替代身份验证方式以及 Codex 执行器的详细说明，请参阅安装配置指南。

以下示例展示了如何安装 LAB、运行单项任务，并完成评估与结果报告。调用模型会产生 API 费用，建议先从少量任务开始体验：

bash harness2-setup lab harness2-check --bench lab harness2 --bench lab --k 1 --mode parallel
--only-domains antitrust-competition --max-tasks 1 harness2-judge --bench lab --k 1 --mode parallel harness2-report --bench lab --k 1 --mode parallel

若要使用序列递归模式，只需在相同命令中添加 --mode sequential 即可。--k 代表任务级递归预算，详见实验部分说明。执行评判（Judging）和报告（Reporting）时，必须与微调运行使用相同的基准测试、执行器、子集、--k、--mode 以及随机种子（seed）。在 harness2 命令中添加 --dry-run 可以在不发起模型调用的情况下预览选取的任务。

每次命令请指定一个基准测试。对于 WorkBuddy，安装完成后在后续每个命令中还需附加 --subset office、--subset web 或 --subset code。基准测试指南详细介绍了各基准的数据构成、涉及领域和评分规则。

任务级递归可以通过两种方式进行扩展：并行递归会探索多个候选 Harness，并从它们的执行佐证中组合出优化方案；序列递归则会在连续的“提议–探测–组合”步骤中持续沿用优化后的 Harness 和累积的佐证。

并行递归探索更广的解空间；序列递归带来更深度的精细优化。

在每个任务完成后，Harness 级递归都会保留最终采纳的 Harness 及循证经验教训，用于指导未来的优化。关于任务选择、并发设置、可复现性以及基线对比等内容，请参阅实验指南。

执行器的网络访问权限默认不受限，且 Codex 在运行时处于沙箱禁用状态。公开的基准评分细则可能被在线检索到。在正式评估前，请务必配置好网络限制策略，以避免系统接触到基准答案与评分规则。

运行结果保存在 runs/ 目录下，并根据基准评测和子集（如适用）进行分类归档。内容包含任务摘要、生成产物、采纳的 Harness、反思笔记以及评估结论。

重复执行相同命令可从断点恢复运行。如果更换了模型或调整了任务选择，请设置新的 HARNESS2_RUNS_DIR 路径。详情请参阅结果查看与断点恢复指南；各命令均支持通过 --help 查看完整参数用法。

bibtex @article{xu2026harness2, title = {Harness$^2$: Recursive Agent Harnessing for an Open World}, author = {Xu, Ruiyao and Chen, Yanfei and CuiZhu, Zhongying and Dalvi Mishra, Bhavana and Ming, Yifei and Yu, Han and Han, Rujun and Lee, Chen-Yu and Pfister, Tomas}, journal = {arXiv preprint arXiv:2610.14224}, year = {2026}, url = {https://arxiv.org/abs/2610.14224} }

参与贡献请参阅 CONTRIBUTING.md。

Harness² 基于 Apache 2.0 协议开源。位于 third_party/ 目录下的第三方代码、基准测试与数据集遵循各自的上游开源协议，详见第三方许可声明。

本项目非 Google 官方正式支持产品，亦不适用于 Google 开源软件漏洞奖励计划（Google Open Source Software Vulnerability Rewards Program）。

Recursive Agent Harnessing for an Open World

A harness is the system around a model that enables it to act as an agent: prompts, tools, memory, skills, orchestration, and more. Harnessing is the process of optimizing that system.

Harness² is a plug-and-play framework for recursive agent harnessing in an open world, where agents encounter tasks unseen during harness design. It connects two levels of recursion while keeping model weights fixed:

• Task-level recursion refines the execution harness for the current task through a contrastive Propose–Probe–Compose step: propose edits, probe their effects on the same task, and compose a refined harness from behavioral contrasts between executions, without ground-truth feedback.
• Harnessing-level recursion updates a persistent improvement harness with adopted harnesses and evidence-supported lessons from each task to guide future refinement.

Across three professional benchmarks and four model–harness configurations, a single Harness² step improves GLM-5 and Gemini 3.5 Flash over their base harnesses, with gains of up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB's partial pass rate. Increasing the task-level recursion budget brings further gains.

The release supports parallel and sequential recursion , OpenCode and Codex executors , and JobBench, LAB, and WorkBuddy-Bench.

harness2-video.mp4

Harness² exposes eight editable components around the model: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and plugins. They control context construction, the tool interface, delegation, and behavior inside the tool loop.

Use Linux, Python 3.12, anduv. From the repository root:

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

The local demo runs both recursion modes with deterministic editors and real Python executions. It requires no model credentials.

For model-backed benchmarks, install Node.js 20+, the Google Cloud CLI , and the benchmark dependencies. LAB needs Pandoc and Poppler; WorkBuddy-Bench needs Docker. Then configure your models:

cp .env.example .env
# Edit .env with your Vertex AI project and model IDs.
gcloud auth application-default login

See the setup guide for model settings, authentication alternatives, and the Codex executor.

The following example installs LAB and runs one task, then evaluates and reports its result. Model-backed runs incur API costs; start with a small task selection.

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
  --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

Sequential recursion runs the same commands with --mode sequential. --k is the task-level recursion budget; see Experiments. Judging and reporting must use the same benchmark, executor, subset, --k, --mode, and seed as the refinement run. Add --dry-run to harness2 to preview task selection without model calls.

Choose one benchmark per command. WorkBuddy also takes --subset office, --subset web, or --subset code on every command after setup. The benchmark guides cover each one's data, domains, and grading.

Scale task-level recursion in two ways: parallel recursion explores multiple candidate harnesses and composes refinements from their execution evidence; sequential recursion carries the refined harness and accumulated evidence through successive Propose–Probe–Compose steps.

Parallel recursion explores wider; sequential recursion refines deeper.

After each task, harnessing-level recursion retains the adopted harness and evidence-supported lessons to guide future refinement. See the experiment guide for task selection, concurrency, reproducibility, and baseline comparisons.

Executor network access is unrestricted, and Codex runs with its sandbox disabled. Public benchmark rubrics may be accessible online. Before evaluation, configure network restrictions to prevent access to benchmark answers and grading criteria.

Results are saved under runs/, organized by benchmark and subset where applicable. They include task summaries, deliverables, adopted harnesses, reflection notes, and evaluation results.

Repeat the same command to resume. Use a new HARNESS2_RUNS_DIR when changing models or task selection. See results and resume for details; each command also supports --help.

@article{xu2026harness2,
  title   = {Harness$^2$: Recursive Agent Harnessing for an Open World},
  author  = {Xu, Ruiyao and Chen, Yanfei and CuiZhu, Zhongying and Dalvi Mishra, Bhavana and Ming, Yifei and Yu, Han and Han, Rujun and Lee, Chen-Yu and Pfister, Tomas},
  journal = {arXiv preprint arXiv:2610.14224},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.14224}
}

See CONTRIBUTING.md.

Harness² is licensed under Apache 2.0. Third-party code under third_party/, benchmarks, and datasets retain their upstream licenses; see third-party notices.

This is not an officially supported Google product. This project is not eligible for the Google Open Source Software Vulnerability Rewards Program.