商业产业 Hacker News AI 2026-08-28T00:06 正文 ok 2063 words

🔶 Terminal-Bench-Science:评估 AI Agent 科学研究工作流

英文标题:🔶 Terminal-Bench-Science: Evaluating AI agents on scientific research workflows

Terminal-Bench-Science 是由斯坦福大学研究人员牵头、Terminal-Bench 团队与全球多学科科学家共同构建的持续性基准,旨在评估 AI Agent 执行真实科学研究工作流的能力。其 0.1 版本覆盖生命、物理、地球、数学和工程科学的 70 项专家策划任务,最强系统 Claude Opus 5 配合 Claude Code 的任务解决率为 30.0%。该基准强调由一线科研人员定义任务、通过可复现的任务专属测试验证成果,并随前沿模型发展持续更新。对开发者而言,结果显示当前 Agent 在真实科研流程中仍有显著提升空间,且成本、Token 消耗和不同学科领域的表现存在明显权衡。
打开原文相关 5 · 机会 2
media17.6 KB media17.5 KB media82.3 KB media89.1 KB media49.9 KB media49.9 KB

中文

Terminal-Bench-Science 根据研究人员自身的工作流程评估 AI Agent。科学家而非模型开发者或数据供应商,为 AI 的科学能力设定标准。

Terminal-Bench-Science 是一项由斯坦福大学研究人员牵头的基准测试,由Terminal-Bench背后的团队与来自全球不同科学学科及研究机构的领域专家合作构建。它通过一组多样化、具有挑战性、由专家策划并源自科学研究的工作流,衡量 AI Agent 的能力。

Terminal-Bench-Science 是一项与前沿 AI 同步演进的持续性基准,在科学需求与 AI 开发之间建立反馈循环。首个版本包括来自生命科学、物理科学、地球科学、数学科学和工程科学的70 项任务。在已评估模型中,最强的 Claude Opus 5 在 Terminal-Bench-Science 0.1 上的解决率达到 30%。

Terminal-Bench-Science 0.1 排行榜

图表:Terminal-Bench-Science 0.1 上 70 项科学工作流任务的解决率。Claude Opus 5(Claude Code)30.0%;GPT-5.6 Sol(Codex)22.4%;Claude Fable 5(Claude Code)21.4%;Claude Opus 4.8(Claude Code)10.5%;GPT-5.6 Terra(Codex)8.6%;GLM 5.3(Claude Code)8.1%;Kimi K3(Claude Code)7.1%;Grok 4.6(Grok Build)7.1%;GPT-5.6 Luna(Codex)3.3%。

尽管 Terminal-Bench 已推动 AI Agent 在软件工程领域取得进展,Terminal-Bench-Science 将同样的雄心带入科学领域。我们的目标是推动具备科学能力的 Agent 发展,使其成为有用的研究助手。这些 Agent 应能执行技术要求高、耗时长的工作流,让科学家将更多时间投入最需要人类判断的科学环节:定义研究问题、形成假设、解释和验证结果,以及传达研究发现。在这一角色中,AI Agent 可以扩展研究人员的产出,并帮助加速科学发现。

实现这一目标需要能够反映真实科学实践、提供可验证能力证据、并与 AI 前沿同步演进的基准。

我们需要源自真实科学工作流的基准。 科学能力应在真实研究实践中接受评估,而非通过教科书问题或标准化练习;任务应由一线科学家本人贡献。Terminal-Bench-Science 让跨领域科学家拥有直接发声渠道和共享平台,为 AI 在其关切问题上的进展设定标准。科学的利害关系过于重大,其基准必须反映科学共同体的优先事项,而非外部利益。

我们需要可验证的科学能力证据。 如果没有可靠评估,我们无法判断 Agent 能力是否正在提升,也无法了解其局限仍在哪里。Terminal-Bench-Science 在现实环境中评估 Agent,并以可复现、任务专属的测试,对分析、仿真、证明、代码和数据产品等具体产物进行评分。

我们需要能够跟上前沿步伐的基准。 科学基准常常被当作需要发表的论文,而不是推动进步的机制。它们发布一次后便被搁置,尽管模型持续进步,已知局限仍然存在。Terminal-Bench-Science 是一项与 AI 前沿共同演进的持续性基准。通过定期发布,科学家可贡献新的工作流、改进既有任务,并在科学需求与 AI 开发之间建立反馈循环。

Terminal-Bench-Science 反馈循环

嵌入内容:Terminal-Bench-Science 反馈循环——科学共同体贡献工作流,前沿 AI Agent 与模型接受评估和改进,共同加速科学发现。

Terminal-Bench-Science 0.1 包含横跨生命、物理、地球、数学及工程科学的 70 项任务。任务涵盖科学数据分析、统计推断、仿真、优化、定理证明、图像重建、信号处理、逆问题、传感器校准、模型拟合、分类以及科学机器学习。

Terminal-Bench-Science 0.1 任务覆盖范围

图表:共 70 项任务,覆盖五个科学领域。生命科学 19 项:生物学 8 项、医学与健康 6 项、神经科学 4 项、生态与进化 1 项;物理科学 17 项:天文学与宇宙学 6 项、物理学 5 项、材料科学 4 项、化学 2 项;地球科学 8 项:地球科学 5 项、大气与气候 1 项、海洋与海事 1 项、环境与可持续性 1 项;数学科学 17 项:应用数学与科学计算 6 项、运筹学与优化 5 项、形式数学与定理证明 3 项、统计学 3 项;工程科学 9 项:机械与航空航天工程 5 项、电气与计算机工程 2 项、化学与过程工程 1 项、土木与结构工程 1 项。

研究人员通过GitHub上的开放流程贡献任务,并在Discord的 #tb-science 频道中讨论和反馈。贡献从提案开始,评审者会讨论每个想法、留下反馈,并批准那些看起来高度契合的提案:即值得在基准中衡量、具有科学依据的工作流。获批提案会以 pull request 形式实现,评审者将确认每项任务都可客观验证、确实对 AI Agent 构成挑战,并非当今前沿系统已能轻易解决的问题。要完成合并,领域评审者须评估科学有效性和真实性,技术评审者须检查任务构建和验证方式,而质量把关人(bar raiser)则进行最终质量检查。在 920 项提案中,464 项获准进入实现阶段,386 个 pull request 被创建,但仅有 70 项任务进入 Terminal-Bench-Science 0.1。这种严格筛选反映出,创建既具有科学趣味性、能挑战前沿 Agent,又有足够明确规范以便严格评估的任务,是多么困难。提案、pull request 和评审的进展可在公开任务仪表板上追踪。

Terminal-Bench-Science 0.1 为面向科学研究的 AI Agent 留下了巨大的进步空间。每个受评估模型均在全部 70 项任务中、每项任务进行 3 次独立试验。使用 Claude Code 的 Claude Opus 5 以 30.0% 的解决率位居首位,其后是使用 Codex 的 GPT-5.6 Sol(22.4%)和使用 Claude Code 的 Claude Fable 5(21.4%)。Claude Opus 4.8 位于中间水平,为 10.5%。GPT-5.6 Terra、Kimi K3 和 Grok 4.6 均解决不到 10% 的任务。GLM 5.3 是最强的开放模型,解决率为 8.1%;GPT-5.6 Luna 则以 3.3% 位列最后。

Terminal-Bench-Science 对系统的区分能力与 Terminal-Bench 3.0 大致相当,但对于每一个同时参与两项评估的模型,其解决率均降低超过 10 个百分点。这一差距是刻意设计的:在评审过程中,任务被校准为能够挑战最新前沿模型。

Terminal-Bench-Science 与 Terminal-Bench 对比

图表:Terminal-Bench 2.1、Terminal-Bench 3.0 与 Terminal-Bench-Science 0.1 的解决率比较。Terminal-Bench 3.0:Opus 5 为 42.7%,GPT-5.6 Sol 为 34.4%,Claude Fable 5 为 33.8%,Claude Opus 4.8 为 21.1%,GPT-5.6 Terra 为 20.8%,GPT-5.6 Luna 为 14.3%。Terminal-Bench 2.1:Claude Fable 5 为 83.8%,Claude Opus 4.8 为 78.9%,GPT-5.6 Terra 为 78.4%,GPT-5.6 Luna 为 75.7%。Terminal-Bench-Science 0.1:Opus 5 为 30.0%,GPT-5.6 Sol 为 22.4%,Claude Fable 5 为 21.4%,Claude Opus 4.8 为 10.5%,GPT-5.6 Terra 为 8.6%,GPT-5.6 Luna 为 3.3%。

性能只是进步的一个维度。成本—解决率图展示了完成全部 70 项任务的总评估成本与解决率之间的关系。GPT-5.6 Luna、Kimi K3 和 GPT-5.6 Terra 位于前沿模型的低成本端。GPT-5.6 Sol 和 Claude Opus 5 以更高成本达到最高解决率,其中 Opus 5 的成本为 7000 美元。GPT-5.6 Sol 的性能与 Claude Fable 5 相当,但成本不到后者三分之一(4200 美元,对 14200 美元)。Token 使用量则呈现出另一条前沿:Claude Fable 5 的性能与 GPT-5.6 Sol 相当,但 Token 用量约少四分之一(64 亿,对 84 亿)。Kimi K3 处于低 Token 用量端,Claude Opus 5 则处于高解决率端。只有 Kimi K3 和 Claude Opus 5 同时位于两条帕累托前沿上。

Terminal-Bench-Science 0.1 成本与解决率

图表:各评估系统的成本与解决率帕累托前沿。横轴为成本(0、5000、10000、15000、20000 美元),纵轴为解决率(0% 至 40%)。包括 GPT-5.6 Luna(Codex)、Kimi K3(Claude Code)、GPT-5.6 Terra(Codex)、GPT-5.6 Sol(Codex)、Opus 4.8(Claude Code)、Opus 5(Claude Code)和 Fable 5(Claude Code)。

Terminal-Bench-Science 0.1 Token 与解决率

图表:各评估系统的 Token 用量与解决率帕累托前沿。横轴为 Token 用量(20 亿至 160 亿),纵轴为解决率(0% 至 40%)。包括 Kimi K3(Claude Code)、Grok 4.6(Grok Build)、Fable 5(Claude Code)、Opus 4.8(Claude Code)、Opus 5(Claude Code)、GPT-5.6 Terra(Codex)、GPT-5.6 Sol(Codex)、GLM 5.3(Claude Code)和 GPT-5.6 Luna(Codex)。

解决率也因科学领域而异。Anthropic 和 OpenAI 模型在除工程科学外的每一个领域中包揽前两位;在工程科学中,Grok 4.6 以更低成本和 Token 用量,与 GPT-5.6 Sol 并列第二(14.8%)。Claude Opus 5 在除数学科学外的每个领域都领先 GPT-5.6 Sol 和 Claude Fable 5;在数学科学中,Claude Fable 5(33.3%)和 GPT-5.6 Sol(31.4%)占据前两位。按领域划分的完整明细可在排行榜查看。

Terminal-Bench-Science 0.1 按领域划分的解决率

图表:Opus 5、GPT-5.6 Sol 和 Grok 4.6 的领域解决率。生命科学:29.8%、25.5%、7.0%;地球科学:45.8%、27.5%、4.2%;工程科学:29.6%、14.8%、14.8%;数学科学:25.5%、31.4%、5.9%;物理科学:27.5%、23.5%、5.9%。

  • Opus 5
  • GPT-5.6 Sol
  • Grok 4.6

领域解决率:Opus 5、GPT-5.6 Sol 和 Grok 4.6

Terminal-Bench-Science 0.1 是一项由生命、物理、地球、数学和工程科学领域研究人员,以及Terminal-Bench和Harbor团队共同推动的社区项目。这是我们能够以开放方式构建的、最严格的科学 Agent 能力基准,而这仅仅是开始:Terminal-Bench-Science 0.1 是这一持续性基准的首次发布。

定期发布将增加任务,拓宽五个科学领域的覆盖范围,淘汰 Agent 已经达到饱和表现或评审发现规范不足的任务,并随着新的前沿模型发布,持续更新排行榜。每一次发布都将依据当时的前沿进行校准。对于 Terminal-Bench-Science 0.1,校准对象是 Claude Opus 5 和 GPT-5.6 Sol;随着更强模型出现,我们将使用它们评估和校准新的及改进后的任务。任务均已版本化,因此试验可以通过一条 Harbor 命令复用、重新评分或重新运行,从而保持更新结果的低成本。进展将在任务仪表板上公开追踪,每次发布均会在GitHub和Harbor Hub上打上标签。

Terminal-Bench-Science 0.2 的工作已在进行中,pull request 截止日期为 2026 年 10 月 5 日。如果你是一名研究人员,拥有前沿 Agent 应当能够完成、但目前仍无法完成的工作流,我们希望将其纳入该基准。贡献流程为 提案 → 构建 → 评审:通过任务提案表单提出任务,遵循贡献指南构建任务;随后,它将在合并前经历自动检查、并行的领域与技术评审,以及最终质量把关人审批。

欢迎通过Discord的 #tb-science 频道和GitHub加入这项工作,并通过项目日历参加我们的每周会议和答疑时间。让科学家来定义 AI 的科学能力应当是什么样子,并共同对其进行严格衡量。

如果你认为这项工作有用,请引用它。你可以使用 GitHub 上的“引用此仓库”按钮(由CITATION.cff生成),也可以使用以下信息手动引用。

@software{Terminal-Bench-Science_Team_Terminal-Bench-Science_Evaluating_AI_2026,
  author = {{Terminal-Bench-Science Team}},
  doi = {10.5281/zenodo.22110254},
  license = {Apache-2.0},
  month = aug,
  title = {{Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains}},
  url = {https://github.com/harbor-framework/terminal-bench-science},
  version = {v0.1.0},
  year = {2026}
}

感谢 Terminal-Bench-Science 背后的所有任务贡献者、评审者和顾问。

特别感谢我们的项目首席顾问 Ludwig Schmidt 和 Sanmi Koyejo;高级评审者 Allen Hart、Ivan Bercovich、Joseph Janssen、Jiaming Hu、Steffen Bollmann 和 Sergey Aganezov;AI 研究顾问 Ryan Marten、Alex Shaw、Lin Shi、Benjamin Feuer、Mike A. Merrill、Alex Dimakis、Jenia Jitsev、Bodhisattwa Majumder、Peter Clark、Thomas Wolf、Braden Hancock 和 Andy Konwinski;以及科学顾问 Sara Beery、Jo D

English / 原文

Terminal-Bench-Science evaluates AI agents on workflows from researchers' own work. Scientists, not model developers or data vendors, set the bar for scientific capability in AI.

Terminal-Bench-Science is a benchmark led by researchers at Stanford University and built by the team behind Terminal-Bench in collaboration with domain experts from a range of scientific disciplines and research institutions around the world. It measures the AI agent capabilities through a diverse set of challenging, expert-curated workflows drawn from scientific research.

Terminal-Bench-Science is a continuous benchmark that evolves alongside frontier AI, creating a feedback loop between scientific needs and AI development. Our first release includes 70 tasks from the life, physical, Earth, mathematical, and engineering sciences. The strongest model evaluated, Claude Opus 5, achieves a 30% resolution rate on Terminal-Bench-Science 0.1.

Terminal-Bench-Science 0.1 Leaderboard

0%25%50%75%100%Claude Opus 5Claude Code30.0%GPT-5.6 SolCodex22.4%Claude Fable 5Claude Code21.4%Claude Opus 4.8Claude Code10.5%GPT-5.6 TerraCodex8.6%GLM 5.3Claude Code8.1%Kimi K3Claude Code7.1%Grok 4.6Grok Build7.1%GPT-5.6 LunaCodex3.3%

Resolution rates across 70 scientific workflow tasks on Terminal-Bench-Science 0.1

While Terminal-Bench has driven progress in AI agents for software engineering, Terminal-Bench-Science brings the same ambition to science. Our goal is to drive the development of agents with scientific capabilities that make them useful research assistants. These agents should execute technically demanding and time-consuming workflows, freeing scientists to focus more of their time on the parts of science where human judgment matters most: defining research questions, forming hypotheses, interpreting and validating results, and communicating findings. In this role, AI agents can extend what researchers accomplish and help accelerate scientific discovery.

Achieving this requires benchmarks that reflect real scientific practice, provide verifiable evidence of capability, and evolve alongside the AI frontier.

We need benchmarks drawn from real scientific workflows. Scientific capability should be evaluated on real research practice, not textbook questions or standardized exercises, contributed by practicing scientists themselves. Terminal-Bench-Science gives scientists across domains a direct voice and a shared platform to set the bar for AI progress on the problems they care about. The stakes in science are too high, and its benchmarks must reflect the scientific community's priorities rather than outside interests.

We need verifiable evidence of scientific capability. Without reliable evaluation, we cannot tell whether agent capabilities are improving or where their limitations remain. Terminal-Bench-Science evaluates agents in realistic environments and grades concrete artifacts such as analyses, simulations, proofs, code, and data products with reproducible, task-specific tests.

We need a benchmark that keeps pace with the frontier. Too often, scientific benchmarks are treated as papers to publish rather than mechanisms for driving progress. They are released once and then abandoned as models advance and known limitations persist. Terminal-Bench-Science is a continuous benchmark that evolves alongside the AI frontier. Through regular releases, scientists can contribute new workflows, improve existing tasks, and create a feedback loop between scientific needs and AI development.

Terminal-Bench-Science Feedback Loop

SCIENTIFICCOMMUNITYFRONTIER AIAGENTS & MODELSContributeworkflowsEvaluate &improveAccelerate discoveryTerminal-Bench-Science Feedback Loop

Terminal-Bench-Science 0.1 includes 70 tasks across the life, physical, Earth, mathematical, and engineering sciences. Tasks span scientific data analysis, statistical inference, simulation, optimization, theorem proving, image reconstruction, signal processing, inverse problems, sensor calibration, model fitting, classification, and scientific machine learning.

Terminal-Bench-Science 0.1 Task Coverage

LIFE SCIENCES19Biology8Medicine & Health6Neuroscience4Ecology & Evolution1PHYSICAL SCIENCES17Astronomy & Cosmology6Physics5Materials Science4Chemistry2EARTH SCIENCES8Geoscience5Atmosphere & Climate1Ocean & Marine1Environment & Sustainability1MATHEMATICAL SCIENCES17Applied Mathematics & Scientific Computing6Operations Research & Optimization5Formal Mathematics & Theorem Proving3Statistics3ENGINEERING SCIENCES9Mechanical & Aerospace Engineering5Electrical & Computer Engineering2Chemical & Process Engineering1Civil & Structural Engineering170 TASKS IN TOTAL0246810

70 expert-curated tasks across five scientific domains

Tasks are contributed by researchers through an open process on GitHub, with discussion and feedback in the #tb-science channel on Discord. Contributions begin as proposals, where reviewers discuss each idea, leave feedback, and approve those that look like a strong fit: scientifically grounded workflows worth measuring in the benchmark. Approved proposals are implemented as pull requests, where reviewers confirm that each task is objectively verifiable, genuinely challenging for AI agents, and not something today's frontier systems already solve easily. To merge, domain reviewers assess scientific validity and realism, technical reviewers inspect task construction and verification, and a bar raiser performs a final quality check. Of 920 proposals, 464 were approved for implementation and 386 pull requests were opened, but only 70 tasks made it into Terminal-Bench-Science 0.1. That selectivity reflects how difficult it is to create tasks that are scientifically interesting, challenging for frontier agents, and sufficiently well specified for rigorous evaluation. Progress across proposals, pull requests, and reviews is tracked on the public task dashboard.

Terminal-Bench-Science 0.1 leaves substantial room for progress on AI agents for scientific research. Each evaluated model ran three independent trials per task across all 70 tasks. Claude Opus 5 with Claude Code achieves the highest resolution rate at 30.0%, followed by GPT-5.6 Sol with Codex at 22.4% and Claude Fable 5 with Claude Code at 21.4%. Claude Opus 4.8 sits in the middle at 10.5%. GPT-5.6 Terra, Kimi K3, and Grok 4.6 all resolve less than 10% of tasks. GLM 5.3 is the strongest open model at 8.1%, and GPT-5.6 Luna is last at 3.3%.

Terminal-Bench-Science distinguishes between systems about as well as Terminal-Bench 3.0 while pushing resolution rates down by more than 10 percentage points for every model evaluated on both. That gap is deliberate: during review, tasks were calibrated to challenge the newest frontier models.

Terminal-Bench-Science vs. Terminal-Bench

Terminal-Bench 2.1Terminal-Bench 3.0Terminal-Bench-Science 0.142.7%Opus 5 — 30.0%34.4%GPT-5.6 Sol — 22.4%Fable 5 — 83.8%33.8%Fable 5 — 21.4%Opus 4.8 — 78.9%21.1%Opus 4.8 — 10.5%GPT-5.6 Terra — 78.4%20.8%GPT-5.6 Terra — 8.6%GPT-5.6 Luna — 75.7%14.3%GPT-5.6 Luna — 3.3%Resolution Rates on Terminal-Bench 2.1, Terminal-Bench 3.0, and Terminal-Bench-Science 0.1

Performance is only one dimension of progress. The cost-resolution plot shows total evaluation cost across all 70 tasks against resolution rate. GPT-5.6 Luna, Kimi K3, and GPT-5.6 Terra occupy the low-cost end of the frontier. GPT-5.6 Sol and Claude Opus 5 reach the highest resolution rates at greater cost, with Opus 5 at $7.0k. GPT-5.6 Sol matches Claude Fable 5's performance at less than a third of the cost ($4.2k vs $14.2k). Token usage shows a different frontier. Claude Fable 5 matches GPT-5.6 Sol's performance while using about a quarter fewer tokens (6.4B vs 8.4B). Kimi K3 anchors the low-token end and Claude Opus 5 the high-resolution end. Only Kimi K3 and Claude Opus 5 appear on both Pareto frontiers.

Terminal-Bench-Science 0.1 Cost vs. Resolution Rate

$0$5.0k$10.0k$15.0k$20.0k0%10%20%30%40%CostResolution RateGPT-5.6 LunaCodexKimi K3Claude CodeGPT-5.6 TerraCodexGPT-5.6 SolCodexOpus 4.8Claude CodeOpus 5Claude CodeFable 5Claude Code

Pareto frontier of cost and resolution rate across evaluated systems

Terminal-Bench-Science 0.1 Tokens vs. Resolution Rate

2.0B4.0B6.0B8.0B10.0B12.0B14.0B16.0B0%10%20%30%40%TokensResolution RateKimi K3Claude CodeGrok 4.6Grok BuildFable 5Claude CodeOpus 4.8Claude CodeOpus 5Claude CodeGPT-5.6 TerraCodexGPT-5.6 SolCodexGLM 5.3Claude CodeGPT-5.6 LunaCodex

Pareto frontier of token usage and resolution rate across evaluated systems

Resolution rates also vary by scientific domain. Anthropic and OpenAI models take the top two spots in every domain except the engineering sciences, where Grok 4.6 ties GPT-5.6 Sol for second place (14.8%) at lower cost and token usage. Claude Opus 5 leads both GPT-5.6 Sol and Claude Fable 5 in every domain except the mathematical sciences, where Claude Fable 5 (33.3%) and GPT-5.6 Sol (31.4%) take the top two spots. The full breakdown by domain is available on the leaderboard.

Terminal-Bench-Science 0.1 Resolution Rates by Domain

Domain resolution rates: Opus 5, GPT-5.6 Sol, and Grok 4.6LIFESCIENCESEARTHSCIENCESENGINEERINGSCIENCESMATHEMATICALSCIENCESPHYSICALSCIENCES102030405029.8%45.8%29.6%25.5%27.5%17.5%20.8%14.8%31.4%23.5%7.0%4.2%14.8%5.9%5.9%

  • Opus 5
  • GPT-5.6 Sol
  • Grok 4.6

Domain resolution rates: Opus 5, GPT-5.6 Sol, and Grok 4.6

Terminal-Bench-Science 0.1 is a community effort by researchers across the life, physical, Earth, mathematical, and engineering sciences, together with the Terminal-Bench and Harbor team. It is the most rigorous benchmark of scientific agent capabilities we could build in the open, and we are only getting started: Terminal-Bench-Science 0.1 is the first release of a continuous benchmark.

Regular releases will add tasks, broaden coverage across the five scientific domains, retire tasks that agents saturate or that review reveals to be underspecified, and keep the leaderboard current as new frontier models are released. Each release is calibrated against the frontier at the time. For Terminal-Bench-Science 0.1, this was Claude Opus 5 and GPT-5.6 Sol, and as stronger models emerge we will use them to evaluate and calibrate new and improved tasks. Tasks are versioned so that trials can be re-used, re-graded, or re-run with a single Harbor command, which keeps the cost of updating results low. Progress is tracked in the open on the task dashboard, and every release is tagged on GitHub and Harbor Hub.

Work on Terminal-Bench-Science 0.2 is already underway, with a pull request deadline of October 5, 2026. If you are a researcher with a workflow that frontier agents should be able to do but cannot yet, we want it in the benchmark. The contribution flow is Propose → Build → Review : propose your task through the task proposal form, build it following the contributing guide, and it will go through automated checks, parallel domain and technical review, and final bar-raiser approval before merge.

Join the effort in #tb-science on Discord and on GitHub, and drop into our weekly meetings and office hours via the project calendar. Let's let scientists define what scientific capability in AI looks like, and measure it rigorously, together.

If you find this work useful, please cite it. You can use the "Cite this repository" button on GitHub (generated from CITATION.cff) or cite manually using the information below.

@software{Terminal-Bench-Science_Team_Terminal-Bench-Science_Evaluating_AI_2026,
  author = {{Terminal-Bench-Science Team}},
  doi = {10.5281/zenodo.22110254},
  license = {Apache-2.0},
  month = aug,
  title = {{Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains}},
  url = {https://github.com/harbor-framework/terminal-bench-science},
  version = {v0.1.0},
  year = {2026}
}

Thank you to all of the task contributors, reviewers, and advisors behind Terminal-Bench-Science.

Special thanks to our project lead advisors Ludwig Schmidt and Sanmi Koyejo; our senior reviewers Allen Hart, Ivan Bercovich, Joseph Janssen, Jiaming Hu, Steffen Bollmann, and Sergey Aganezov; our AI research advisors Ryan Marten, Alex Shaw, Lin Shi, Benjamin Feuer, Mike A. Merrill, Alex Dimakis, Jenia Jitsev, Bodhisattwa Majumder, Peter Clark, Thomas Wolf, Braden Hancock, and Andy Konwinski; and our scientific advisors Sara Beery, Jo Dunkley, J. Nathan Kutz, Ching-Yao Lai, Scott Linderman, Emma Lundberg, Russ Poldrack, Aviv Regev, and Risa Wechsler.

Terminal-Bench-Science is an open academic collaboration hosted by Stanford University and the Laude Institute, in partnership with the Stanford AI Lab (SAIL), the Stanford Institute for Human-Centered Artificial Intelligence (HAI), Stanford AI Measurement Science (AIMS), the NSF AI Institute for Foundations of Machine Learning (IFML), the Allen Institute, and the Allen Institute for AI (Ai2). As part of the Terminal-Bench franchise, it is built by the Terminal-Bench and Harbor team together with a community of scientific contributors.

We thank the Laude Institute for support through the Slingshots program, Snorkel AI for support through the Open Benchmarks Grants program, the 2077AI Open Source Foundation for PP API credits supporting task review and curation, and UniPat AI and Modal for their support of Terminal-Bench-Science. We thank Bespoke Labs, Anthropic, Google, Moonshot AI, SpaceXAI, and Z.ai for API credits supporting leaderboard evaluations.

Written by: Steven Dillmann