为 Agentic AI 构建企业级运行环境

英文标题：Building the enterprise environment for agentic AI

Agentic AI 的企业价值不止于更好的聊天机器人，而在于能够跨越人员、业务流程、数据和系统端到端执行任务的软件 Agent。企业部署 Agent 时，需要将其视为涵盖任务编排、数据访问、工具调用、延迟管理、治理和基础设施扩展能力的系统性问题，而非单纯的 LLM 推理问题。Intel 扩展了开源 Agent 基准测试工具 Terminal-Bench，通过对 LLM 响应进行确定性的录制与回放，剥离模型输出波动对测试结果的影响。该研究建议开发和平台团队以每 vCPU 的 Agent 密度、任务延迟和任务吞吐量等指标规划容量与扩展策略，而非仅关注 Agent 数量或平均 CPU 利用率。

打开原文相关 8 · 机会 3

中文

Agentic AI 是一个更广泛的系统性问题，而不只是推理问题。
现有的大多数 Agentic AI 测试框架能力有限，且无法衡量整体系统性能。
容量规划应采用每虚拟 CPU（vCPU）的 Agent 密度，而不是 Agent 数量。
应监控 Agent 任务延迟，而不只是平均 CPU 利用率。
对于托管 Agent 的系统，应默认采用横向扩展（scale-out）。仅当单个 Agent 的计算负载更重或受到架构约束时，才采用纵向扩展（scale-up）。

超越推理：将 Agent 用于工作流自动化

Agentic AI 不只是 LLM 推理。其企业价值取决于完整系统，包括任务编排、数据访问、工具执行、延迟管理、治理和可扩展基础设施。Agent 是一种由目标驱动的自动化企业工作流过程：它规划多步骤任务、调用工具、读取结果，并在出现故障时重试。因此，企业 Agent 不只是一个推理问题；它是一个系统问题。

定义何为良好表现

大多数 Agentic AI 指标聚焦于评估所使用的 LLM。平台团队还需要了解任务耗时、一个 Agent 集群能够支持多少 Agent、用户在执行流程结束时的体验，以及随着更多 Agent 同时运行，成本将如何变化。

更适用于企业的视角应关注六项指标：

任务成功率
单任务成本
单任务耗时
任务吞吐量
Agent 密度（每 vCPU 的 Agent 数）
延迟

这些指标共同回答了企业 AI 运营人员所关心的问题：系统是否按预期运行？系统能够持续支持多少个 Agent？应如何扩展以支持更多 Agent？

构建坚实基础

为了更深入地了解 Agentic AI 工作负载性能，Intel 扩展了 Terminal-Bench。这是一个用于评估 AI Agent 的开源基准测试框架，具备性能剖析、遥测和回放能力。这使得研究人员能够了解除 LLM 推理之外，Agent 将时间花在了哪些环节。

该基准测试扩展通过对 LLM 响应进行确定性的录制与回放，将 Agent 性能与 LLM 的输出波动分离开来。LLM 响应只记录一次，并在各次运行中以完全相同的方式回放，从而减少运行间的差异，并建立更可靠的比较基础。

所使用的 Terminal-Bench 任务组合有意覆盖了广泛场景，包括编译、测试、数据库操作、布尔逻辑、解释执行、光线追踪、压缩、线性代数、视频转码和机器学习训练。如此广泛的任务类型，使研究发现与真实企业环境更加相关。

English / 原文

Agentic AI is a larger systems problem, not just one of inference.
The majority of existing agentic AI harnesses are limited and do not measure overall system performance.
Plan capacity is done using agents per virtual CPU (vCPU) density, not agent count.
Monitor agent task latency, not just average CPU utilization.
Default to scale-out for systems hosting agents. Reserve scale-up for workloads with heavier per-agent compute or architectural constraints.

Beyond inference: Agents as workflow automation

Agentic AI is more than LLM inference. Its enterprise value depends on the full system, task orchestration, data access, tool execution, latency management, governance, and scalable infrastructure. An agent is a goal-driven automated enterprise workflow process: It plans a multi-step task, calls tools, reads results, and retries when something fails. Enterprise agents are therefore not just an inference problem; they are a systems problem.

Defining what good looks like

Most agentic AI metrics focus on evaluating the LLM used. Platform teams also need to know how long the tasks take, how many agents a fleet can support, what users experience at the end of the execution process, and how costs change as more agents work simultaneously.

A more useful enterprise view looks at six metrics:

Task success rate
Cost per task
Time per task
Task throughput
Agent density (agents per vCPU)
Latency

Together, these answer the questions enterprise AI operators care about: Is the system performing as expected? How many agents can the system sustain? How should it scale to support more agents?

Building on solid foundations

To gain a deeper insight into agentic AI workload performance, Intel extended Terminal-Bench, an open source benchmarking harness for evaluating AI agents with profiling, telemetry, and replay capabilities. This made it possible to understand where the agents spent time beyond LLM inference.

The benchmark extension used a deterministic record-replay of LLM responses to separate agent performance from LLM variability. LLM responses were recorded once and replayed identically across runs, reducing run-to-run variance and creating a more reliable basis for comparison.

The Terminal-Bench task mix used was intentionally broad. It included compilation, testing, database operations, Boolean logic, interpretation, ray tracing, compression, linear algebra, video transcoding, and machine learning training. That wide variety made the findings more relevant to real enterprise environments.