新 Agent 技能：为你的编程 Agent 提供 Amazon SageMaker 优化生成式 AI 推理能力

工程师正越来越多地使用编程辅助工具来加快开发工作流。今天，Amazon SageMaker AI 优化生成式 AI 推理推出 aws-ai-ml 技能，该技能可通过 Agent Toolkit for AWS 获取。该技能为 Kiro、Claude Code 和 Codex 等编程 Agent 提供深度的推理优化和基准测试专业能力。安装该技能后，你现有的 Agent 可代表你对端点进行基准测试、推荐部署配置、比较性能运行结果，并生成可执行的 SageMaker Python SDK v3 代码。aws-ai-ml 技能是一个可接入任何支持模型上下文协议（Model Context Protocol，MCP）的编程 Agent 的工具包，可将其转变为 SageMaker AI 推理优化专家。

本文将介绍该技能能够实现什么、如何设置，以及它如何帮助你更快地从模型走向生产。

挑战：弥合意图与基础设施之间的鸿沟

Amazon SageMaker AI 支持实时、批处理和异步模式下的有服务器托管。它提供按需和预留容量、异构实例、虚拟私有云（VPC）隔离、自动扩缩容，以及与所有 SageMaker AI 训练路径的集成。其能力覆盖面广且深入，但大多数工程师并不会一开始就知道哪种实例系列或服务容器最适合自己的需求。他们通常带着一个用例而来：希望实现某个性能目标、需要控制在某个成本范围内，或需要在投入生产前评估某个模型。

SageMaker AI 优化生成式 AI 推理的 Agent 化体验弥合了这一鸿沟。你只需告诉 Agent 想要完成什么，它就会生成可执行的 SageMaker Python SDK v3 代码，你可以在自己的环境中审查、修改和运行。Agent 会提出有针对性的澄清问题，基于真实基准测试和实测性能数据生成代码，并像解决方案架构师一样适应你的业务约束。

在整个过程中，你始终保持控制权。每一个步骤都实时可见，并以你可以阅读和质询的代码形式呈现。不会有任何操作隐藏在不透明的 UI 背后。

快速开始

你可以在本地通过 Agent Toolkit for AWS 安装 aws-ai-ml 技能，也可以在 Amazon SageMaker Studio 的 JupyterLab 空间中使用它。无论采用哪种方式，都可以在 10 分钟内从零开始进入可用的对话。

方案 A：与任何编程 Agent 一起使用（Kiro、Claude Code、Codex 或任何兼容 MCP 的 Agent）

第 1 步：安装 Agent Toolkit for AWS。 如果尚未完成设置，请先配置 Agent Toolkit。这要求已安装 AWS Command Line Interface（AWS CLI）2.35+ 以及 uv。

aws configure agent-toolkit

该命令会自动检测你的 Agent、安装技能并配置 AWS MCP Server。有关面向特定 Agent 的设置（插件安装命令、MCP 配置），请参阅 Agent Toolkit for AWS 快速入门指南。

第 2 步：安装 aws-ai-ml 技能。 将 SageMaker AI 优化生成式 AI 推理技能添加到你的 Agent：

npx skills add aws/agent-toolkit-for-aws/skills/aws-ai-ml

第 3 步：确认并开始。 打开编程 Agent 的聊天面板并提问：“有哪些可用技能？”你应该会看到列出了 aws-ai-ml。确认后，用自然语言描述你的意图。你的编程 Agent 现在已内置 SageMaker AI 推理优化专业能力。

前提条件： 你的 AWS 凭证必须具备调用 SageMaker AI API 的权限，包括创建端点、运行基准测试和推荐任务。该技能生成的代码将在你的凭证下运行。技能本身不需要额外的 AWS Identity and Access Management（IAM）配置。

注意： 对于 Kiro 和 Claude Code，Agent 可在运行时发现技能。它们无需进行本地安装，即可通过 AWS MCP Server 按需搜索和加载技能。可以向 Agent 提问：“搜索与数据库相关的 AWS 技能。”有关运行时发现技能的信息，请参阅 readme。

方案 B：在 Amazon SageMaker Studio 中使用

如果你更愿意在托管式 JupyterLab 环境中工作，可以使用带有预配置镜像的 Amazon SageMaker Studio 来运行该技能。

第 1 步：打开 Amazon SageMaker Studio。 在目标 AWS 账户和 AWS 区域中导航至 Amazon SageMaker Studio。选择你的 Studio 域，并从用户配置文件启动 Studio IDE。

第 2 步：创建 JupyterLab 空间。 在 Studio 落地页中，选择 JupyterLab，然后选择 创建 JupyterLab 空间。为该空间命名（例如 my-inference-opt），并将共享设置保持为 私有（技能仅会同步到私有空间）。在 镜像 菜单下，选择包含 SageMaker AI 优化生成式 AI 推理技能的镜像。该镜像预配置了 aws-ai-ml Agent 技能及所有必需依赖项。选择 运行空间，并等待其启动（首次启动需要 5–10 分钟）。

注意： 请使用一个新的空间。复用的空间如果包含本地修改过的技能版本，可能无法采用预配置镜像。

第 3 步：打开 JupyterLab 并启动终端。 空间启动后，打开 JupyterLab 并选择 终端。

第 4 步：授权你的编程 Agent。 在终端中，使用身份提供商验证你的编程 Agent。例如，对于 Kiro：

kiro-cli login --license pro --identity-provider <your-IdP-start-URL> --region us-east-1 --use-device-flow

第 5 步：确认并开始。 打开编程 Agent 的聊天面板并提问：“有哪些可用技能？”你应该会看到列出了 aws-ai-ml。确认后，用自然语言描述你的意图。

故障排除：

如果 Agent 报告没有可用技能，请确认你的空间被设置为 私有。你也可以在终端中检查：

如果该目录为空，但 /etc/sagemaker/skills/ 包含技能文件，请运行 restart-jupyter-server，刷新页面后重试。

你可以做什么

该 Agent 化体验覆盖推理优化生命周期中的以下能力。你无需知道应调用哪项能力；只需描述你想做什么，Agent 会自行判断下一步，或在信息不明确时提出澄清问题。

对现有端点进行基准测试

如果你已在 SageMaker AI 端点上部署模型，可以让 Agent 对其进行基准测试。告诉 Agent 想测试哪个端点，它会生成一个 Python Notebook，使用 SageMaker Python SDK 中的 Workload.synthetic() 和 start_benchmark() API 执行负载测试。

在运行任何基准测试之前，Agent 会确认你的端点是否适合进行负载测试，因为基准测试会向在线端点发送真实流量。

基准测试完成后，你将获得包含以下指标的量化性能报告：

• 吞吐量：每秒请求数、每秒输出 Token 数。
• 延迟：p50、p99、首 Token 时间、Token 间延迟。
• 并发度：支持的同时请求数量。

这些是在真实基础设施上通过真实负载测得的数值，而非估算值。Agent 还会推荐提升性能的机制（例如预填充解码）。

示例提示词： “Benchmark my Llama endpoint on SageMaker AI.”

为你的模型找到合适的实例类型

如果你拥有一个模型，并需要找到适合部署到 SageMaker AI 的实例类型，只需告诉 Agent。无论模型存放在哪里，或你是如何获得它的，都不影响：

• 位于 Amazon Simple Storage Service（Amazon S3）的微调或自定义模型：你训练或下载了模型，并将其存储在 S3 中。请提供 S3 URI 和优化目标。示例提示词：“I want to find the cheapest instance type to deploy my fine-tuned model on SageMaker.”
• 来自 Amazon SageMaker JumpStart 的公开基础模型：你希望从 JumpStart 目录部署基础模型（FM），随后提供模型 ID。示例提示词：“Find the best instance for model huggingface-reasoning-qwen3-8b on SageMaker AI.”
• 位于 Hugging Face Hub 的模型：你希望使用托管于 Hugging Face 的模型。请提供模型名称。对于受限模型（例如 Llama 变体），Agent 会展示许可条款，并要求你接受条款及提供 Hugging Face Token。示例提示词：“I want to deploy a Llama model from Hugging Face Hub. What’s the cheapest option?”

在所有情况下，Agent 都会生成代码，在候选实例和配置上评估你的模型，然后给出经过排序的部署选项及具体性能指标：吞吐量、延迟百分位数、首 Token 时间和并发度。你可根据成本和性能需求作出选择。

比较基准测试运行结果

如果你运行了多次基准测试（例如配置变更前后，或在两种实例类型上测试），可以让 Agent 对它们进行比较。提供两个基准测试任务名称后，Agent 会生成对比结果，计算关键指标的变化量，包括吞吐量、延迟百分位数和首 Token 时间。

结果会以百分比变化呈现，其中正值表示更优，让你能通过一份易于理解的摘要判断变更是否改善了性能、降低了性能，或没有产生有意义的影响。

如果某次基准测试运行不存在，Agent 会先询问是否需要运行该测试，然后再继续进行比较。

示例提示词： “I have two benchmark runs and I want to compare them. Which one is faster?”

基准测试结果

下表比较了在相同基准测试负载下（512/256 Token，并发度为 4）部署的两个模型。Δ% 列显示模型 B（Qwen3-8B）在每项指标上比模型 A（Qwen3-1.7B）快多少。正值表示模型 B 更优。

指标	Qwen3-1.7B（模型 A）	Qwen3-8B（模型 B）	Δ%
输出 Token 吞吐量	188.2 tokens/s	271.2 tokens/s	+44.1% ✅
单用户吞吐量	47.5 tokens/s	69.4 tokens/s	+45.9% ✅
请求吞吐量	0.736 req/s	1.08 req/s	+46.7% ✅
Token 间延迟	20.9 ms	14 ms	+33.0% ✅
请求延迟	5,382 ms	3,658 ms	+32.0% ✅
首 Token 时间	67.5 ms	166.3 ms	−146.5% ❌

这两个模型运行在不同的硬件上。Qwen3-8B 使用 4 GPU 的 ml.g5.12xlarge（4 个 A10G），而 Qwen3-1.7B 使用单个 L4 GPU 的 ml.g6.4xlarge。这些差异反映的是约 4 倍的计算能力，并非仅由模型本身造成。

结论是：Qwen3-8B 的吞吐量高出约 44–47%，端到端延迟也更低，这在很大程度上得益于额外的 GPU 算力。Qwen3-1.7B 仅在首 Token 时间上胜出，这是较小模型运行在单 GPU 上的预期优势。

整合使用

你不需要记住能力名称，也无需知道应该请求哪种工作流。下表展示了常见请求如何映射到相应结果。

你说	你将获得
“我已经部署了一个模型，想知道它有多快。”	来自真实负载的量化性能报告：吞吐量、延迟百分位数和并发指标。
“我在 S3 中有一个模型，但不知道应部署在哪种实例上。”	按排名提供的部署选项，包含每种候选配置的成本、吞吐量和延迟。
“我想部署一个 JumpStart 模型并找到最便宜的实例。我只有模型 ID。”	按排名提供的部署选项，无需暂存到 S3；如有需要，会展示受限模型的替代方案。
“我在一次变更前后运行了基准测试。哪个更快？”	所有指标的百分比变化，指出性能改善或回退。
“我想优化 Hugging Face Hub 上的 Llama 模型。”	展示许可证，将模型暂存到 S3，然后输出标准推荐结果。

如果你的请求跨越多项能力（例如，先暂存 Hugging Face 模型，再获取部署推荐），Agent 会在同一对话中自然地串联这些步骤。

对 Agent 的预期

配备 aws-ai-ml 技能的 Agent 会遵循以下行为，以使体验具备可预测性和安全性：

• 它会索取所需信息。 如果缺少信息（如端点名称或 S3 URI），Agent 会要求你提供，而不会自行猜测。
• 它会在执行有影响的操作前确认。 在运行会向在线端点发送真实流量的基准测试前，Agent 会警告你相关影响，并要求获得明确确认。
• 它会说明超出范围的事项。 如果你要求 Agent 执行它无法完成的事项（例如部署模型），它会说明自己可以提供什么替代帮助，例如生成你所需的部署配置。
• 它会生成 SageMaker Python SDK v3 代码。 每一项输出都是可执行代码，你可以在自己的环境中检查、修改和运行。

清理

为避免持续产生费用，请删除你创建的资源：

结论

面向 Amazon SageMaker AI 优化生成式 AI 推理的 aws-ai-ml 技能，可将你现有的编程 Agent 转变为 SageMaker AI 推理优化专家。无论你需要对在线端点进行基准测试、为模型寻找最便宜的实例、比较不同配置，还是暂存 Hugging Face 模型以供评估，你只需描述需求，Agent 就会交付可量化的结果。

要开始使用，请通过 Agent Toolkit for AWS 安装该技能并将其添加到你已使用的编程 Agent，或者启动一个预配置的 JupyterLab 空间。

Engineers increasingly use coding assistance tools to accelerate their development workflows. Today, Amazon SageMaker AI optimized generative AI inference introduces the aws-ai-ml skill, available through the Agent Toolkit for AWS. This skill gives coding agents like Kiro, Claude Code, and Codex deep expertise in inference optimization and benchmarking. Install the skill, and your existing agent can benchmark endpoints, recommend deployment configurations, compare performance runs, and generate executable SageMaker Python SDK v3 code on your behalf. The aws-ai-ml skill is a toolkit that plugs into any coding agent that supports the Model Context Protocol (MCP), turning it into a SageMaker AI inference optimization expert.

In this post, we walk through what the skill enables, how to set it up, and how it helps you move faster from model to production.

The challenge: Bridging intent and infrastructure

Amazon SageMaker AI supports serverful hosting across real-time, batch, and asynchronous modes. It offers on-demand and reserved capacity, heterogeneous instances, virtual private cloud (VPC) isolation, automatic scaling, and integration with every SageMaker AI training path. The surface area is broad and deep, but most engineers do not arrive knowing which instance family or serving container will best serve their needs. They arrive with a use case: a performance target they want to hit, a cost envelope they need to stay within, or a model they need to evaluate before committing to production.

The agentic experience for SageMaker AI optimized generative AI inference closes this gap. You tell the agent what you want to accomplish, and it produces executable SageMaker Python SDK v3 code that you can review, modify, and run in your own environment. The agent asks targeted clarifying questions, generates code grounded in real benchmarks and measured performance data, and adapts to your business constraints the way a solutions architect would.

Throughout, you stay in control. Every step is visible in real time and expressed as code you can read and question. Nothing happens behind an opaque UI.

Getting started

You can install the aws-ai-ml skill through the Agent Toolkit for AWS on your local machine, or use it within an Amazon SageMaker Studio JupyterLab space. Either way, you can go from zero to a working conversation in 10 minutes.

Option A: Use with any coding agent (Kiro, Claude Code, Codex, or any MCP-compatible agent)

Step 1: Install the Agent Toolkit for AWS. If you haven’t already, set up the Agent Toolkit. This requires AWS Command Line Interface (AWS CLI) 2.35+ and uv installed.

aws configure agent-toolkit

This auto-detects your agents, installs skills, and configures the AWS MCP Server. For agent-specific setup (plugin install commands, MCP config), see the Agent Toolkit for AWS getting started guide.

Step 2: Install theaws-ai-ml skill. Add the SageMaker AI optimized generative AI inference skill to your agent:

npx skills add aws/agent-toolkit-for-aws/skills/aws-ai-ml

Step 3: Confirm and begin. Open your coding agent’s chat panel and ask: “What skills are available?” You should see aws-ai-ml listed. After you confirm, describe your intent in natural language. Your coding agent now has SageMaker AI inference optimization expertise built in.

Prerequisites: Your AWS credentials must have permissions to call SageMaker AI APIs (creating endpoints, running benchmark and recommendation jobs). The skill generates code that runs under your credentials. No additional AWS Identity and Access Management (IAM) configuration is needed for the skill itself.

Note: For Kiro and Claude Code, agents can discover skills at runtime. They can search for and load skills on demand through the AWS MCP Server, without any local installation. Ask your agent: “Search for AWS skills related to databases.” Refer to the readme for discovering skills at runtime.

Option B: Use within Amazon SageMaker Studio

If you prefer to work inside a managed JupyterLab environment, you can use the skill in Amazon SageMaker Studio with a pre-configured image.

Step 1: Open Amazon SageMaker Studio. Navigate to Amazon SageMaker Studio in your target AWS account and AWS Region. Select your Studio domain and launch the Studio IDE from your user profile.

Step 2: Create a JupyterLab space. From the Studio landing page, choose JupyterLab , then choose Create JupyterLab space. Name the space (for example, my-inference-opt) and keep the sharing setting Private (skills only sync on private spaces). Under the Image menu, select the image that includes the SageMaker AI optimized generative AI inference skill. This image ships pre-configured with the aws-ai-ml agent skill and all necessary dependencies. Choose Run space and wait for it to boot (5–10 minutes the first time).

Note: Use a fresh space. A reused space with a locally modified skill version may not pick up the pre-configured image.

Step 3: Open JupyterLab and launch a terminal. After the space boots, open JupyterLab and choose Terminal.

Step 4: Authorize your coding agent. In the terminal, authenticate with your coding agent using your identity provider. For example, with Kiro:

kiro-cli login --license pro --identity-provider <your-IdP-start-URL> --region us-east-1 --use-device-flow

Step 5: Confirm and begin. Open your coding agent’s chat panel and ask: “What skills are available?” You should see aws-ai-ml listed. Once confirmed, describe your intent in natural language.

Troubleshooting:

If the agent reports no skills are available, verify that your space is set to Private. You can also check from the terminal:

If that directory is empty but /etc/sagemaker/skills/ contains the skill files, run restart-jupyter-server, refresh the page, and retry.

What you can do

The agentic experience covers the following capabilities across the inference optimization lifecycle. You don’t need to know which capability to invoke. Describe what you want, and your agent figures out the next step or asks clarifying questions if anything is unclear.

Benchmark an existing endpoint

If you already have a model deployed on a SageMaker AI endpoint, you can ask the agent to benchmark it. Tell the agent which endpoint you want to test, and it generates a Python notebook that runs a load test using the Workload.synthetic() and start_benchmark() APIs from the SageMaker Python SDK.

Before running any benchmark, the agent confirms that your endpoint is safe to load-test, because benchmarking drives real traffic to a live endpoint.

When the benchmark completes, you get a quantitative performance report with:

• Throughput : requests per second, output tokens per second.
• Latency : p50, p99, time-to-first-token, inter-token latency.
• Concurrency : number of simultaneous requests supported.

These are measured values from real load on real infrastructure, not estimates. The agent also recommends improvement mechanisms (such as prefill decoding) to boost performance.

Example prompt: “Benchmark my Llama endpoint on SageMaker AI.”

Find the right instance type for your model

If you have a model and need to find the right instance type to deploy it on SageMaker AI, tell your agent. It doesn’t matter where your model lives or how you obtained it:

• Fine-tuned or custom model in Amazon Simple Storage Service (Amazon S3) : You trained or downloaded a model and stored it in S3. Provide the S3 URI and your optimization goal. Example prompt: “I want to find the cheapest instance type to deploy my fine-tuned model on SageMaker.”
• Publicly available foundation model fromAmazon SageMaker JumpStart: You want to deploy a foundation model (FM) from the JumpStart catalog then provide the model ID. Example prompt: “Find the best instance for model huggingface-reasoning-qwen3-8b on SageMaker AI.”
• Model on theHugging Face Hub: You want to use a model hosted on Hugging Face. Provide the model name. For gated models (such as Llama variants), your agent surfaces the license terms and asks you to accept them and provide your Hugging Face token. Example prompt: “I want to deploy a Llama model from Hugging Face Hub. What’s the cheapest option?”

In every case, your agent generates code that evaluates your model against candidate instances and configurations, then presents ranked deployment options with concrete performance metrics: throughput, latency percentiles, time-to-first-token, and concurrency. You choose based on your cost and performance requirements.

Compare benchmark runs

If you ran multiple benchmarks (for example, before and after a configuration change, or across two instance types), you can ask the agent to compare them. Provide the two benchmark job names, and the agent generates a comparison that computes deltas across key metrics: throughput, latency percentiles, and time-to-first-token.

The results show percentage changes where positive means better, giving you a single, interpretable summary of whether your change improved performance, degraded it, or had no meaningful effect.

If one of the benchmark runs doesn’t exist, the agent offers to run it first before proceeding with the comparison.

Example prompt: “I have two benchmark runs and I want to compare them. Which one is faster?”

Benchmark results

This table compares two deployed models on the same benchmark workload (512/256 tokens, concurrency 4). The Δ% column shows how much faster Model B (Qwen3-8B) is than Model A (Qwen3-1.7B) on each metric. A positive value means Model B is better.

Metric	Qwen3-1.7B (Model A)	Qwen3-8B (Model B)	Δ%
Output token throughput	188.2 tokens/s	271.2 tokens/s	+44.1% ✅
Per-user throughput	47.5 tokens/s	69.4 tokens/s	+45.9% ✅
Request throughput	0.736 req/s	1.08 req/s	+46.7% ✅
Inter-token latency	20.9 ms	14 ms	+33.0% ✅
Request latency	5,382 ms	3,658 ms	+32.0% ✅
Time to first token	67.5 ms	166.3 ms	−146.5% ❌

The two models run on different hardware. Qwen3-8B uses a 4-GPU ml.g5.12xlarge (4x A10G), while Qwen3-1.7B uses a single L4 GPU (ml.g6.4xlarge). The deltas reflect approximately 4x the compute, not just the models themselves.

The takeaway: Qwen3-8B delivers approximately 44–47 percent higher throughput and lower end-to-end latency, largely thanks to the additional GPU compute. Qwen3-1.7B wins only on time-to-first-token, the expected advantage of a smaller model on a single GPU.

Putting it all together

You don’t need to memorize capability names or know which workflow to request. The following table shows how common requests map to outcomes.

You say	What you get
“I already deployed a model and want to know how fast it is.”	Quantitative performance report: throughput, latency percentiles, concurrency metrics from real load.
“I have a model in S3 and I don’t know what instance to deploy on.”	Ranked deployment options with cost, throughput, and latency for each candidate configuration.
“I want to deploy a JumpStart model and find the cheapest instance. I only have the model ID.”	Ranked deployment options, no S3 staging required. Gated-model alternative surfaced if needed.
“I ran a benchmark before and after a change. Which one is faster?”	Percentage change across all metrics indicating improvement or regression.
“I want to optimize a Llama model from Hugging Face Hub.”	License surfaced, model staged to S3, then standard recommendation output.

If your request spans multiple capabilities (for example, staging a Hugging Face model and then getting deployment recommendations), the agent chains them naturally within the same conversation.

What to expect from the agent

Your agent, equipped with the aws-ai-ml skill, follows a few behaviors that make the experience predictable and safe:

• It asks for what it needs. If information is missing (such as an endpoint name or S3 URI), the agent asks you to provide it rather than guessing.
• It confirms before impactful actions. Before running a benchmark that drives real traffic to a live endpoint, the agent warns you about the impact and asks for explicit confirmation.
• It tells you when something is out of scope. If you ask for something the agent can’t do (such as deploying a model), it explains what it can offer instead, such as generating the deployment configuration that you need.
• It generates SageMaker Python SDK v3 code. Every output is executable code you can inspect, modify, and run in your own environment.

Clean up

To avoid ongoing charges, delete the resources you created:

Conclusion

The aws-ai-ml skill for Amazon SageMaker AI optimized generative AI inference turns your existing coding agent into a SageMaker AI inference optimization expert. Whether you need to benchmark a live endpoint, find the cheapest instance for your model, compare configurations, or stage a Hugging Face model for evaluation, you describe what you want and your agent delivers measurable results.

To get started, install the skill through the Agent Toolkit for AWS and add it to the coding agent you already use, or launch a pre-configured JupyterLab space in Amazon SageMaker Studio. For more information, see the Amazon SageMaker AI documentation.

---

About the authors

Mona Mona

Mona currently works as Sr AI/ML specialist Solutions Architect at Amazon SageMaker AI Team. She is a published author of three books and her latest book is AI Agents on AWS. She has authored 20+ blogs on AI/ML and cloud technology and a co-author on a research paper on CORD19 Neural Search which won an award for Best Research Paper at the prestigious AAAI (Association for the Advancement of Artificial Intelligence) conference.

Aryaman Kothari

Aryaman is a Sr Product Manager at Amazon SageMaker AI, with a focus on enabling generative AI model inference and deployment on Amazon SageMaker HyperPod. A Chicago Booth graduate, Aryaman previously led product and go-to-market for Quantiphi’s Conversational AI business.

Muzart Tuman

Muzart is a software engineer using his experience in fields like deep learning, machine learning optimization, and AI-driven applications to help solve real-world problems in a scalable, efficient, and accessible manner. His goal is to create impactful tools that not only advance technical capabilities but also inspire meaningful change across industries and communities.