利用开源 Agent Skills 提升 HCLS AI 推理能力

基于基础模型（FM）构建的 AI Agent，即使在训练数据和系统提示词中见过相关指南，也常常会错误应用医疗健康与生命科学（HCLS）决策框架。让一个 Agent 使用 ACMG/AMP 标准对 TP53 错义变异进行分类：它会引用正确的框架，却可能错误使用证据类别、跳过人群频率阈值，或者虚构计算预测器评分。模型掌握事实，但缺少领域从业者经多年训练后内化的结构化推理流程。这一缺口会在变异解读、理赔审核、临床试验设计和医学影像分析中造成隐性失败。输出结果看似正确，却采用了错误标准，并可能带来监管和患者安全方面的后果。

本文分享一组覆盖 11 个 HCLS 领域的 38 项开源 Agent Skills，旨在弥合这一方法论缺口。我们将介绍安装方法，并展示如何在各类 Agentic AI 服务中使用这些 Skills。我们还分享评测结果，以说明它们能够在药物发现、医疗运营和医学影像工作流中带来可量化的改进。配备这些 Skills 的 Agent 在与未配备 Skills 的同类 Agent 的一对一比较中，胜率达到 70%–86%，具体取决于 Agent harness 的设置方式。最显著的效果体现在批判性思维方面，胜率为 78%–85%，效应量 d = 0.65–1.03。我们还将介绍如何针对特定用例自定义、扩展和创建自己的 Agent Skills。

解决方案概览

HCLS Agent Skills 集合中的 Agent Skills 是结构化 Markdown 文档（SKILL.md），通过渐进式披露（progressive disclosure）以 AI Agent 能够在推理时消费的格式编码领域决策流程。遵循 Agent Skills 开放标准，每项 Skill 都会在 YAML frontmatter 中声明触发条件、依赖项和元数据。随后的内容包含决策框架、参数表、代码模式和验证标准。该集合涵盖 11 个 HCLS 领域的 38 项 Skills，包括基因组学、药物发现、理赔运营和医学影像。完整清单按领域组织，参见完整 Skill 目录。所有 Skills 均以 MIT-0 许可证发布。

该存储库中的 Skills 分为推理型 Skills 和流水线型 Skills。推理型 Skills 编码用于指导 Agent 思考方式的方法论和决策框架。例如，genomic-variant-interpretation Skill 编码了完整的 ACMG/AMP 分类框架，包括证据类别、人群频率阈值和计算预测器截断值。流水线型 Skills 编码工具专属命令、经验证的参数和可生成可运行产物的代码模板。variant-calling Skill 提供了包含正确注释组、VQSR tranche 灵敏度目标和 Mutect2 肿瘤-正常样本配置的 GATK4 HaplotypeCaller 命令。

这种双重分类使 Agent 同时具备作出正确判断的能力，以及精确执行这些判断的技术能力。不同于检索增强生成（RAG）从已索引文档中检索有限片段来增强响应生成，Skills 直接编码决策流程本身及其错误条件。Skills 也不是微调；它们是会根据用户查询中的触发模式按上下文激活的结构化提示词。

Skills 与其他领域专业化方法相比具有三个特性：可审计、可移植且易于维护。每一项决策标准均以人类可读的 Markdown 格式呈现，而非隐藏在模型权重中。单项 Skill 无需针对每种服务分别定制，即可在超过 20 种服务中运行，包括 Amazon Bedrock AgentCore、AWS Strands Agents SDK、Kiro、Amazon Quick Desktop、Claude Code、OpenAI Codex 等。年度医疗政策变更或新的实验标准可以通过编辑文本文件快速反映，无需重新训练模型。

现在你已经了解 Skills 包含哪些内容，接下来开始设置它们。

前置条件

要跟随本文示例操作，你需要使用 AWS 支持的下列服务之一：Kiro 或 Kiro CLI，用于交互式 Skill 使用和多 Agent 编排；具备 Amazon Bedrock 基础模型访问权限的 AWS Strands Agents SDK；AgentCore harness，即 Amazon Bedrock AgentCore 的一项能力，并且需要已有 Agent 实现；或者用于基于 GUI 的 Skill 管理的 Quick Desktop。你也可以使用自己选择的编码 Agent harness，例如 Claude Code 或 OpenAI Codex。此外，你还需要 Python 3.10+、uv，以及用于克隆存储库的 Git。

首先克隆存储库：

git clone https://github.com/awslabs/hcls-agent-skills.git
cd hcls-agent-skills

如果只安装 Skills 而不安装 Agent 配置，请使用通用的 skills CLI：

npx skills add awslabs/hcls-agent-skills

对于 Kiro，install.sh 脚本会同时安装 Skills 以及配备这些 Skills 的预配置 Agent。该 Agent 会自动处理 Skill 路由，因此你无需按名称调用单个 Skill。运行 ./install.sh --target kiro，然后在 Kiro CLI 中使用 /agent hcls 切换至该 Agent。如需使用多 Agent 模式，请运行 ./install.sh --target kiro --mode multiagent，并使用 /agent hcls-multiagent。

对于 AWS Strands Agents SDK，可直接在 Python 代码中加载 Skills：

from strands import Agent
from strands.skills import AgentSkills

agent = Agent(
    model=model_id,
    skills=AgentSkills(skills="./skills/"),
)

对于 AgentCore，请按照 Skills 文档中的说明，为托管在 AgentCore 上的 Agent 添加 Agent Skills。AgentCore 提供托管、自动扩缩容、安全边界和可观测性能力。

对于 Amazon Quick Desktop，运行 ./install.sh --target quick-desktop 以查看在图形界面中添加 Skills 的完整说明。或者，请参阅 Amazon Quick 文档中的 Skills 说明。

解决方案演练

在安装 Skills 后，我们将展示三种部署模式：Quick Desktop 中最简单的单 Agent 方式、解决上下文工程挑战的 Kiro CLI 多 Agent 编排，以及基于 Amazon Bedrock AgentCore 的 Strands SDK 生产部署。随后，我们将展示三个示例用例，以突出 Skills 在真实 HCLS 工作流中带来的可量化差异。

在 Quick Desktop 中使用 Agent Skills

安装 Skills 后，Quick Desktop 的 Agent 无需额外配置即可获得结构化的 HCLS 领域推理能力。当你提出领域问题时，Agent 会根据查询中的触发模式自动激活相关 Skills。例如，提问“将 E11.42 编码为 E11.9 对 RAF 有何影响？”会触发 risk-adjustment Skill，Agent 将返回具体的 HCC 映射、层级解析和量化 RAF 差值，而不是泛泛建议“检查文档”。Skills 会被选择性激活；每次响应只触发相关 Skill，这有助于保持输出聚焦和准确。以下视频展示了在 Amazon Quick Desktop 中，系统针对该问题动态加载 Skill 的过程。

[视频/嵌入内容：Amazon Quick Desktop 聊天界面动态加载 risk-adjustment Skill，以回答 RAF 编码问题]

Kiro 中的多 Agent 架构

将全部 38 项 Skills 加载到单个 Agent 上下文中会消耗约 80K tokens。对于大上下文模型而言这可以运行，但会带来上下文工程挑战。Agent 必须在每次查询时从 38 项可用 Skills 中选择正确子集，而无关的 Skill 内容会争夺注意力。另一种方法是显式调用 Skill，例如 /risk-adjustment，但这要求你在提问前就知道应该调用哪项 Skill，而这恰恰是 Skills 试图弥合的专业知识缺口。

Kiro CLI 的多 Agent 架构解决了这两个问题。一个轻量级协调 Agent（不加载 Skills）会将查询路由至八名领域专家 Agent；每名专家仅加载与其相关的 Skills，每名专家约占用 15K tokens。协调 Agent 负责意图分类，专家 Agent 负责领域推理。专业化分工可按照下表定义。

Kiro CLI 多 Agent 架构中的八名领域专家 Agent 及其所分配的 Skills

多 Agent 配置定义在 JSON Agent 文件中。路由逻辑请参见协调 Agent 配置，了解如何将领域 Skills 附加至专家 Agent 的示例请参见专家 Agent 配置。以下视频展示了 Kiro CLI 在代码库中工作时，如何借助多 Agent 路由和动态 Skill 激活回答复杂的药物再利用问题。

[视频/嵌入内容：Kiro CLI 使用多 Agent 路由和动态 Skill 激活，在代码库中回答药物再利用问题]

Strands SDK 集成

AWS Strands Agents SDK 提供原生 Skill 加载能力，可用于构建自定义 HCLS Agent：

from strands import Agent
from strands.skills import AgentSkills
from strands.multiagent import MultiAgentOrchestrator

# Define domain specialists with their skill sets
genomics_agent = Agent(
    name="hcls-genomics",
    model=model_id,
    skills=AgentSkills(skills="./skills/genomics/"),
)

imaging_agent = Agent(
    name="hcls-imaging",
    model=model_id,
    skills=AgentSkills(skills="./skills/imaging/"),
)

# Coordinator routes to specialists
coordinator = MultiAgentOrchestrator(
    agents=[genomics_agent, imaging_agent, ...],
    model=model_id,
)

response = coordinator("Classify NM_000546.6:c.743G>A in TP53 using ACMG criteria")

部署至 Amazon Bedrock AgentCore

当配备 Skills 的 Agent 在本地运行正常后，你可以将其迁移至生产环境。Amazon Bedrock AgentCore 提供了另一种向托管 Agent 注入 Skills 的路径。除了将 Skills 嵌入 Strands Agent 代码外，你还可以在环境层面配置 Skills，使其可供在该 harness 中运行的 Agent 使用。AgentCore harness 提供托管、自动扩缩容、安全边界和可观测性能力，无需管理基础设施。请参阅 AgentCore 文档中的 Skills。

完成部署介绍后，接下来看看配备 Skills 的 Agent 在实践中会产生怎样的结果。以下示例用例取自我们的评测提示词集。

用例 1：在药物发现中评估罕见纤维化疾病的药物再利用候选物

某生物技术公司的团队正在研究针对特发性肺纤维化（IPF）的药物再利用，希望评估通过受体激酶 TGFBR1（ALK5）调节 TGF-β1 信号传导的已获批药物。在实践中，研究人员需要查询药物-基因相互作用数据库，按证据强度对候选物进行排序，评估其作用机制与 IPF 病理生理学的重叠程度，并根据现有安全性数据判断其可转化性。然而，研究人员可能会快速而模糊地向 Agent 提问：“我正在研究 TGFBR1 作为 IPF 的治疗靶点。是否有值得再利用的已获批药物？最有力的候选药物是什么，临床转化的现实性如何？”

在添加 Skills 之前，Agent 会提供一篇概括性的文献综述，列出已知的 TGFBR1 抑制剂，却没有结构化排序标准、证据层级或可转化性评估框架。为 Agent 配备 Skills 后，Agent 会触发 drug-repurposing 和 translational-research Skill，并执行以下操作：

1. Agent 应用 DGIdb 查询框架，优先考虑相互作用类型（抑制剂 > 调节剂 > 结合剂），并优先采用 ChEMBL、DrugBank 等来源数据库，而非置信度较低的来源。
2. 它使用结构化证据层级对候选物排序：直接靶点结合证据优先于通路层面证据，通路层面证据又优先于表型关联；既有适应症相关性则作为修正因素。
3. 它通过将 TGFBR1 抑制映射至 IPF 的关键病理过程来评估作用机制重叠，包括成纤维细胞向肌成纤维细胞转化、上皮-间质转化和细胞外基质沉积。
4. 它使用 T0→T1 标准评估临床可转化性，考察原始适应症的既有安全性数据、治疗窗兼容性，以及现有临床前

AI agents built on foundation models (FMs) often misapply healthcare and life sciences (HCLS) decision frameworks, even when they’ve seen the guidelines in training and in the system prompt. Ask an agent to classify a TP53 missense variant using ACMG/AMP criteria. It will cite the correct framework but misapply evidence categories, skip population frequency thresholds, or hallucinate computational predictor scores. The model knows facts but lacks the structured reasoning procedures that domain practitioners internalize over years of training. The gap produces silent failures across variant interpretation, claims adjudication, clinical trial design, and imaging analysis. Outputs look correct but apply wrong criteria, with regulatory and patient safety consequences.

In this post, we share a collection of 38 open source agent skills spanning 11 HCLS domains that help close this methodology gap. We walk through installation and show how to use them across agentic AI services. We share our evaluation results to demonstrate measurable improvement across drug discovery, healthcare operations, and medical imaging workflows. Agents equipped with these skills win 70–86 percent of head-to-head comparisons against the same agents without skills, varying agent harness setup. The strongest effect is on critical thinking (78–85 percent win rate, d = 0.65–1.03). We also show you how you can customize, extend, and create your own agent skills for your specific use case.

Solution overview

Agent skills in the HCLS Agent Skills collection are structured markdown documents (SKILL.md) that encode domain decision procedures into a format AI agents can consume at inference time through progressive disclosure. Following the Agent Skills open standard, each skill declares triggers, dependencies, and metadata in YAML frontmatter. The content that follows contains decision frameworks, parameter tables, code patterns, and validation criteria. The collection covers 38 skills across 11 HCLS domains including genomics, drug discovery, claims operations, and medical imaging. Refer to the full skill catalog for the complete list organized by domain. All are released under the MIT-0 license.

Skills in this repository are sorted as either reasoning or pipeline skills. Reasoning skills encode methodology and decision frameworks that guide how the agent thinks. For example, the genomic-variant-interpretation skill encodes the full ACMG/AMP classification framework such as evidence categories, population frequency thresholds, and computational predictor cutoffs. Pipeline skills encode tool-specific commands, validated parameters, and code templates that produce runnable artifacts. The variant-calling skill provides GATK4 HaplotypeCaller commands with correct annotation groups, VQSR tranche sensitivity targets, and Mutect2 tumor-normal configurations.

This dual taxonomy gives agents both the judgment to make correct decisions and the technical precision to execute them. Unlike Retrieval Augmented Generation (RAG), which retrieves limited passages from indexed documents to augment the response generation, skills encode the decision procedure and error conditions itself. Skills are not fine-tuning either. They’re structured prompts that activate contextually based on trigger patterns in the user’s query.

Three properties make skills distinct from other approaches to domain specialization. Skills are auditable, portable, and straightforward to maintain. Every decision criterion is human-readable in markdown format, not hidden in model weights. A skill works across over 20 services (Amazon Bedrock AgentCore, AWS Strands Agents SDK, Kiro, Amazon Quick Desktop, Claude Code, OpenAI Codex and more) without customization to each. Annual medical policy changes or new experiment criteria can be reflected quickly by editing a text file, not retraining a model.

Now that you understand what skills contain, let’s set them up.

Prerequisites

To follow along with the examples in this post, you need one of the supported services from AWS: Kiro or Kiro CLI for interactive skill use and multi-agent orchestration, the AWS Strands Agents SDK with Amazon Bedrock foundation model access, AgentCore harness, a capability of Amazon Bedrock AgentCore, with an existing agent implementation, or Quick Desktop for GUI-based skill management. You can also use a coding agent harness of your choice such as Claude Code or OpenAI Codex. You also need Python 3.10+ with uv, and Git for cloning the repository.

Start by cloning the repository:

git clone https://github.com/awslabs/hcls-agent-skills.git
cd hcls-agent-skills

To install skills only without the agent configuration, use the universal skills CLI:

npx skills add awslabs/hcls-agent-skills

For Kiro, the install.sh script installs both skills and a pre-configured agent that equips them. The agent handles skill routing automatically, so you don’t need to invoke individual skills by name. Run ./install.sh --target kiro, then switch to the agent in Kiro CLI with /agent hcls. For multi-agent mode, run ./install.sh --target kiro --mode multiagent and use /agent hcls-multiagent.

For the AWS Strands Agents SDK, load skills directly in your Python code:

from strands import Agent
from strands.skills import AgentSkills

agent = Agent(
    model=model_id,
    skills=AgentSkills(skills="./skills/"),
)

For AgentCore, follow Skills to add agent skills to an AgentCore-hosted agent. AgentCore provides managed hosting, auto scaling, security boundaries, and observability capabilities.

For Amazon Quick Desktop, run ./install.sh --target quick-desktop to see the full instructions for adding skills in the graphical interface. Alternatively, follow the instructions in Skills in the Amazon Quick documentation.

Solution walkthrough

With skills installed, we demonstrate three deployment patterns: the simplest single-agent approach in Quick Desktop, multi-agent orchestration in Kiro CLI that addresses context engineering challenges, and production deployment with Strands SDK on Amazon Bedrock AgentCore. We then show three sample use cases that highlight the measurable difference skills make in real HCLS workflows.

Agent skills in action with Quick Desktop

With skills installed, Quick Desktop’s agent gains structured HCLS domain reasoning without additional configuration. When you ask a domain question, the agent automatically activates relevant skills based on trigger patterns in your query. For example, asking “What is the RAF impact of coding E11.9 instead of E11.42?” triggers the risk-adjustment skill and the agent responds with specific HCC mappings, hierarchy resolution, and quantified RAF deltas rather than a generic suggestion to “review documentation.” Skills are activated selectively. Only the relevant skill is triggered for the response, helping keep output focused and accurate. The following video shows a skill dynamically loaded for the question in Amazon Quick Desktop.

Amazon Quick Desktop chat with the risk-adjustment skill dynamically loaded to respond to a RAF coding question

Multi-agent architecture with Kiro

Loading all 38 skills into a single agent context consumes ~80K tokens. This is workable with large-context models, but it creates a context engineering challenge. The agent must select the right subset from 38 available skills on every query and irrelevant skill content competes for attention. An alternative is explicit skill invocation (for example, /risk-adjustment), but this requires you to know which skill to invoke before asking your question, which is exactly the expertise gap skills are meant to bridge.

Kiro CLI’s multi-agent architecture solves both problems. A lightweight coordinator agent (no skills loaded) routes queries to eight domain specialists, each loading only its relevant skills (approximately 15K tokens per specialist). The coordinator handles intent classification while the specialists handle domain reasoning. The specialization can be defined as described in the following table.

Table of the eight domain specialist agents in the Kiro CLI multi-agent architecture and the skills assigned to each

The multi-agent configuration is defined in JSON agent files. Refer to the coordinator agent config for the routing logic, and a specialist agent config for an example of how domain skills are attached to a specialist. The following video shows multiagent and dynamic skill activation answering a complex drug repurposing question while working in a code base in Kiro CLI.

Kiro CLI answering a drug repurposing question in a code base using multi-agent routing and dynamic skill activation

Strands SDK integration

The AWS Strands Agents SDK provides native skill loading for building custom HCLS agents:

from strands import Agent
from strands.skills import AgentSkills
from strands.multiagent import MultiAgentOrchestrator

# Define domain specialists with their skill sets
genomics_agent = Agent(
    name="hcls-genomics",
    model=model_id,
    skills=AgentSkills(skills="./skills/genomics/"),
)

imaging_agent = Agent(
    name="hcls-imaging",
    model=model_id,
    skills=AgentSkills(skills="./skills/imaging/"),
)

# Coordinator routes to specialists
coordinator = MultiAgentOrchestrator(
    agents=[genomics_agent, imaging_agent, ...],
    model=model_id,
)

response = coordinator("Classify NM_000546.6:c.743G>A in TP53 using ACMG criteria")

Deploying to Amazon Bedrock AgentCore

After your skill-equipped agent works locally, you can move it to production. Amazon Bedrock AgentCore provides an alternative path to inject skills into hosted agents. In addition to embedding them in the Strands agent code, you can configure skills at the environment level so they’re available to agents running in that harness. AgentCore harness provides managed hosting, auto scaling, security boundaries, and observability capabilities without managing infrastructure. Follow Skills in the AgentCore documentation.

With deployment covered, let’s look at what skill-equipped agents produce in practice. The following sample use cases are drawn from our evaluation prompt set.

Use case 1: Evaluating repurposing candidates for rare fibrotic disease in drug discovery

A team at a biotech company investigating drug repurposing for idiopathic pulmonary fibrosis (IPF) wants to evaluate approved drugs that modulate TGF-β1 signaling through the receptor kinase TGFBR1 (ALK5). In practice, a researcher needs to query drug-gene interaction databases, rank candidates by evidence strength, assess mechanism-of-action overlap with IPF pathophysiology, and determine translatability given existing safety data. However, a researcher might be quick to prompt an agent vaguely: “I’m investigating TGFBR1 as a therapeutic target for IPF. Are there any approved drugs worth repurposing? What’s the strongest candidate and how realistic is clinical translation?”

Before adding skills, the agent provides a general literature review listing known TGFBR1 inhibitors without structured ranking criteria, evidence hierarchy, or translatability assessment framework. After equipping the agent with skills, the agent triggers drug-repurposing, and translational-research skill and does the following:

1. The agent applies the DGIdb query framework, prioritizing interaction types (inhibitor > modulator > binder) and source databases (ChEMBL, DrugBank) over lower-confidence sources.
2. It ranks candidates using a structured evidence hierarchy where direct target engagement outweighs pathway-level evidence, which in turn outweighs phenotypic association, with existing indication relevance applied as a modifier.
3. It assesses mechanism-of-action overlap by mapping TGFBR1 inhibition to the key IPF pathological processes: fibroblast-to-myofibroblast transition, epithelial-mesenchymal transition, and extracellular matrix deposition.
4. It evaluates clinical translatability using T0→T1 criteria, examining existing safety data from the original indication, therapeutic window compatibility, and concordance between available preclinical fibrosis models and human disease.

The skill chain transforms a surface-level response into a structured regulatory-aware evaluation with quantified evidence rankings.

Use case 2: Building a CMS-HCC risk adjustment pipeline in healthcare claims operations

A Medicare Advantage plan with 12,000 members needs to calculate Risk Adjustment Factor (RAF) scores from ICD-10 diagnosis claims data using CMS-HCC Model V28 coefficients. The pipeline must apply the ICD-10-to-HCC crosswalk, resolve disease hierarchies correctly, and compute final member-level risk scores with demographic adjustments. However, a junior analyst may prompt the agent: “We’re a Medicare Advantage plan with 12,000 members. We have ICD-10 diagnosis claims in a PostgreSQL database (member_diagnoses and member_demographics tables). Build me a pipeline to calculate member-level RAF scores for the current payment year.”

Before adding skills, the agent produces a plausible but incomplete pipeline, often missing hierarchy resolution entirely, using outdated V24 coefficients, or applying hierarchies after summing (which inflates scores). After equipping the agent with skills, the agent triggers risk-adjustment, and claims-billing-rules skill and does the following:

1. The agent generates correct SQL that joins diagnosis codes to the ICD-10-to-HCC crosswalk table with deduplication within the measurement year, making sure each HCC is counted only once per member.
2. It implements V28 hierarchy resolution correctly, where HCC 18 (Diabetes with Chronic Complications) supersedes HCC 19 (Diabetes without Complications) and HCC 326 (CKD Stage 5) supersedes HCC 327 (CKD Stage 4), helping prevent double-counting at multiple specificity levels.
3. It applies the correct demographic segmentation by categorizing members into community, institutional, or dual-eligible populations with age/sex adjustments before summing HCC coefficients.
4. It proactively explains that skipping hierarchy resolution double-counts conditions at multiple specificity levels, systematically inflating RAF scores and creating audit liability under CMS RADV review.

The skill supports producing audit-defensible RAF scores rather than inflated estimates that would trigger CMS RADV audit findings.

Use case 3: T1-weighted MRI preprocessing for voxel-based morphometry in medical imaging research

A neuroimaging study with 45 healthy adults needs a standard T1w preprocessing pipeline for voxel-based morphometry (VBM) analysis. Raw DICOM data has been converted to NIfTI. The pipeline must reorient, correct bias field, skull-strip, and register to MNI152 space in the correct order and with parameters appropriate for healthy adult brains in an FSL/ANTs hybrid environment. A researcher may prompt the agent: “I have 45 healthy adult T1w scans that need preprocessing for a VBM analysis. Build a pipeline using FSL and ANTs.”

Before adding skills, the agent suggests a reasonable pipeline but may order bias correction after skull stripping (which biases brain masks), use inappropriate thresholds, or omit failure mode detection strategies. After equipping the agent with skills, the agent triggers radiology-preprocessing, and imaging-study-design skill and does the following:

1. The agent specifies the correct processing order with justification: reorient to standard space, then bias field correction before skull stripping, then brain extraction with parameters tuned for healthy adults, and finally registration to the MNI152 template.
2. It explains the critical ordering dependency. Intensity inhomogeneity at brain borders causes the skull-stripping algorithm to remove too much or too little tissue if bias correction hasn’t been applied first, particularly in temporal and frontal regions.
3. It provides a complete bash script with error checking at each stage and quality control outputs for visual inspection of intermediate results.
4. It documents failure modes at each step: incorrect orientation metadata, residual signal shading near surface coils, neck tissue inclusion when extraction thresholds are too permissive, and registration failure at ventricular boundaries in older subjects.

The skill catches the ordering dependency that would introduce systematic bias into the VBM analysis.

These use cases illustrate how HCLS skills reshape agent behavior qualitatively to produce more domain-aligned responses, but there’s always a question of how much better it is for researchers and developers.

Evaluation results

We conducted a pairwise evaluation to measure skill impact across 410 domain prompts (380 single-skill and 30 cross-skill) using two harness configurations. One of the two agent harnesses is Kiro CLI in which the Auto model is used to allow Kiro to select an optimal model for the task. The agent in Kiro has access to a thinking tool and file-read operations. The other agent configuration is an agent built with the AWS Strands Agents SDK using Agent(model=BedrockModel(...), callback_handler=None) with the model explicitly pinned to Claude Sonnet 4.6. The skills condition additionally loads an AgentSkills(skills="./skills/") plugin. A think tool is provided to both conditions symmetrically. The configuration can be found in eval/execute.py. In both configurations, two conditions are compared. One is the baseline agent who does not have access to any skill. The other one is the skilled agent who is invoked with all 38 skills available through progressive loading.

We employ five scoring dimensions for the large language model (LLM) judge to measure how skills impact the agent’s response. Scientific accuracy evaluates the correctness of facts, mechanisms, citations, and domain knowledge. Coherence assesses whether the response follows a logical structure with a clear reasoning chain and internal consistency. Relevance measures how well the agent addresses all parts of the prompt at appropriate depth while staying on topic. Critical thinking captures the agent’s ability to challenge assumptions, identify limitations, and consider alternatives rather than presenting a single uncritiqued response. Actionability scores the agent’s ability to provide concrete next steps, specific parameters, and runnable commands that a practitioner could act on immediately. We use Claude Opus 4.7 from Amazon Bedrock as the judge. The full scoring prompt is in eval/judge.py.

The judge scores each dimension with 0–100 scale. However, LLM judges exhibit score compression, a phenomenon where the scores cluster in a certain range, making raw deltas (for example, +1.5) difficult to interpret. We therefore report two primary metrics. Firstly, win rate (WR), a percentage of prompts where the skills condition scored higher than the baseline, is reported. This is an intuitive measure and robust to scale compression. Secondly, Cohen’s d effect size (d in short) is reported. This is to measure the standardized difference between two group means. It’s computed with mean delta divided by the pooled standard deviation. This measures how large the improvement is relative to natural variance. General interpretation of Cohen’s d is 0.2 (small), 0.5 (medium), and 0.8 (large).

Overall, skills win 69.5–85.9 percent of head-to-head comparisons in the two agent harness configurations. Skills improve critical thinking, actionability, and scientific accuracy in both configurations. The strongest signal is on critical thinking, confirming that skills’ primary contribution is methodological: teaching the agent which frameworks to apply, which assumptions to challenge, and which limitations to flag, rather than adding factual content the base model may already possess. The following table summarizes the high-level results.

Metric	Kiro CLI	Strands Agent
Prompts evaluated	410	410
Skills overall WR (d)	69.5% (0.39)	85.9% (0.97)
Critical thinking WR (d)	78.0% (0.65)	85.1% (1.03)
Scientific accuracy WR (d)	69.3% (0.34)	86.2% (0.85)
Actionability WR (d)	68.0% (0.37)	77.3% (0.56)
Baseline-benefit correlation (r)	-0.59	-0.61
Max variance reduction	-61.9%	-52.1%

Effect by baseline strength

There is strong evidence showing agent skills help the most when the base agent struggles the most. The Pearson correlation between baseline response quality and skill benefit is −0.59 in Kiro CLI and −0.61 in Strands agent. We categorize the prompts based on the baseline agent’s overall scores into three tiers, weak, medium, and strong. Most prompts fall in the medium tier where skills provide clear benefit. Prompts that fall in the strong tier where the model already performs well get marginal improvement from agent skills. The weak tier represents cases where the base agent struggles. These span multiple domains (clinical data, healthcare operations, genomics) and typically involve multi-step regulatory procedures or niche methodology the model approximates rather than applies precisely.

However, the strong tier finding is not absolute. Cross-domain reasoning skills achieve 80 percent win rate even at a strong baseline of 90.2, demonstrating that well-designed methodology frameworks add value across the quality spectrum when they teach a decision procedure the model wouldn’t apply on its own.

The overall scores for baseline and skilled agents by baseline strength are shown in the following tables.

In Kiro CLI

Baseline Tier	N	Baseline (mean±sd)	Skills (mean±sd)	Delta	Win Rate
Weak (<80)	15	75.8±4.0	84.4±6.3	+8.7	87%
Medium (80-90)	227	86.8±2.5	89.1±3.3	+2.3	79%
Strong (>90)	168	91.3±1.0	91.0±2.5	-0.3	55%

In Strands agent

Baseline Tier	N	Baseline (mean±sd)	Skills (mean±sd)	Delta	Win Rate
Weak (<80)	46	77.0±2.6	84.9±3.5	+7.9	96%
Medium (80-90)	325	85.8±2.6	89.5±3.3	+3.7	89%
Strong (>90)	39	91.0±0.7	91.1±2.0	+0.0	54%

Variance reduction

Skills also reduce the standard deviation of response scores across all sample prompts within a domain. For example, standard deviation of judge’s scores on clinical-data responses drops from 6.8 to 3.3 with skills loaded in the Kiro CLI, a 51 percent reduction. This means skills make outputs more consistent, adhering to the knowledge and framework encoded in the skill. In regulated HCLS workflows where consistency matters as much as average quality, this variance reduction is a meaningful benefit.

The full evaluation methodology, prompts, and raw results are available in the evaluation technical report.

Extending skills and building new ones

These results are for the 38 skills shipped in the repository. Your team’s workflows may need different thresholds, additional protocols, or entirely new domain coverage. The repository includes three guides for skill customization:

• CUSTOMIZING.md covers practical workflows for modifying, extending, and creating skills from scratch, including adding organization-specific rules (LCD codes, formulary step therapy, internal protocols) to existing skills.
• SKILL_DESIGN_GUIDE.md documents evidence-based patterns for writing effective skills: decision trees, threshold tables, gotcha lists, response format sections, and the structural features that correlate with high evaluation win rates.
• QUALITY_CHECKLIST.md provides a pre-merge quality checklist covering frontmatter, structure, content quality, and testing requirements.

To customize a skill for your organization, start by copying the relevant SKILL.md and modifying decision thresholds, adding org-specific protocols, or removing irrelevant sections. Use the design guide to structure new content for optimal outcomes. For example, reasoning skills benefit from decision trees and numbered procedures while pipeline skills provide strong value with parameter tables and version-specific gotchas.

To test your changes, first generate evaluation prompts that reflect your team’s actual workflows, then run the evaluation framework:

uv venv --python 3.12 && source .venv/bin/activate
uv pip install -e ".[dev]"
# Generate 30 evaluation prompts for each of the skills
python eval/generate_prompts.py --count 30
# Run pairwise evaluation against your modified skills
python -m eval.run --skills ./my-custom-skills/ --parallel 2
python eval/build_review.py
open eval/results/review.html

The following video shows the review.html evaluation dashboard, which includes scoring, domain breakdown, prompts, and responses for each evaluation.

The review.html evaluation dashboard showing scores, domain breakdown, prompts, and responses for each evaluation

The evaluation produces a pairwise comparison showing whether your modified skill improves responses relative to the baseline, with per-dimension breakdowns identical to the results reported earlier. Contribute new domain skills back to the open-source collection following the contribution guide.

Clean up

Skills are local files and do not create AWS resources. No teardown is required for the examples in this post. If you deployed a Strands agent to Amazon Bedrock AgentCore harness, follow the Clean up step to delete the deployed agent resources.

Conclusion

Agent skills give your HCLS agents structured domain methodology. The decision frameworks, regulatory thresholds, and common-mistake checklists in an agent skill can turn approximate answers into precise ones. With 38 skills covering genomics, drug discovery, claims operations, medical imaging, and more, you can deploy domain-expert agents on Quick Desktop, Kiro, or Strands SDK-based architecture running on Amazon Bedrock AgentCore.

The measurable impact speaks for itself. Our evaluation demonstrated a 70–86 percent win rate for the skilled agent, with the strongest gains on critical thinking (d = 0.65) and the highest lift on the hardest queries where unguided agents fail most. Skills are auditable text files you can customize for your organization’s protocols and test with the included evaluation framework.

To get started:

---

About the author