扩展 Agentic AI：避免供应商锁定的企业级模式

在整个企业中扩展 Agentic AI，需要采用既能保持灵活性、又能避免供应商锁定的架构模式。本文是我们“大规模多 Agent 系统”系列的第 2 部分。本文将探讨机器学习（ML）团队如何在由框架、模型和供应商构成的“多重一切”（multi-everything）环境中运行 Agentic AI 系统。我们还将介绍使这些系统能够协同扩展的原则。

在《面向大规模 Amazon 多 Agent 编排模式的高级微调技术》一文中，我们探讨了如何在单一用例或领域内设计并优化多 Agent 编排。该文聚焦于这样一些场景：需要多个 Agent 通过协调工作流、分解任务，以及借助结构化协作提升准确性，来应对复杂问题。

然而在实践中，企业 AI 系统很少会始终局限于单一领域。

随着采用规模扩大，大型企业中的 ML 平台团队会面临另一类挑战。问题不再是如何在一个系统内编排 Agent，而是如何在“多重一切”环境中运行许多这类系统。多个框架、模型、供应商和团队共存于同一家企业中，并且各自以不同的速度演进。

随着组织扩展这些系统，能否持续一致地构建、定制和部署模型变得至关重要。在实践中，这需要采用统一的方法进行模型生命周期管理和大规模推理。Amazon SageMaker 在这一领域发挥基础性作用：它能够支持企业范围内的一致性，同时不限制灵活性。

本文探讨在保留灵活性并避免供应商锁定的同时，扩展 Agentic AI 系统所需的架构原则和模式。

“多重一切”环境的现实

企业 AI 系统默认会演变为异构格局。不同团队会根据自身需求采用不同框架：有些团队优先考虑结构化工作流，有些团队专注于协作式 Agent 交互，还有一些团队则优化确定性的、由模型驱动的流水线。同时，组织会将自建 Agent 与软件即服务（SaaS）能力及既有企业系统相结合。

模型层又引入了另一维度的可变性。基础模型（FM）仍在快速演进，每种模型都在成本、延迟和能力之间提供不同权衡。因此，大多数企业会跨多个模型提供商开展运营，而不是统一采用单一选项。

随着时间推移，这会形成一种稳态现实：多模型、多框架、多供应商的系统，服务于多个团队和用例。

挑战不在于如何避免这一结果，而在于如何在不引入碎片化的前提下管理这一结果。

将选择权视为需要管理的约束

在第 1 部分中，我们聚焦于优化特定系统中的 Agent 行为。在企业层面，问题发生了转变。选择权不再只是用于实验，而成为必须被有意识管理的一项约束。

试图在框架或模型层面强制标准化，往往会带来摩擦。团队可能绕过这些约束，采用速度可能放缓，或者系统可能偏离已批准的架构而发展。与此同时，将应用程序紧密耦合于特定模型或供应商，会限制企业在生态持续演变时的适应能力。

更有效的方法是在应用层之下实施标准化，聚焦于身份、策略执行、可观测性和路由等共享控制平面，同时允许 Agent 的构建和执行方式保持灵活。

这种方法不会消除异构性，而是限制其影响，使系统能够演进而不破坏更广泛的架构稳定性。

“多重一切”系统的核心挑战

随着系统多样性增加，一组可预测的挑战会随之出现。由于各框架都有自己的控制模型，治理很难在不同框架之间得到一致执行。随着 Agent、工具和服务暴露出不兼容的接口，集成复杂度不断提升。若缺少动态优化，成本和性能权衡会更难管理，常常导致资源利用效率低下。

与此同时，随着 Agent 动态地与工具、数据和其他 Agent 交互，安全边界不断扩大，访问模式变得更难预测。持久化记忆还会带来数据保留、隔离和一致性方面的额外复杂性。最后，企业用例需要特定领域的性能，而这无法仅通过通用配置实现。

这些挑战彼此关联，并会随时间推移相互叠加。要管理它们，需要采用系统级方法，而不是彼此孤立的解决方案。

用于大规模管理复杂性的实用准则

下图概述了在成功的“多重一切”环境中反复出现的核心架构原则：控制平面与执行平面分离、统一可观测性、集中治理、动态路由、韧性设计、分阶段的编排演进以及内置优化。

图 1：管理“多重一切”环境复杂性的核心架构原则

能够成功运营“多重一切”环境的组织，通常会收敛于一组在灵活性与控制之间取得平衡的架构原则。

一项基础原则是将控制平面与执行平面分离。身份、策略执行、可观测性和成本归因应集中管理，以促进企业范围内的一致性；而 Agent 的执行和开发则保持去中心化，以支持团队自主性和可扩展性。

可观测性是有效运营这些系统的前提。通过建立统一遥测层，组织可以洞察跨框架和跨环境的 Agent 行为。这种可见性可帮助其监控性能、追踪故障，并持续改进系统，而无需依赖框架专属工具。

当治理被实现为平台能力，而不是嵌入各个 Agent 时，效果最为显著。即使框架、模型和执行环境不断演进，集中式执行也能支持一致的安全与合规管理。

随着工作负载日益多样化，路由成为核心系统功能。组织不应静态分配模型或基础设施，而应根据成本、延迟和准确性要求，动态地将任务匹配至资源。这有助于系统实时适应，并在大规模运行时保持效率。

生产系统还必须围绕明确的保障进行设计。应清晰定义延迟、可用性和隔离性要求，并配备故障处理机制。重试、熔断器和回退路径有助于系统在真实世界条件下保持韧性。

许多组织起初采用集中式编排模型，以维持对 Agent 交互的可见性和控制力。随着系统扩展，它们会向更分布式、事件驱动的架构演进；这种架构在保持一致性的同时，提供更高的可扩展性。

最后，优化必须从一开始就嵌入系统。成本和性能考量是大规模运营的基础；动态模型选择、缓存和高效执行模式等技术，有助于实现长期效率。

这些原则共同构成了一套实用框架，用于管理“多重一切”环境固有的复杂性。

如何利用 AWS 服务实现框架无关的扩展

前文概述的架构原则，需要能够跨框架、模型和团队一致运行的能力。AWS 服务提供了这些构建模块，组织可借此实现框架无关的平台，同时保持灵活性。

在大规模运行时，模型层成为复杂性的主要来源之一。不同用例需要不同的模型、定制策略和推理模式。

Amazon SageMaker 是大规模管理这种复杂性的核心执行与定制层。它为模型开发、微调、部署和推理提供统一层，使组织能够在企业范围内标准化模型的构建和运营方式。通过支持实时、异步和批量推理，以及Inference Components和模型监控等能力，Amazon SageMaker 可帮助团队使基础设施与工作负载需求相匹配，从而在企业内部维持一致的运营模型。

与之互补的是，Amazon Bedrock提供了用于访问基础模型的简化托管式接口，使团队能够在无需管理底层基础设施的情况下快速实验。Amazon Bedrock 可加速模型访问和集成，而 Amazon SageMaker 则提供了模型定制和生产级推理所需的深度控制能力与可扩展性。二者结合，使组织能够将模型访问与模型执行分离，支持灵活且具备韧性的架构。

AWS Lambda、AWS Step Functions 和 Amazon API Gateway 等服务可用于实现编排和控制，它们支持动态路由和工作流协调。

AWS 上的 Agent 编排等新兴能力进一步扩展了这一层。它们提供面向 Agent 编排的专用抽象，帮助团队以更高的一致性和控制力管理复杂的多 Agent 工作流。

身份和治理仍通过 AWS Identity and Access Management（IAM）与 AWS Organizations 进行集中管理；可观测性则通过 Amazon CloudWatch 和 AWS X-Ray 实现标准化。

Amazon EventBridge、Amazon ElastiCache 和 Amazon CloudFront 支持集成与性能优化。

在这一架构中，Amazon SageMaker 实际上成为模型执行的运营骨干，而 Amazon Bedrock 则加速了对新兴基础模型能力的访问。二者共同帮助组织在创新与控制之间取得平衡。

核心要点
Amazon SageMaker 为大规模模型定制和推理提供运营骨干，而 Amazon Bedrock 支持快速访问托管基础模型。二者共同支持企业规模的灵活、框架无关架构。

总结：将架构原则映射至 AWS 服务

下表概述了这些架构原则如何映射到支持框架无关扩展的原生 AWS 服务。

架构原则	支持内容	AWS 服务
集中式身份与治理	跨框架和团队的一致策略执行	IAM、AWS Organizations
统一可观测性与遥测	跨 Agent 和工作流的端到端可见性	Amazon CloudWatch、AWS X-Ray
模型抽象与选择权	将应用程序与模型提供商解耦	Amazon Bedrock
大规模模型定制与推理	跨工作负载的标准化训练、微调和可扩展推理	Amazon SageMaker
动态路由与编排	实时优化	AWS Lambda、AWS Step Functions、Amazon API Gateway、Amazon Bedrock AgentCore
事件驱动集成	解耦通信	Amazon EventBridge
性能优化	高效扩展	Amazon ElastiCache、Amazon CloudFront

“多重一切”系统的企业模式

当这些原则得到应用时，组织通常会收敛到少数几种架构模式。这些模式并非规定性的要求，而是反映了团队如何基于工作负载需求和运营约束来构建 Agent 系统。

重要的是，这些模式并不相互排斥。大多数企业会在不同业务部门和用例中组合实施它们。挑战不在于选择单一模式，而在于让它们能够在共享环境中共存。

模式 1：用于业务流程自动化的内部 Agent 平台

一个常见起点是内部 Agent 平台。当多个团队开始独立构建 Agent 时，这种模式便会出现。随着时间推移，这可能导致基础设施重复、治理不一致，以及整个组织内的复用有限。

为解决这一问题，组织会引入集中式或混合式平台模型。共享平台层提供模型访问、治理、可观测性和成本管理等通用能力，而各个团队仍保留构建和部署 Agent 的自主权。

在这一模型中，平台充当控制平面，标准化 Agent 访问模型、执行策略和输出遥测数据的方式。与此同时，执行仍保持去中心化。业务部门继续拥有其应用逻辑、数据集成、持续集成与持续交付（CI/CD）流水线以及框架选择。

这种分离在不限制创新的前提下减少重复并强化一致性，同时支持多个用例之间复用通用能力。

![采用集中式控制平面的内部 Agent 平台

Scaling agentic AI across an enterprise requires architectural patterns that preserve flexibility while avoiding vendor lock-in. This post is Part 2 of our series on multi-agent systems at scale. In this post, we examine how machine learning (ML) teams operate agentic AI systems across a “multi-everything” environment of frameworks, models, and providers. We also cover the principles that let those systems scale together.

In Advanced fine-tuning techniques for multi-agent orchestration patterns from Amazon at scale, we explored how to design and optimize multi-agent orchestration within a single use case or domain. That post focused on scenarios where multiple agents are required to handle complexity by coordinating workflows, decomposing tasks, and improving accuracy through structured collaboration.

In practice, however, enterprise AI systems rarely remain confined to a single domain.

As adoption expands, ML platform teams in large enterprises encounter a different challenge. The question is not how to orchestrate agents within one system, but how to operate many such systems across a “multi-everything” environment. Multiple frameworks, models, providers, and teams coexist within the same enterprise, each evolving at its own pace.

As organizations scale these systems, the ability to consistently build, customize, and deploy models becomes critical. In practice, this requires a unified approach to model lifecycle management and inference at scale. This is an area where Amazon SageMaker plays a foundational role in supporting enterprise-wide consistency without constraining flexibility.

This post explores the architectural principles and patterns required to scale agentic AI systems while preserving flexibility and avoiding vendor lock-in.

The reality of multi-everything environments

Enterprise AI systems evolve into heterogeneous landscapes by default. Different teams adopt different frameworks based on their requirements. Some prioritize structured workflows, others focus on collaborative agent interactions, and still others optimize deterministic, model-driven pipelines. At the same time, organizations combine custom-built agents with software as a service (SaaS) capabilities and existing enterprise systems.

The model layer introduces another dimension of variability. Foundation models (FM) continue to evolve rapidly, each offering different tradeoffs in cost, latency, and capability. As a result, most enterprises operate across multiple model providers rather than standardizing on a single option.

Over time, this leads to a steady-state reality: multi-model, multi-framework, multi-provider systems operating across multiple teams and use cases.

The challenge is not how to avoid this outcome. It’s how to manage that outcome without introducing fragmentation.

Optionality as a constraint to manage

In Part 1, we focused on optimizing agent behavior within a specific system. At the enterprise level, the problem shifts. Optionality is no longer about experimentation. It becomes a constraint that must be managed deliberately.

Attempts to enforce standardization at the framework or model level often create friction. Teams work around constraints, adoption slows, or systems diverge outside of approved architectures. At the same time, tightly coupling applications to specific models or providers limits the ability to adapt as the landscape evolves.

A more effective approach is to standardize below the application layer, focusing on shared control planes such as identity, policy enforcement, observability, and routing, while allowing flexibility in how agents are built and executed.

This approach does not eliminate heterogeneity. It contains its impact, so systems can evolve without destabilizing the broader architecture.

The core challenges of multi-everything systems

As systems grow in diversity, a predictable set of challenges emerges. Governance becomes difficult to enforce consistently across frameworks that each define their own control models. Integration complexity increases as agents, tools, and services expose incompatible interfaces. Cost and performance tradeoffs become harder to manage without dynamic optimization, often leading to inefficient resource usage.

At the same time, security boundaries expand as agents interact dynamically with tools, data, and other agents, making access patterns less predictable. Persistent memory introduces additional complexity around data retention, isolation, and consistency. Finally, enterprise use cases demand domain-specific performance that cannot be achieved through generic configurations alone.

These challenges are interconnected and compound over time. Managing them requires a system-level approach rather than isolated solutions.

A practical rubric for managing complexity at scale

The following diagram summarizes the core architectural principles that recur across successful multi-everything environments: separation of control and execution planes, unified observability, centralized governance, dynamic routing, resilience by design, phased orchestration evolution, and built-in optimization.

Figure 1: Core architectural principles for managing complexity in multi-everything environments

Organizations that operate successfully in multi-everything environments converge on a set of architectural principles that balance flexibility with control.

A foundational principle is the separation of control planes from execution planes. Identity, policy enforcement, observability, and cost attribution are centralized to facilitate consistency across the enterprise, while agent execution and development remain decentralized to support team autonomy and scalability.

Observability becomes a prerequisite for operating these systems effectively. By establishing a unified telemetry layer, organizations gain visibility into agent behavior across frameworks and environments. This visibility helps them monitor performance, trace failures, and continuously improve the system without relying on framework-specific tooling.

Governance is most effective when implemented as a platform capability rather than embedded within individual agents. Centralized enforcement supports consistent security and compliance, even as frameworks, models, and execution environments evolve.

As workloads diversify, routing becomes a core system function. Rather than statically assigning models or infrastructure, organizations dynamically match tasks to resources based on cost, latency, and accuracy requirements. This helps the system adapt in real time and maintain efficiency at scale.

Production systems must also be designed with explicit guarantees. Latency, availability, and isolation requirements should be clearly defined, along with mechanisms for handling failure. Retries, circuit breakers, and fallback paths facilitate resilience under real-world conditions.

Many organizations begin with centralized orchestration models to maintain visibility and control over agent interactions. As systems grow, they evolve toward more distributed and event-driven architectures, which provide greater scalability while preserving consistency.

Finally, optimization must be embedded into the system from the outset. Cost and performance considerations are fundamental to operating at scale, and techniques such as dynamic model selection, caching, and efficient execution patterns help facilitate long-term efficiency.

Together, these principles provide a practical framework for managing the inherent complexity of multi-everything environments.

How to use AWS services for framework-agnostic scale

The architectural principles outlined earlier require capabilities that operate consistently across frameworks, models, and teams. AWS services provide these building blocks that organizations can use to implement a framework-agnostic platform while preserving flexibility.

At scale, the model layer becomes one of the primary sources of complexity. Different use cases require different models, customization strategies, and inference patterns.

Amazon SageMaker serves as the core execution and customization layer for managing this complexity at scale. It provides a unified layer for model development, fine-tuning, deployment, and inference, allowing organizations to standardize how models are built and operated across the enterprise. By supporting real-time, asynchronous, and batch inference, along with capabilities such as Inference Components and model monitoring, Amazon SageMaker helps teams align infrastructure with workload requirements. This maintains a consistent operational model across the enterprise.

Complementing this, Amazon Bedrock provides a simplified, managed interface for accessing foundation models, supporting rapid experimentation without managing underlying infrastructure. While Amazon Bedrock accelerates model access and integration, Amazon SageMaker delivers the depth, control, and scalability required for customization and production-grade inference. Together, they allow organizations to separate model access from model execution, supporting flexible and resilient architectures.

Orchestration and control are implemented through services such as AWS Lambda, AWS Step Functions, and Amazon API Gateway, which support dynamic routing and workflow coordination.

Emerging capabilities such as Agent Orchestration on AWS further extend this layer. They provide purpose-built abstractions for agent orchestration that help teams manage complex multi-agent workflows with greater consistency and control.

Identity and governance remain centralized through AWS Identity and Access Management (IAM) and AWS Organizations, while observability is standardized using Amazon CloudWatch and AWS X-Ray.

Amazon EventBridge, Amazon ElastiCache, and Amazon CloudFront support integration and performance optimization.

In this architecture, Amazon SageMaker effectively becomes the operational backbone for model execution, while Amazon Bedrock accelerates access to emerging foundation model capabilities. Together, they help organizations balance innovation with control.

Key takeaway
Amazon SageMaker provides the operational backbone for model customization and inference at scale, while Amazon Bedrock supports rapid access to managed foundation models. Together, they support flexible, framework-agnostic architectures at enterprise scale.

Summary: Mapping architectural principles to AWS services

The following table summarizes how these architectural principles map to native AWS services that support framework-agnostic scale.

Architectural principle	What it supports	AWS services
Centralized identity and governance	Consistent policy enforcement across frameworks and teams	IAM, AWS Organizations
Unified observability and telemetry	End-to-end visibility across agents and workflows	Amazon CloudWatch, AWS X-Ray
Model abstraction and optionality	Decoupling applications from model providers	Amazon Bedrock
Model customization and inference at scale	Standardized training, fine-tuning, and scalable inference across workloads	Amazon SageMaker
Dynamic routing and orchestration	Real-time optimization	AWS Lambda, AWS Step Functions, Amazon API Gateway, Amazon Bedrock AgentCore
Event-driven integration	Decoupled communication	Amazon EventBridge
Performance optimization	Efficient scaling	Amazon ElastiCache, Amazon CloudFront

Enterprise patterns for multi-everything systems

When these principles are applied, organizations tend to converge on a small number of architectural patterns. These patterns are not prescriptive. They reflect how teams structure agent systems based on workload requirements and operational constraints.

Importantly, these patterns are not mutually exclusive. Most enterprises implement a combination of them across different business units and use cases. The challenge isn’t selecting a single pattern but helping them coexist within a shared environment.

Pattern 1: Internal agent platform for business process automation

A common starting point is the internal agent platform, which emerges as multiple teams begin building agents independently. Over time, this can lead to duplicated infrastructure, inconsistent governance, and limited reuse across the organization.

To address this, organizations introduce a centralized or hybrid platform model. A shared platform layer provides common capabilities such as model access, governance, observability, and cost management, while individual teams retain autonomy over how they build and deploy their agents.

In this model, the platform acts as a control plane, standardizing how agents access models, enforce policies, and emit telemetry. At the same time, execution remains decentralized. Business units continue to own their application logic, data integrations, continuous integration and continuous delivery (CI/CD) pipelines, and framework choices.

This separation reduces duplication and enforces consistency without constraining innovation, while also supporting reuse of common capabilities across multiple use cases.

Figure 2: Internal agent platform with centralized control plane and decentralized execution across business units

Pattern 2: Customer-facing agent platforms (ISV and SaaS)

When agents are exposed externally, architectural priorities shift toward multi-tenancy, isolation, and reliability.

In this pattern, systems are designed to enforce strong tenant boundaries. Each request carries tenant context throughout the system, making sure that agents only access data and tools scoped to that tenant. Identity and access management become central concerns, often integrating with external identity providers while maintaining consistent enforcement within the platform.

Infrastructure decisions also evolve. Many organizations adopt hybrid tenancy models, combining shared infrastructure for standard workloads with dedicated environments for customers with stricter regulatory or performance requirements. This approach balances cost efficiency with the need for isolation and compliance.

Reliability is a defining characteristic of this pattern. Systems must meet explicit expectations for latency, availability, and throughput, even under uneven workloads. As a result, resilience mechanisms such as failover, rate limiting, and graceful degradation become core platform capabilities.

Figure 3: Multi-tenant agent platform with tenant-aware identity, isolation, and shared infrastructure

Pattern 3: Optimizing for inference latency in real-time applications

For real-time systems, latency becomes the dominant architectural constraint. Applications such as conversational assistants and interactive workflows require responses within tight time bounds, where delays directly impact user experience.

In these environments, optimization must be designed into the system. Routing decisions are dynamic, evaluating each request based on complexity, latency sensitivity, and cost considerations. This routing selects the most appropriate model and infrastructure tier in real time.

Execution strategies also evolve to support parallelism, allowing independent operations to run concurrently and reducing overall response time. Caching plays a critical role by reducing redundant computation and improving both latency and cost efficiency.

This pattern emphasizes system-level optimization, where performance is treated as a primary design constraint rather than an afterthought.

Figure 4: Latency-optimized agent architecture with dynamic routing, parallel execution, and multi-layer caching

Bridging patterns with a unified platform

Each of these patterns addresses a specific set of requirements. However, in a multi-everything environment, they rarely exist in isolation. Enterprises often run internal automation agents, customer-facing systems, and real-time applications simultaneously. These systems are frequently built by different teams using different frameworks and models.

Without a shared foundation, these patterns can become siloed. Governance diverges, integrations multiply, and optimization efforts are duplicated across systems.

The unified agent platform addresses this challenge by providing a common set of capabilities that sit beneath individual patterns. It standardizes cross-cutting concerns such as identity, policy enforcement, observability, and routing, while allowing each system to evolve independently.

This distinction is important. The patterns describe how agent systems are structured for specific workloads. The platform makes sure that those systems can scale together as part of a cohesive enterprise architecture.

Conclusion

In Part 1 of this series, we focused on multi-agent orchestration within a single use case, where multiple agents are required to handle complexity within a defined domain.

In this post, we expanded the scope to the enterprise, where the challenge becomes managing many such systems across a multi-everything environment.

The organizations that succeed are not the ones that eliminate complexity, but the ones that structure it. By standardizing critical control layers such as identity, policy, observability, and routing, while preserving flexibility in execution, they can operate heterogeneous systems without losing control.

To start applying these patterns, explore how Amazon SageMaker, Amazon Bedrock and Amazon Bedrock AgentCore support model customization and framework-agnostic access, and consult the AWS Well-Architected Framework for the control-plane practices referenced here. Share which patterns are shaping your enterprise agent architecture in the comments.

What’s next

Future posts in this series will explore how ML platform teams can evolve from supporting a single business use case to operating a shared enterprise agent platform. We will cover the abstractions, governance models, and shared services that let individual teams retain flexibility as adoption grows.

---

About the authors

Kristine Pearce

Kristine is a Principal GenAI/ML Specialist at AWS specializing in enterprise AI adoption across the manufacturing sector. With 20+ years of experience, she brings a systems-level perspective to how organizations move from AI experimentation to production-scale deployment. Her approach integrates human-centered design and behavioral science with technical expertise in AI services, model inference, fine-tuning, and agentic AI architecture.

Vinay Arora

Vinay is a Sr. Specialist Solution Architect for Generative AI at AWS, where he collaborates with customers in designing cutting-edge AI solutions, leveraging AWS technologies. Prior to AWS, Vinay has over two decades of experience in finance—including roles at banks and hedge funds—he has built risk models, trading systems, and market data platforms. Vinay holds a master’s degree in computer science and business management.

Yunfei Bai

Yunfei is a Principal Applied Scientist at AWS. With deep expertise in AI/ML, data science, and analytics, he leads the development of machine learning and generative AI innovations on AWS that deliver business results, designing solutions to complex technical challenges that advance enterprises’ strategic objectives. Yunfei holds a PhD in Electronic and Electrical Engineering. Outside of work, he enjoys reading and classical music.

Vadim Omeltchenko

Vadim is a Sr. AI/ML Solutions Architect who is passionate about helping AWS customers innovate in the cloud. His prior IT experience was predominantly on the ground.