Uber 首席信息安全官谈如何为 AI 构建内部威胁防范计划

Uber 运营着全球最大的实时市场平台之一，每天处理超过 4000 万笔出行和配送订单。Uber 首席信息安全官（CISO）Philip Martin 领导的团队负责保护 Uber 的数据，以及在现实世界中使用该平台的乘客、司机和配送员。早在 Claude Code 出现之前，他的团队就开始使用 Claude 进行威胁建模，当时还需要手动收集上下文。如今，借助 Claude Fable，Claude 可以自行从代码、设计文档和工单历史中获取这些上下文。

Martin 认为，AI Agent 需要一套与企业现有员工内部威胁防范计划类似的机制。我们与他探讨了这一想法、他为何如此关注无意误用，以及 Uber 的开源监控工具如何与 Enterprise Frontier Safeguards 协同工作。

前沿模型正在如何改变您的安全团队的工作？

以往的模型与当今前沿模型之间真正巨大的区别，在于后者能够在较长的时间跨度内整合更多上下文，进而生成针对性很强的输出。当我们对某个服务应用进行安全审查或威胁建模时，可以让模型处理更多有关该服务的上下文，无论是代码本身、设计文档、Jira 中的问题，还是关于服务预期功能及其交互对象的历史背景。

我看到 Agent 团队也正在呈现同样的趋势。我可以想象，在不太遥远的未来，每项服务都配有一个非常了解该服务的 Agent，而这些 Agent 可以就威胁模型相互沟通，就像服务在生产环境中彼此交互一样：这是我的威胁模型，这是我的数据来源。Agent 团队的协作效应将使我们的工作效能迈上新台阶。

Uber 与我们的团队合作使用 Enterprise Frontier Safeguards。为什么这对 Uber 很重要？

Anthropic 和 Uber 有一个共同点：尽管所处的环境截然不同，两家公司都从根本上重视安全。Uber 考虑安全时，会想到乘客安全和个人数据保护。Anthropic 希望防止其前沿模型被误用，而 Uber 则需要保护客户数据、商业秘密和其他专有信息。Enterprise Frontier Safeguards 让我们能够取得平衡：既有安全防护措施，同时未经我们许可，Anthropic 无法访问我们的数据。

它让我们能够掌控自己的数据，也能掌控风险缓解措施，以及安全防护机制触发后我们决定采取的行动。我认为，我们共同承担着确保这些模型在现实世界中得到安全、有效使用的责任。

当你们的团队向我们介绍设计方案时，很明显你们听取了我们的反馈。你们把主导权交给了我们。这让我们既能保护自己的数据，也能负责任地管理这些模型带来的能力。

Enterprise Frontier Safeguards 如何与 Uber 已经针对 AI 部署开展的安全监控相配合？

从一开始，我们就在如何于 Uber 内部安全部署 AI 方面投入了大量工作。我们最近开源了一款名为 AI Detection and Response（ADR，检测与响应）（在新标签页中打开）的产品，旨在从我们部署的 Agent 运行框架中收集实时遥测数据并实时采取行动。

它与 Anthropic 的安全防护措施相辅相成，因为我们现在能够获取更丰富的数据，用于检测 Agent 何时越界并实时响应。我们既能通过自己的视角了解正在发生的情况，也能获得 Anthropic 分类器的输出，并将两者整合到整体检测环境中，这与传统网络安全环境非常相似。关键在于有效整合上下文，以尽可能快、尽可能有把握地作出决策。

我认为，Enterprise Frontier Safeguards 将成为 ADR 的一个数据源。它们与我们的 LLM 网关共同构成了整体 AI 安全架构；该网关负责建立安全边界，并控制 Uber 内部如何使用 AI。

您认为，接下来在保障 AI 系统安全方面，防御人员应该优先关注哪些领域？

第一是实时检测和干预：尽可能在用户于运行框架中输入提示词的当下介入，告诉他们“你可能不想这么做”，或者“这个模型即将调用的工具、即将连接的位置，可能不是你希望它使用或连接的”。

第二是更好地理解新出现的模式。我们会观察模型行为趋势，发现此前不知道可能出现的新行为。这会形成一个循环：我们查看数据，很多时候借助 LLM，在这些行为出现时发现它们，并将针对性的防护措施重新纳入实时平台。随后又会出现新行为，因为限制本身会催生新行为，而我们也会发现它们。

对于那些对前沿模型所需监控心存疑虑的其他领导者，您会说些什么？

采用 AI 是企业发展的必然要求。不管我们愿不愿意，它都会到来，并将为世界各地的企业和人们带来实实在在的好处。但它也确实可能造成伤害；随着模型能力增强，承担更加复杂、自主的工作，风险也会随之升高。

每家公司都已经针对员工实施内部威胁防范计划。对此没有人提出异议。我们接受有时相当繁重的数据留存要求，也会监控一些原本不愿监控的事项，因为我们知道，人们偶尔会出于各种原因超出其职责范围行事。

前沿模型也带来了同样的风险。这些模型可能被指示采取超出我们预期约束范围的行动。因此，我们需要为 AI 建立同等的防范能力。针对模型的内部威胁防范计划，将是安全可靠地推广这些系统的关键。

了解有关 Enterprise Frontier Safeguards（在新标签页中打开）的更多信息。

Uber runs one of the world's largest real-time marketplaces, with over 40 million trips and delivery orders a day. Philip Martin, Uber's Chief Information Security Officer (CISO), leads the team that protects Uber's data and the riders, drivers, and couriers who use the platform in the real world. His team started working with Claude on threat modeling before Claude Code existed, assembling context by hand. Now, with Claude Fable, Claude gathers that context itself from code, design docs, and ticket history.

Martin thinks AI agents need the same kind of insider threat program companies already run for their employees. We spoke with him about that idea, why he thinks so much about accidental misuse, and how Uber's open-source monitoring tools work alongside Enterprise Frontier Safeguards.

How are frontier models changing your security team’s work?

The really big difference between prior models and today’s frontier models is their ability to incorporate additional context over long time horizons, and as a result, generate really specific outputs. When we're doing security reviews or threat modeling for a given service application, we can let the models consume a lot more context about the service, whether that's the code itself, design documents, issues in Jira, or historical context on what the service is supposed to do and what it interacts with.

I see the same thing emerging with teams of agents. I can imagine a world in the not-too-distant future where each service has an agent associated with it that understands that service very well, and those agents can talk to each other, the same way the services might in production, about the threat model: this is my threat model, this is where my data comes from. That team-of-agents effect will take us to the next level of efficacy.

Uber worked with our team Enterprise Frontier Safeguards. Why was this important for Uber?

One thing Anthropic and Uber have in common is a bone-deep focus on safety, albeit in very different contexts. When Uber thinks about safety, we think about rider safety, protecting personal data. Anthropic wants to prevent the misuse of its frontier models, but Uber needs to protect our customer data, our trade secrets, and any other proprietary information. Enterprise Frontier Safeguards allow us to strike this balance: the safeguards are in place, but our data is not accessible to Anthropic without our permission.

It gives us control of our data and control of the risk mitigation and the actions we decide to take after a safeguard is triggered. I think we share a responsibility to make sure these models are used safely and effectively in the world.

When your team walked us through the design, it was clear that you heard our feedback. You put us in the driver's seat. This allows us to protect our data while still being responsible stewards of the capabilities of these models.

How Enterprise Frontier Safeguards fit with the safety monitoring you already run on AI deployments Uber?

We've done a ton of work since the very beginning on how we deploy AI safely within Uber. We recently open-sourced a product called AI Detection and Response (ADR) (opens in new tab), designed to gather real-time telemetry and take real-time action in the agent harnesses we deploy.

It complements Anthropic's safeguards, because we now have access to enhanced data to detect when an agent is acting outside its bounds and respond in real time. We have our own view of what's going on, plus output from Anthropic's classifier, and we can integrate both into our overall detection environment, much like a traditional cybersecurity one. It's all about assembling context effectively to reach a decision as fast, and with as much certainty, as possible.

I think we'll see Enterprise Frontier Safeguards feed into ADR as a data source. Together with our LLM gateway, which builds a perimeter and controls how AI is used within Uber, those make up our overall AI security architecture.

Where do you think defenders should prioritize their focus next when it comes to securing AI systems?

Number one is real-time detection and intervention: getting as close as we can to the human entering a prompt in a harness, to be able to tell them, "You probably don't want to do that," or "The tool this model is about to call, the place it's about to connect to, is probably not something you want to happen."

Number two is better understanding emerging patterns. We look at trends in model behavior and detect new behaviors we didn't know were possible. That becomes a cycle. We look at the data, in many cases with the assistance of LLMs, discover these behaviors as they appear, and build safeguards against them back into the real-time platform. Then new behaviors emerge, because constraints create new behaviors, and we find those too.

What would you tell other leaders who are hesitant about the monitoring frontier models require?

AI adoption is a business imperative. It's coming whether we want it or not, and it's going to bring real benefits to businesses and people around the world. But there's also real potential for harm, and the stakes rise as models get more capable and take on more complex, autonomous work.

Every company already runs an insider threat program for its people. Nobody argues about that. We accept the sometimes onerous data retention, and we monitor things we'd rather not monitor, because we know humans occasionally act outside the bounds of their role, for any number of reasons.

Frontier models introduce the same risk. These models may be directed to act outside the constraints we expect them to operate within. So we need the equivalent capability for AI. That insider threat program for models is going to be essential to rolling these systems out safely and securely.

Learn more aboutEnterprise Frontier Safeguards (opens in new tab).