为自主工业 AI 构建更安全的发展路径

Arti： 是的，我认为，对我们而言，负责任 AI 包含一项“三重使命”：它必须安全、高效——其中也包括环境效率——并且最重要的是，真正保障人类安全和人类监督。对我们来说，这些正是负责任 AI 的一些核心支柱。这意味着，在内部构建工具时，我们会采用多层治理方法，既覆盖我们作为一家公司如何使用 AI，也覆盖我们如何将 AI 部署到产品中。我们在这两方面采用类似的框架，即一种联合治理模式：既管理我们如何利用 AI，也管理我们如何将 AI 融入产品，使客户能够在其运营中利用 AI。

但我想说，所有这些原则之下的一个核心基础是：在人们思考如何利用 AI 时，人类仍然处于中心位置。因此，人类判断、人类责任和人类伦理，仍然是我们判断 AI 能在哪些领域带来益处的核心。我们更倾向于将 AI 视为对人类的增强，而不是在关键决策闭环中取代人类。

Megan： 对，这确实是一个非常重要的区别，不是吗？就下一阶段的发展而言，越来越多人认为，Agent AI 很快将使这些工业系统能够以更高程度的自主性运行。我想知道，工业场景中的风险与纯数字环境中的 AI 风险有何不同？反过来看，同样由软件定义的自动化又能带来哪些益处和机会？

Arti： 我认为，我们可以先谈风险，再谈收益。归根结底，我认为工业环境中最大的风险在于，我们正在与真实的物理系统交互。这些物理系统往往能够带来维持世界运行的重要结果，无论是输送电力，还是开采自然资源。但通常，这些物理系统也运行在危险环境中，设备本身也可能影响人类安全。任何时候，只要软件与可能产生现实后果的物理系统相连接，就必须格外谨慎。AVEVA 从一开始就将这种意识融入自身基因：始终关注最终用户和最终应用场景，因为这并不只是电脑屏幕上的某个东西。

即使在过去数十年中将 AI 部署到我们的系统时，我们通常也会倾向于真正选择最匹配的模型，并深入理解该模型的行为。例如，我们拥有一种专有的异常检测模型，可在许多运营环境中发挥作用。如今，当我们思考这些较新的 AI 能力时，它们在设计上就很难让人理解其行为方式；它们并不像我们过去所理解的 AI 或统计模型那样具有根本上的可解释性。实际上，随着它们学习并针对新能力进行调优，其行为还可能随时间发生变化。基于会随时间演化的能力来自动化物理系统，这正是潜在风险所在，也是我们必须高度警惕并认真思考的问题：我们愿意在哪些地方将这种更高程度的自动化付诸实践？

但与此同时，我认为这也蕴含着真正的机会。刚才我略过了一个观点：模型会演化，但人类同样会演化，而这有时是一件好事。我们常说，人类拥有专业知识，并会在职业生涯中逐步加深对系统如何运作的理解。如果我们能够真正利用这些新型 AI 能力——这类推理模型通常具备底层 Agent 能力——以更快速度学习和积累经验，并且可能更重要、更有价值的是，将一个站点学到的经验应用到另一个站点，那么这就是一个真正的机会。它能够利用已经在人的身上被证明有效的方式：有时你就是必须在实践中学习，并通过 AI 将这种能力应用和规模化。这正是我认为该领域可能存在重大机会的地方。

工业领域中有一件我们高度关注的事情，就是劳动力结构正在发生变化。我认为，在未来五年内，接近一半——但还不到一半——的工业劳动力将退休。这意味着该行业将失去大量专业知识和经验。如果有办法以可执行的形式捕捉这些知识，同时帮助习惯于以不同方式获取经验的新一代劳动者运用这些专业能力和知识，我认为这将是工业领域中的巨大机遇。

Megan： 是的，完全同意。正如你所概述的，特别是在其他市场和劳动力变化的背景下，这显然蕴含着巨大机会。在此基础上，当这些自主工业 AI 系统以负责任的方式构建和部署时，它们有何潜力推动更快速、更可持续的工业流程？

Arti: Yeah, so I think for us, we think about a triple mandate around responsible AI, and that means that it's secure, it's efficient, and that includes environmentally efficient, and then it really preserves human safety and human oversight above all. And for us, that's what some of our core pillars of responsible AI include. And for us, that means we think about internally as we build tools, a multi-layered governance approach where we've got both for how we use AI as a company and then how we deploy AI into our products. We've got a similar framework that we leverage across both, a kind of joint governance model, both around how we're leveraging AI and also how we're infusing AI into our products to allow our customers to leverage AI in their operations.

But I would say one of the core foundational principles underneath all of these is that human beings still remain centered in how we think about how AI is leveraged. And so human judgment, human responsibility, human ethics are still core to how we think about where AI can add benefits, and we think about it more as augmenting rather than replacing human beings in critical decision loops, if you will.

Megan: Right, which is such an important distinction, isn't it? And in terms of the next phase of development, there's growing sentiment that agentic AI will soon allow these industrial systems to operate much more autonomously. I wonder, what makes the risks in industrial settings different from AI risk in a purely digital context? And I suppose on the flip side, what are the benefits and opportunities of that same software defined automation?

Arti: I think we can start with the risks and then go to the benefits. I think ultimately, I would say the biggest risk in the industrial setting is that we are interacting with real physical systems, often physical systems that are very capable of delivering important outcomes that keep our world running, whether that's delivering power, whether that's mining for natural resources. But usually. Those physical systems are also operating in hazardous environments, the equipment itself is capable of also having human safety implications. Anytime you're interfacing software with a physical system that can have real world consequences, you have to be extra careful. And we've always had that, again, we've had that in our DNA at AVEVA from the beginning, being mindful of that end user and that end application, which is not just something on a computer screen.

Even as we've deployed AI over the past few decades into our systems, we've typically leaned toward really trying to pick the best fit model, really understand that model's behavior if we're going to use, for example, we have a proprietary anomaly detection model that works in a lot of operational environments. Now, as we're thinking about some of these newer AI capabilities, which by design are hard to understand how they behave, they're not fundamentally explainable in the way we've thought about even AI or statistical models in the past, and actually their behavior can change over time as they learn and get tuned to new capabilities. That's really the potential risk of automating a physical system based on capabilities that can evolve over time is one of the things that we have to be really mindful of and really thoughtful around, where are we willing to put some of that increased automation into practice?

But at the same time, I think there's a real opportunity there, because I glossed over this idea that, okay, well, models evolve, but so do human beings, and sometimes that's a good thing. We talk about humans having expertise and they gain understanding over their careers of how a system works. If we can actually leverage the ability of some of these newer AI capabilities, these sort of reasoning models often have underlying agentic capabilities, to learn and gather experience faster, and potentially even more importantly or more valuably, take experience that's learned at one site and apply it to another site, then that's a real opportunity to take what we already know works for human beings, which is that sometimes you just have to learn by doing and be able to apply that and scale that through AI. That's where I see potentially a real opportunity in this space.

One of the things that we're really aware of in the industrial sector is just the way that our workforce is changing. I think that almost half, not quite half, of the industrial workforce is set to retire in the next five years, and that's a lot of expertise and experience that means that we're going to lose in the sector. If there's ways to make sure that we can capture that in ways that are actionable, in ways that can also help a newer generation of workers that are used to experience things in a different way, apply that expertise, apply that knowledge, then I think that's a huge opportunity within the industrial space.

Megan: Yeah, absolutely. Clearly, some huge opportunities there particularly against the backdrop of other market and workforce changes as you've outlined there. I suppose building on that, what is the potential for these autonomous industrial AI systems when built and deployed responsibly to facilitate even faster, more sustainable industrial processes?