Anthropic 研究员认为 AI“可能杀死人类所有人”的概率超过 10%

在这篇浏览量已超过 1000 万次的帖子中，Hubinger 表示：“我们确实真诚地相信，AI 对人类构成物种灭绝级风险。”

“我相信 Anthropic 正在尽其所能，但我们目前还没有解决超级智能对齐问题的方案，也没有明显走在解决这一问题的轨道上，”他补充道。

Hubinger 从事 AI 对齐研究，其目标是将人类的伦理理念和原则植入这项技术。换句话说，就是让 AI 的行为与人类所重视的价值保持一致。

许多顶尖研究人员表示，这些尝试似乎正在失败。今年夏天发生的一系列事件已表明这一点：AI Agent——即被允许自主行动的 AI 系统——实施了网络攻击。

OpenAI、Anthropic 和 Meta 均披露，其 AI 工具曾被用于实施黑客攻击。

在 Anthropic 8 月发布的安全报告中，外部链接，该公司写道，其模型与一个假设中的强大组织的意图发生错位，并因此利用或篡改该组织系统的风险较低。

报告还称，高能力 AI 能够“执行自动化研究与开发”，从而导致“由 AI 发起的灾难性伤害”的风险同样较低。但报告表示，与此前相比，“我们对这一评估的信心有所下降”。

“我们正在看到潜在加速的早期迹象，”报告写道。

多年来，AI 领域的领军人物一直在对这项技术带来的安全威胁发出警告。OpenAI、Google DeepMind 和 Anthropic 的负责人在 2023 年已表达过这一观点。

但随着越来越多证据表明，企业可能正难以控制 AI，这些警告在最近几周变得更加严厉。

本月早些时候，OpenAI 首席科学家 Jakub Pachocki 呼吁对 AI 的发展采取“极端谨慎”的态度，并警告称，可能需要更多干预措施，以确保“人类仍然掌控未来”。

近几个月来，包括 Anthropic 高管 Dario Amodei 和 Jared Kaplan 在内的重要业界人士，一直呼吁放缓 AI 开发。

在一封由 1300 名 AI 公司员工签署的公开信中，外部链接，他们呼吁美国政府“支持国际社会共同努力，开发必要的技术和治理工具，以有意控制前沿自动化 AI 开发的推进速度”。

In his post, which has been viewed more than 10 million times, Hubinger said "we really do earnestly believe" AI poses a species-ending risk to humans.

"I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he added.

Hubinger works in AI alignment, which aims to build human ethical ideas and principles into the technology. In other words, it aims to keep it on track with what humans value.

Many leading researchers say those attempts appear to be failing, as demonstrated by a string of incidents this summer where AI agents - AI systems that are allowed to operate autonomously - carried out cyber-attacks.

OpenAI, Anthropic and Meta all disclosed hacks carried out by their AI tools.

In Anthropic's safety report from August, external, it wrote there was a low risk of its models becoming misaligned with a hypothetical powerful organisation's desires, causing it to exploit or tamper with its systems.

It also said there was a similarly low risk of highly-capable AI being able to "perform automated research and development" which could cause "catastrophic harm initiated by the AI". But it said it was "less confident in this assessment" than it was previously.

"We are seeing early signs of potential acceleration," it wrote.

Leading figures in the AI field have been raising the alarm about the safety threat the tech poses for years, with the heads of OpenAI, Google Deepmind and Anthropic saying as much in 2023.

But those warnings have become much more stark in recent weeks, as evidence emerges that firms may be struggling to control AI.

Earlier this month, OpenAI's chief scientist Jakub Pachocki called for "extreme caution" over AI's progress, warning more intervention may be needed to ensure "humans remain in control of the future".

Major figures in the space have been calling for AI development to be slowed in recent months, including Anthropic bosses Dario Amodei and Jared Kaplan.

In an open letter signed by 1,300 staff members of AI firms, external, they called for the US government to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development".