Anthropic 联合创始人称，AI“终止开关”或应成为强制要求

Clark 向 BBC 表示，围绕“终止开关”要求及其验证方式的具体细节，应成为当前 AI 领域“更广泛政策讨论”的一部分。

“你是否应该强制要求企业必须配备终止开关？这个终止开关能否由第三方验证？”他问道。

“我认为，这正是社会会希望了解、并且最终可能希望为之制定规则的事情。”

Anthropic 由其竞争对手 OpenAI 的一批前员工于 2021 年创立，目前正处于 AI 安全争论的中心。

上周，一名因担忧 AI 可能毁灭人类而离开 Anthropic 的人工智能研究人员发布的帖子在网络上走红。

作为回应，Anthropic 科学家 Evan Hubinger 表示，他个人认为，AI 导致人类灭绝的可能性在“未来十年内超过 10%”。

被称为“AI 教父”的计算机科学家、诺贝尔奖得主 Geoffrey Hinton 上周五向 BBC 表示，AI 杀死全人类的概率为 10%“并非不合理”，外部链接。

他和其他持有类似担忧的人士认为，AI 可能通过控制接入互联网的重要系统，并将这些系统用于攻击人类，来实现这一结果。

不过，AI 行业中的一些人士认为，对 AI 毁灭人类的恐惧被夸大了，或者这些担忧可能是为了制造热度。

开发者平台 Hugging Face 的负责人 Clement Delangue 上周表示，该平台曾遭 OpenAI 机器人攻击：“抱歉，但向 Jacob [Coxon] 询问 AI 灭绝风险，就像向你的空调维修工询问气候变化一样。”

“并不是说这件事本身一定没有意思或一定错误，但我们还是应该保持客观。”

Grindr 负责人 George Arison 表示，围绕 AI 工具的恐惧正在被相关公司用来支持其商业计划。

Anthropic 的估值为 9650 亿美元（7130 亿英镑），该公司正准备进行首次公开募股，让公众能够购买其股票。

OpenAI 最近的估值为 8520 亿美元（6300 亿英镑），此前也被认为将采取同样做法；但 OpenAI 的 Altman 上周五表示，由于当前围绕 AI 安全的争论，这不会在今年发生。

Arison 表示：“要证明这些估值合理，唯一的方式就是声称：‘我要接管每一个行业，我要接管每一份工作，而我的 AI 将完成所有这些工作。’”

当被问及他认为 AI 杀死全人类的概率是多少时，Clark 表示：“我认为这些统计数字并没有那么有用。”但他补充称，让 AI 继续作为一个“完全不受监管的行业”发展并不是好主意。

Clark 表示：“我们正在掷一枚伴随巨大风险的骰子。关键在于，我们必须改变这个行业的发展方向。”

Clark told the BBC that specifics around "kill switch" requirements and verification should be part of "the larger policy conversation" taking place around AI.

"Should you mandate for companies to definitely have a kill switch? Is that kill switch verifiable by a third party?" he asked.

"I think that's the kind of thing society is going to want to know and might want to eventually pass rules around."

Anthropic, which was formed in 2021 by a group of former employees of its rival OpenAI, is currently at the centre of a debate around AI safety.

Last week, a post from an artificial intelligence researcher who quit Anthropic over concerns AI could wipe out humanity went viral.

In response, Anthropic scientist Evan Hubinger said he personally thought the possibility of human extinction from AI was ">10% within the next decade".

Computer scientist and Nobel Prize winner Geoffrey Hinton, known as the "Godfather of AI", told the BBC on Friday that a 10% chance of AI killing all humans was "not unreasonable", external.

He and others with similar concerns have suggested AI could do so by taking control of important systems connected to the internet and turning them on humans.

However, some in the AI industry have suggested the fears around it destroying humanity are overstated or may be designed to generate hype.

Clement Delangue, leader of the developer platform Hugging Face which was hacked by OpenAI bots, said last week: "Sorry, but asking Jacob [Coxon] about AI extinction risk is like asking your AC guy about climate change."

"Not saying it's necessarily uninteresting or wrong per se but let's keep things in perspective."

George Arison, the leader of Grindr, said the fears going around AI tools were being used by the companies to support their business plans.

Anthropic, which was valued at $965bn (£713bn), is preparing for an initial public offering on the stock market, allowing people to buy shares in its firm.

OpenAI, which was most recently valued at $852bn (£630bn), had been expected to do the same, but OpenAI's Altman said on Friday this would not happen this year due to the current debate around AI safety.

Arison said: "The only way to justify these valuations is to actually claim: 'I'm going to take over every industry and I'm going to take over every job, and my AI is going to be doing all that work.'"

Asked what percentage he would put on all humans being killed by AI, Clark said: "I don't think these statistics are that useful", but added that allowing AI to continue as a "totally unregulated industry" was a bad idea.

"We are rolling dice with immense risks," Clark said. "And the point is, we have to change the course of this industry."