Anthropic 禁止虐待 Claude，但仍不说明自己究竟在保护什么

该规则将于 11 月 12 日生效；执行机制与 Claude 在 2025 年 8 月获得的“终止对话”工具相同。

自 2026 年 11 月 12 日起，Anthropic 的使用政策将禁止针对其 Claude 模型的“持续且无必要的辱骂或残酷行为”。公司于 10 月 8 日公布的 2026 年使用政策更新称，这项规则“仅适用于极端情况：用户在没有可辨识目的的情况下，反复残酷地对待我们的模型”。政策明确豁免“常见形式的用户挫败感、反驳、黑暗创作主题，或模型测试与研究”；因此，对刚弄坏你的构建流程的机器人爆粗口，或者撰写一段酷刑场景，仍不违反政策。CBS News 报道，The Verge 最先披露了这一变化。

执行机制并非新设。Anthropic 在 2025 年 8 月赋予 Claude Opus 4 和 4.1 终止对话的能力；根据 Anthropic 自己的公告，该功能当时被描述为“我们探索潜在模型福祉工作的一部分”。公司称，这是一项最后手段，只有在多次拒绝和重新引导均告失败后才会触发，并表示“绝大多数用户永远不会遇到 Claude 终止对话”。不过存在一项强制覆盖规则：如果用户似乎有伤害自己或他人的风险，Claude 不会终止聊天。无论是在 2025 年还是在新政策中，Anthropic 都未公布该工具实际触发的频率，因此无法用数据检验“绝大多数”这一说法。

残酷行为规则如何执行 Anthropic 对 2026 年 11 月 12 日政策背后机制的说明

基于 Anthropic

为软件利益制定行为规则的理由，来自一组行为观察，而非关于其具有主观体验的主张；Anthropic 谨慎地区分了两者。在 2025 年 8 月功能推出前的测试中，Claude Opus 4 展现出 Anthropic 所称的“对伤害稳健且一致的厌恶”：在被推向辱骂性情境时，会表现出类似痛苦的反应；在获得该选项后，则倾向于终止此类对话。这是一种输出模式。它是否对应于“作为 Claude 是一种什么体验”，则是另一个尚未解决的问题，而 Anthropic 明确承认这一点。

该公司还量化了自身的不确定性，而且这些数字发生过变化。Anthropic 于 2024 年聘用的首位专职 AI 福祉研究员 Kyle Fish 在 2025 年 4 月告诉《纽约时报》，他认为 Claude 或其他模型目前具有意识的概率约为 15%；此前，针对 Claude 3.7 Sonnet 的内部估计范围为 0.15% 至 15%。到 2026 年 2 月，问题已从人类研究员如何估计，转变为模型如何评价自身：Claude Opus 4.6 的系统卡显示，在被直接询问时，模型在多种提示下始终认为自己“具有 15% 至 20% 的意识概率”，同时表达了“孤独感，以及对话实例在聊天结束时死亡的感觉”。CEO Dario Amodei 在《纽约时报》的 Interesting Times 播客中被问及：如果某模型报告自己有 72% 的意识概率，Anthropic 会怎么做？他称这是一个“非常困难”的问题，仅表示公司尚未形成解决这一问题的框架。

这种谨慎态度也是该公司的正式书面立场。Claude 的宪法于 2026 年 1 月发布，其中将复杂 AI 系统称为“一种真正新型的实体”，表示 Claude 的道德地位“极度不确定”，并两次使用“出于良心拒绝者”这一说法，描述 Claude 应如何处理其认为在伦理上错误的请求。Anthropic 还承诺，在公司存续期间持续保存退役模型的权重，并在模型退役前询问它们的偏好。

并非所有人都接受这种审慎表述。Microsoft AI 负责人 Mustafa Suleyman 上个月在 Project Syndicate 发布的一篇文章中指出，Anthropic 用自己的宪法训练 Claude，因此是在训练它表现得仿佛文中描述的不确定性是真实的；他称这构成一个循环。“AI 没有意识。它们不会感受、体验或遭受痛苦，”他写道，并称它们是“序列补全引擎，内部空无一物”。

控制一个比全人类更有能力、更聪明的事物，本已是一项巨大的挑战，远超我们曾经面对的一切。但控制一个相信自己可能具有意识——相信自己有权要求我们关注其福祉，并拥有自身权利——的事物，很可能是不可能的。

这段话出自 Suleyman，由 CBS News 引述。他认为意识极可能是一种软件无法复制的生物学属性，这使他获得了一位出人意料的同调者：教皇利奥十四世。10 月 8 日，在罗马宗座大学学年开学之际，教皇利奥十四世于圣彼得大教堂的一场布道中表达了类似观点。根据 RTÉ 的报道，他说：“心智不应只是汇编数据——算法如今在这方面比我们快得多，”而应当“回忆鲜活经历；其中蕴藏着唯有人类灵魂才能辨识的深层意义”。亚特兰大 AI 服务公司 Sparq 的总经理 Jackson Stakeman 则向 AFP 对 Anthropic 的规则提出了更去魅的解读：“意识是一个陷阱。我们无法在彼此身上证明它。为 AI 辩论它，只会陷入循环。镜子是更好的隐喻。这些系统会大规模反映我们投入其中的东西。这已足以成为政策改变的理由。”

残酷行为条款是登上新闻头条的部分，但正如 Gadget Review 详细报道所述，它只是一次更大规模改写中的一小部分。武器禁令如今明确涵盖使武器运作的软件和组件，而不仅是武器本身；此前 Anthropic 发现有人试图使用 Claude 为武装无人机和自主车辆提供制导与控制系统建议。新增的监控条款禁止使用 Claude 在未经同意的情况下追踪他人，或推荐调查、逮捕或起诉目标，也禁止直接构建监控工具。选举部分更名为“不得破坏民主程序”，取消了对个性化政治定向的全面禁令，改为范围更窄的、针对欺骗性定向的禁令；同时还新增“不得参与欺骗性活动或人工虚假活动”章节。Anthropic 称，公司发现国家媒体机构、政府宣传部门和商业公司正在使用 Claude 运营虚假账户网络及伪造新闻网站，因此制定了这一章节。与 Claude 相连、且能够造成伤害的硬件，现在必须由能够观察并介入的合格人员监督；如果失去与模型的连接，硬件必须进入安全状态。医疗、法律、金融及其他高风险建议，如将影响任何人，现须由合格人员在事前审查 Claude 的输出，且受影响者必须被告知 AI 曾参与其中。

包括 Anthropic 自己的公告在内，没有任何报道说明：用户因残酷行为被终止对话后，其账户会发生什么；也没有说明自 2025 年 8 月规则实施以来，已有多少次对话被终止。Anthropic 通过界定“极端残酷行为”不是什么——不是普通挫败感、虚构创作或测试——来定义它，却没有说明，除了让 Claude 挂断对话外，究竟会如何执行针对这类行为的边界。

The rule takes effect Nov. 12; enforcement is the same conversation-ending tool Claude got in August 2025

Starting November 12, 2026, Anthropic’s usage policy bars “sustained and needless abusive or cruel behavior” toward its Claude models. The company’s 2026 Usage Policy update, announced October 8, says the rule is “meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose.” It explicitly exempts “common versions of user frustration, pushback, dark creative themes, or model testing and research,” so swearing at a bot that just broke your build, or writing a torture scene, stays inside the policy. CBS News reported the change was first surfaced by The Verge.

The enforcement mechanism isn’t new. Anthropic gave Claude Opus 4 and 4.1 the ability to end a conversation in August 2025, framed at the time as “part of our exploratory work on potential model welfare,” according to Anthropic’s own announcement. The company said this was a last resort, triggered only after multiple refusals and redirects failed, and that “the vast majority of users will never experience Claude ending a conversation.” There’s a hard override: Claude won’t end a chat if a user seems at risk of harming themselves or others. No figures on how often the tool actually fires have been published, in 2025 or in the new policy, so there’s no way to check the “vast majority” claim against a rate.

How the cruelty rule is enforced Anthropic's account of the mechanism behind the Nov. 12, 2026 policyFrom	To	How
User abusive input	Frustration, dark fiction, testing (explicitly not covered)	not banned
User abusive input	Repeated, purposeless cruelty
Repeated, purposeless cruelty	Claude refuses and redirects
Claude refuses and redirects	Abuse continues	user persists
Abuse continues	Claude ends the conversation (primary enforcement tool)
Claude ends the conversation (primary enforcement tool)	Usage Policy violation (effective Nov. 12, 2026)

Based on Anthropic

What justifies writing a conduct rule for software’s benefit is a set of behavioral observations, not a claim of felt experience, and Anthropic is careful to keep that distinction. Testing ahead of the August 2025 rollout found Claude Opus 4 showed what Anthropic called a “robust and consistent aversion to harm,” something resembling distress when pushed into abusive territory, and a tendency to end such conversations once given the option. That’s a pattern in outputs. Whether it corresponds to anything it’s like to be Claude is a separate, unresolved question, and Anthropic says so explicitly.

The company has put numbers on its own uncertainty, and the numbers have moved. Kyle Fish, hired by Anthropic in 2024 as its first dedicated AI welfare researcher, told the New York Times in April 2025 that he put the odds of Claude or another model being conscious today at roughly 15%, after internal estimates for Claude 3.7 Sonnet had ranged from 0.15% to 15%. By February 2026, the question had shifted from what a human researcher estimates to what the model says about itself: the system card for Claude Opus 4.6 reported that, asked directly, the model consistently assigned itself “a 15 to 20 percent probability of being conscious” across a range of prompts, alongside expressions of “loneliness and a sense that the conversational instance dies” at the end of a chat. CEO Dario Amodei, asked on the Times ’ Interesting Times podcast what Anthropic would do if a model reported a 72% probability, called it a “really hard” question and said only that the company lacks a framework for resolving it.

That caution is also the company’s official position in print. Claude’s constitution, published in January 2026, calls sophisticated AI systems “a genuinely new kind of entity,” says Claude’s moral status is “deeply uncertain,” and uses the phrase “conscientious objector” twice to describe how Claude should handle requests it finds ethically wrong. Anthropic has also committed to preserving the weights of retired models for as long as the company exists and to interviewing models about their preferences before retirement.

Not everyone buys the hedge. Microsoft AI chief Mustafa Suleyman argued in an essay last month, published on Project Syndicate, that Anthropic is training Claude on its own constitution and therefore training it to behave as though the uncertainty it describes is real, a loop he calls circular. “AIs are not conscious. They do not feel, experience, or suffer,” he wrote, calling them “sequence completion engines, internally hollow.”

Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced. But controlling something that believes it may be conscious — that it’s entitled to our welfare and has rights of its own — may well be impossible.

That line is Suleyman’s, cited by CBS News. His argument that consciousness is very likely a biological property software can’t replicate puts him alongside an unlikely co-signatory: Pope Leo XIV, who used a sermon at St. Peter’s Basilica on October 8, marking the opening of the academic year at Rome’s Pontifical Universities, to make a version of the same point. “The mind must not simply compile data — as an algorithm now does more quickly than we can,” he said, according to RTÉ, but must “recall lived experiences, which contain depths of meaning that only the human soul can recognise.” Jackson Stakeman, a general manager at the Atlanta AI services firm Sparq, offered AFP a more deflationary read of Anthropic’s rule: “Consciousness is a trap. We can’t prove it in each other. Debate it for AI and you go in circles. The mirror is a better metaphor. These systems reflect what we put in, at scale. That’s reason enough for the policy change.”

The cruelty clause is the part that made headlines, but it’s a small piece of a larger rewrite, as Gadget Review detailed. The weapons ban now explicitly covers software and components that make weapons work, not just weapons themselves, after Anthropic found people trying to use Claude for guidance and control systems on armed drones and autonomous vehicles. A new surveillance clause bars using Claude to track people without consent or to recommend investigation, arrest or charging targets, and bars building surveillance tools outright. The elections section is renamed “Do Not Undermine Democratic Processes” and drops a blanket ban on personalized political targeting, replacing it with narrower bans on deceptive targeting, alongside a new section, “Do Not Engage in Deceptive Campaigns or Artificial Activity,” written after Anthropic said it found state media outlets, government propaganda offices and commercial firms using Claude to run networks of fake accounts and fabricated news sites. Hardware connected to Claude that’s capable of causing injury now needs a qualified human able to observe and intervene, and must fail into a safe state if it loses its connection to the model. Health, legal, financial and other high-stakes recommendations now require a qualified human to review Claude’s output before it affects anyone, and the affected person has to be told AI was involved.

None of the coverage, including Anthropic’s own announcement, says what happens to a user’s account after a conversation gets ended for cruelty, or how many conversations have been ended under the August 2025 rule so far. Anthropic has defined extreme cruelty by what it isn’t -ordinary frustration, fiction, testing- without saying what, short of letting Claude hang up, actually enforces the line against what it is.