Anthropic 禁止对 Claude 实施“持续且无必要的辱虐或残酷行为”

Anthropic 一年多来首次修改其使用政策，以应对新出现的高风险滥用情形——包括选举干预、武器研发、监控，以及医疗和金融用途。但其中最重要的变化之一，是禁止对 Claude 实施“持续且无必要的辱虐或残酷行为”。

去年 8 月，该公司宣布，作为其“模型福利”（model welfare）研究的一部分，将允许 Claude 主动结束与“持续造成伤害或辱虐”的用户之间的对话。此次更新称，终止对话仍是“主要执法机制”；Anthropic 未就是否还会采取进一步执法措施发表评论，例如可能封禁用户。Anthropic 写道，这项新政策更新“仅适用于极端情形，即用户在没有任何可辨识目的的情况下，反复残酷地对待我们的模型。它不适用于常见的用户挫败感、质疑或反驳、黑暗创作主题，或模型测试与研究。”

除模型福利相关变化外，Anthropic 还将数项原本“分散”的既有限制整合为一项新禁令，禁止欺骗性的商业或政治活动，以应对日益严重的 AI 生成宣传问题。该政策限制“试图掩盖信息背后主体身份，或通过虚假账号或帖子放大内容”的行为。Anthropic 写道，其选举相关章节还禁止欺骗选民和扰乱选举，包括传播“有关候选人或投票方式的虚假信息、冒充候选人或选举官员，或试图压低投票率”。

终止对话仍是“主要执法机制”。

Anthropic 表示，尽管其使用政策一直公开禁止使用 Claude 进行武器研发，但公司已发现多次有人试图利用 Claude 开发武器指导方案和控制软件——因此，新使用政策将禁令扩大至“让武器得以运作的软件和组件，以及为无人机及其他自主载具武装等行为”。此外，该公司上一份威胁情报报告显示，人们越来越多地使用由 Claude 驱动的监控系统来识别和追踪“政治异见人士”，因此该公司进一步明确了其监控禁令。

Anthropic 写道：“未经他人同意追踪个人是被禁止的，无论追踪是在实时进行，还是通过分析此前收集的数据实现。Claude 不得被用于决定或建议在执法或刑事司法程序中应调查、逮捕或起诉谁。我们还禁止将 Claude 用于构建或改进专门为监控设计的工具。”

Anthropic 在其政策说明中表示，若“与某些政府客户签订的合同……在 Anthropic 判断下，其合同使用限制和适用保障措施足以减轻潜在危害”，这些规则可能会被修改。该公司此前曾与美国军方签约。

Anthropic 已发现多次有人试图利用 Claude 开发武器指导方案和控制软件。

Anthropic 还新增了一项规则：当 Anthropic 的 AI 模型“连接到能够自主执行物理动作、且可能具备造成伤害能力的硬件”时，“必须由合格操作员能够观察该设备，并在必要时将其停止”。（目前尚不清楚该操作员是否必须是人类。）这反映了 AI 实验室正日益投资机器人及其他硬件，以赋予 AI 系统实体载体的趋势，也可能暗示 Anthropic 在这一领域未来的合作方向。

过去一年中，Anthropic 多次表现出对其 AI 模型可能在某种意义上具有意识这一想法的兴趣。负责该公司模型福利研究的 Kyle Fish 在今年 2 月接受 The Verge 采访时表示：“随着模型变得更加复杂和更有能力，有关潜在内部体验、意识、道德地位和福利的问题，都是我们正在研究的严肃问题。”同月，Anthropic CEO Dario Amodei 在一档播客中表示：“我们不知道这些模型是否具有意识。”

AI 行业的其他参与者则明确反对这类观点。去年 9 月，Microsoft AI 的行为准则因对 AI 福利采取强硬立场而成为新闻焦点。Mustafa Suleyman 发布的一份草案写道：“模型福利这一想法是错误的。AI 不应拥有权利或法律人格。”但该文件后来某个时候改为更审慎的表述：“尽管 AI 意识科学远未定论……我们拒绝追求法律人格，也拒绝模型可能值得获得福利或享有权利这一观点。”

Anthropic is making changes to its usage policy for the first time in over a year to reflect new and high-risk cases of misuse — including election interference, weapons development, surveillance, and health and financial uses. But one of the most significant changes prohibits “sustained and needless abusive or cruel behavior” toward Claude.

Last August, the company announced it would allow Claude to end conversations with “persistently harmful or abusive” users as part of its research into “model welfare.” The new update says that terminating conversations is still the “primary enforcement mechanism”; Anthropic did not provide a comment on whether there would be further enforcement mechanisms, such as potential user bans. Anthropic wrote that the new policy update “is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”

Beyond the model welfare change, Anthropic has also gathered several “scattered” existing restrictions into a new ban on deceptive commercial or political campaigns, addressing the growing issue of AI-generated propaganda. It restricts “efforts to obscure who is behind a message or amplify content through fake accounts or posts.” Its election section also prohibits voter deception and election disruption, including spreading misinformation “about candidates or how to vote, impersonating candidates or election officials, or trying to suppress turnout,” Anthropic wrote.

Terminating conversations is still the “primary enforcement mechanism.”

Anthropic said that though its usage policy has always publicly banned weapons development using Claude, the company has seen multiple attempts to use Claude to develop guidance for and control software for weapons — so the new usage policy expands the ban to include “ software and components that make weapons work, as well as actions like arming drones and other autonomous vehicles.” And since the company’s last threat intelligence report suggested that people were using Claude-powered surveillance more and more to identify and track “political dissidents”, the company has made its surveillance bans clearer.

“Tracking people without their consent is prohibited, whether it happens in real time or through analysis of previously collected data,” Anthropic wrote. “Claude cannot be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process. We also prohibit Claude from being used to build or improve tools designed for surveillance.”

Anthropic states that these rules may be modified for “contracts with certain governmental customers … if, in Anthropic’s judgment, the contractual use restrictions and applicable safeguards are adequate to mitigate the potential harms.” The company has previously contracted with the US military.

Anthropic has seen multiple attempts to use Claude to develop guidance for and control software for weapons.

Anthropic also added a new rule that when Anthropic’s AI models are “connected to hardware that takes autonomous physical actions and which might be capable of causing injury,” then a “qualified operator must be able to observe the equipment and stop it if needed.” (It’s not clear if that operator must be human or not.) This reflects the growing trend of AI labs investing in robotics and other hardware to give AI systems physical embodiments and may offer hints at Anthropic’s future partnerships in that field.

Over the past year, Anthropic has repeatedly flirted with the idea that its AI models could be conscious in some way. “Questions about potential internal experience, consciousness, moral status, and welfare are serious ones that we’re investigating as models become more sophisticated and capable,” Kyle Fish, who leads the company’s model welfare research, told The Verge in February. That same month, Anthropic CEO Dario Amodei said on a podcast, “We don’t know if the models are conscious.”

Other parts of the AI industry have been expressly against such ideas. This past September, Microsoft AI’s code of conduct made headlines for taking a firm stance on AI welfare, with a draft posted by Mustafa Suleyman reading, “The idea of model welfare is wrong. AI’s should not have rights or legal personhood.” But at some point, the document changed to have a more measured take: “Whilst the science of AI consciousness is far from settled … We reject the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.”

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• Hayden Field

• AI

• Anthropic

• Report