商业产业 Hacker News Claude 2026-08-16T21:53 正文 ok 4636 words

Anthropic 在 Claude 中对文本“加水印”的篡改,是对写作的扭曲

英文标题:🔶 Anthropic's 'Watermark' Text Adulteration in Claude Is a Perversion of Writing

Anthropic 宣布,为遵守欧盟相关规定,未来将在全球所有 Claude 模型生成的内容中加入水印,其中包括文本。其文本水印并非隐藏字符,而是通过在推理时偏向特定词语选择的概率式语义隐写技术实现;文本越长,检测结果才越可靠。作者认为,这种做法会为了可追溯性而干预模型的措辞选择,即使在私密对话中也可能牺牲清晰度、准确性和写作品质。文章还批评欧盟规则容易被改写工具绕过,却可能主要给诚实用户和辅助性使用者带来麻烦。
打开原文相关 2 · 机会 0
media3.9 KB media2.7 KB media123.1 KB media23.2 KB

中文

Anthropic 在 Claude 中对文本“加水印”的篡改,是对写作的扭曲

2026 年 8 月 16 日,星期日

本周我在这篇文章中写到,Anthropic 宣布全球所有 Claude 模型将很快开始为其生成的一切内容添加“水印”,包括文本,以遵守这项欧盟法规。当时我们只能猜测这究竟会如何运作,因为 Anthropic 对运作方式连模糊的描述都没有提供——尽管其公告标题荒谬且侮辱性地写着:“Claude 如何标记 AI 生成内容”。

我最初猜测,他们或许会在文本中隐藏不可见、不可打印的 Unicode 字符。只是随口推测。事实证明,他们并不打算这么做。他们要做的是采用一种隐写术:在推理时通过词语选择(或其他 token 输出)留下指纹,之后或许可以通过概率方式检测出来。

我最初猜测是“不可见字符”,并不是因为我没有想到基于语义词语选择的技术,而是因为我是个傻瓜,竟然相信了 Anthropic 对其做法的描述。他们最初的支持文档声称:

当支持的 Claude 模型生成文本时,它会将不可察觉的水印直接织入文本本身。你看不见它,而且它不会改变 Claude 回应的含义、质量或可读性。

他们说的是“不可察觉”,以及“不会改变含义、质量或可读性”。这是他们自己的话。不是几乎不可察觉。不是轻微地改变含义、质量或可读性。这在我看来很合理,因为这正是我对任何亲自使用的工具所希望——不,所要求——的。为了在文本中嵌入暗示其来源的隐藏线索,让工具牺牲一丝一毫的清晰度、连贯性、含义、质量等等,都是不可接受的。这正是我会要求、也将要求的。而 Anthropic 的(原始)支持文档明确声称,他们的系统能够做到这一点。因此,如果那是真的,我看不出除了在文本中隐藏不可见字符以外还能有什么办法。

我的错误,是相信 Anthropic 所说的:他们的系统不会掺杂和损害其模型生成文本的语义。事实上,这恰恰就是他们计划要做的事。我居然相信一份名为“Claude 如何标记 AI 生成内容”的文件中的任何一句话——而这份文件根本没有解释 Claude 如何标记(或将如何标记)AI 生成内容——真该去检查一下脑子。

它实际上将如何运作

昨天,Anthropic 在一个与原文“Claude 如何标记 AI 生成内容”文章完全不同的网站上——就是那篇完全没有解释其工作原理的文章——发布了“Claude 的文本水印如何运作”。这篇文章确实以普通人能够理解的方式解释了其工作机制。稍后我会回到 Anthropic 最新的、高度委婉且略有误导性的描述。

关于这一主题已有大量研究,其中一些我也会在下文链接。但对于这项技术背后总体思路的最佳说明,是 James Padolsey 的互动文章:“AI 文本水印如何运作”。这篇文章论述极其清晰,互动元素也出色地展示了主要概念。A+ 水准。如果你对此有任何兴趣,我敢说你必须阅读——并亲手玩一玩——Padolsey 的文章。

不过,以下是我尝试给出的、面向普通读者的高层次总结。假如你抛掷一枚硬币 N 次并记录结果,就可以以一定的确定性判断这枚硬币是否公平,或者是否有偏向。以当前流行形态而言,LLM 是非确定性的。向同一个模型提出同样的问题,经常会得到至少略有不同的答案。或许含义相同,但措辞不同。在生成下一个 token 的每个决策点,模型都会作出选择。采用这些语义水印技术时,模型会根据可称为“绿名单”和“红名单”的词表,在部分 token 上作出不同选择。在每一个决策点,它从绿名单中选择词语的概率会略高于从红名单中选择。这并不意味着它绝不会选择红名单中的词。只是与没有采用这种掺杂式标记技术时相比,选择红名单词语的概率更低。(就像一枚有偏的 51:49 硬币,平均每抛 100 次,仍会有 49 次落在“错误”的一面。)

词语或短语会在每个“下一个 token”生成点即时、确定性地被划入绿名单和红名单。因此,某个特定词语有时会在绿名单上,另一些时候则会在红名单上。持有密钥的人可以确定某个词在每个 token 生成点属于哪张名单(这正是检测水印的方式);没有密钥的人则无法做到。这意味着,永远不会存在一份 Claude 偏好或刻意回避的词语清单。

对于抛硬币来说,N 越高——即抛掷次数越多——你就越能有把握地判断硬币是否公平或存在偏向。语义水印也是如此。文本中的词语越多,判断该文本是否由特定 AI 模型生成的分析就越准确。抛硬币次数太少时,根本无法对硬币的公平性得出有把握的结论。词语(或 token)太少时,也无法有把握地判断一段文本是否由 AI 生成。

对于一段需要检查特定水印系统痕迹的文本,如果其中被标记为绿色的词多于原本应有的数量、被标记为红色的词少于原本应有的数量,就可以以一定置信度将该文本标记为由采用特定密钥水印系统的 AI 系统生成,或者仅仅修改过。显然,判断的置信度会因文本字符串的长度,以及赋予绿名单和红名单词语的随机权重而出现显著变化。但只有 Anthropic 能够判定文本看起来是否由 Claude 生成,而且 Anthropic 也只能检测 Claude 施加的水印。Claude 无法检测例如 Gemini 生成的隐藏水印信号,Gemini 也无法检测 Claude 创建的隐藏水印信号,因为每一种实现都以仅由 LLM 提供商持有的密钥为基础。

对技术前提的异议

我对此有一个根本问题:没有任何两个同义词承载完全相同的含义。“他迫不及待地抓住了机会”与“他欣然接受了这个机遇”是表达相同大致情感的非常相似的句子,但它们并不相同。我们写作时选择的确切词语很重要。我希望我使用的任何 LLM,都能在每一个决策点选择最好的、最精确的词语。我接受一个显而易见的约束:时间和计算资源。在快速执行推理、并以每 token 一定成本运行的约束下,我希望得到最好的词语。这个约束与人类写作相同。假如我花更长时间写作,我肯定能写出更好的专栏文章。我写作时会衡量,自己应该为每一个词语和标点的选择投入多少心力。当我的直觉告诉我有必要时,我会在某些段落、句子,甚至个别词语选择上花更多时间。

换句话说,这些是必要的权衡。这些因素全都符合我的利益:速度、成本、质量。理想情况下,我希望获得完美的写作、瞬时的生成速度和零成本。没有一项是可能的。计算不是免费的(而且基于云端的领先模型 LLM 推理实际上相当昂贵)。推理并非瞬时完成。而无论是自然还是人工的优秀写作,都只能接近完美。

文本是为我生成的,任何并非出于我的需求的因素也要介入其生成,这个想法显然令人冒犯。

这不只是关于人们可能意图将其冒充为自己自然创作的文本。甚至也不只是关于 LLM 对手写作品的校对。Anthropic 的意思是,所有新的 Claude 模型都将掺杂它们生成的、超过 200 个 token(约 150 个词)的每一段文本,包括向用户展示供其阅读的一切内容。因此,即使是在用户与 Claude 之间、除用户外永远不会被任何人阅读的私密对话中,Claude 也会开始为了以统计学上可预测的方式标记其输出而作出词语选择,而不是最大化清晰度和准确性。

即使是当今所谓的前沿模型,也已经明显缺乏清晰度。Claude、ChatGPT、Grok 等等的“写作能力”比大多数人更强,写出的散文也比普通人中位数水平更好。但这还用说。大多数人都是糟糕的写作者。“普通人”相当愚蠢,而其中一半人比这还愚蠢。而且有许多聪明、有趣的人,写作却糟糕透顶。因此,尽管 LLM 令人印象深刻,门槛其实很低。我从这些模型中看到的最佳写作,也比不上任何我愿意为了享受阅读而主动选择的作品。而现在 Anthropic 说,它们要故意把它弄得更糟,目的根本不会以任何方式让我受益?哪怕只是稍微变差?

去你的。

对欧盟法规的异议

说到异议,促成这一切的相关欧盟法规——“AI 生成内容透明度行为准则”——是充满繁文缛节、保姆国家式的空想废话。以下是 Ben Thompson 在本周一篇设有付费墙的 Stratechery 更新中所作的总结:

  • 该法规适用于超过 200 个 token 的文本。
  • 提供商必须在服务条款中要求用户不得移除水印。
  • 该方案应能抵抗“典型处理手段”的规避,例如截图、扫描和 OCR、复制粘贴、翻译等。

从字面理解,合规 LLM 的服务条款必须禁止用户改写遵守该法规的模型输出,因为词语选择本身就是标记。但试图将这项荒谬、不切实际、会助长猎巫的法规强加给全世界的并非欧盟,而是 Anthropic。

遵守这项规定,尤其就文本而言,只会给诚实用户制造问题。试图把 AI 生成文本冒充为自己写作的非诚实用户(学生、员工或任何人),只需通过不合规的 AI 改写工具,就可以规避检测。

James Padolsey——我在上文链接了他对这些方案工作原理的互动式可视化解释——在一篇题为“Anthropic 的脆弱水印迎合了一部软弱法律”的文章中解释了这一点(如果这个标题让你觉得耳熟,我今天早些时候曾在一篇独立文章中链接过它):

导致这项法律出现的同一种思维,在计算器刚问世时也本可适用于它,假如计算器的输出会通过某些痕迹暴露自身。幸运的是,源于大脑的计算结果与由计算器产生的结果并未被区别对待。拼写检查器也是如此。只有当工具强大到足以写出完整句子时,才令其辅助行为变得可疑,这并不是一条有原则的界线。这是在给“困难本身”赋予道德溢价。

尽管如此,Anthropic 还是选择了一种覆盖整个模型层面的全面实现方案,其范围似乎比法律最低要求更广。这或许是方便的合规工程做法,但它抛弃了法律明确试图保留的区分。结果是:该信号宽泛到足以牵连无害和辅助性的使用,却又脆弱到只要有动机的人进行大幅度重写便可被移除。它可能会将怀疑集中在普通用户和辅助性用户身上,同时对蓄意欺骗却最为无力。

Padolsey 是 Declaude 的创建者。这是一款简洁得令人愉悦的网页应用,让你可以“粘贴带有 AI 味道的文本,并获得内容相同的普通散文”。Declaude 最初的用途,是清除文本中 Claude 那种甜腻的人格臭味(无论文本由 Claude 还是任何其他 LLM 创建);但如果 Anthropic 坚持其已宣布的计划,开始掺杂 Claude 生成的所有文本,Declaude 也将成为通过复制粘贴、多加一个步骤即可消除这些标记的方式。Declaude 已经很有趣也很有用,但它正好说明了这项欧盟法规在散文文本方面是多么考虑不周且徒劳无功。

Google SynthID

Google 已经部署了一套名为 SynthID 的水印系统,用于 AI 生成的图像、视频、音频和文本。本文只关心文本。对于多媒体内容,嵌入式水印可以是文件内部的元数据,在观看或收听时确实不会影响作品的体验质量。但对于文本,我们讨论的是实际被选择出来的词语。以下摘自 Google DeepMind 对 SynthID 的说明中的“AI 生成文本”部分:

我们已将 SynthID 扩展至对 Gemini 应用和网页体验所生成文本进行加水印和识别。大语言模型

English / 原文

Anthropic’s ‘Watermark’ Text Adulteration in Claude Is a Perversion of Writing

Sunday, 16 August 2026

When I wrote this week about Anthropic’s announcement that all Claude models, worldwide, would soon begin “watermarking” everything they generate, including text, to comply with this EU regulation, we were left to speculate how this was going to work, because Anthropic offered not even a vague description of how it would work — despite the fact that the title of the announcement was, absurdly and insultingly, “How Claude Marks AI-Generated Content”.

My initial speculation was that maybe they’d hide invisible non-printing Unicode characters in the text. Just spitballing. Turns out that’s not what they’re going to do. What they’re going to do is apply a form of steganography, where the choice of words (or other token output) at inference time will leave fingerprints that can later, maybe, be detected probabilistically.

I initially guessed “invisible characters” not because I didn’t think of the semantic word-choice technique, but because I was a fool who took Anthropic at its word in their description of what they would do. Their original support document claims:

When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response.

They say “imperceptible” and “doesn’t change the meaning, quality, or readability”. Their words. Not almost imperceptible. Not slightly changes the meaning, quality, or readability. That made sense to me, because that’s absolutely what I want — nay, demand — from any tools I use personally. It’s unacceptable for a tool to sacrifice an iota of clarity, coherence, meaning, quality, etc. for the purpose of embedding hidden clues within the text to suggest its provenance. That’s what I would and will demand. And Anthropic’s (original) support document unambiguously claims that’s what their system will enable. So if that were true, I couldn’t see what was left other than hiding invisible characters within the text.

My error was believing Anthropic that their system wouldn’t adulterate and corrupt the semantics of the text their models generate. That is in fact exactly what they plan to do. I should have my head examined for believing a single word of a document titled “How Claude Marks AI-Generated Content” that doesn’t explain, at all, how Claude marks (or will mark) AI-generated content.

How It’s Actually Going to Work

Yesterday, on an entirely different website than the original “How Claude marks AI-generated content” article (the one that didn’t explain anything at all about it works), Anthropic published “How Claude’s Text Watermark Works”, which does actually explain in layman-accessible terms how it’s going to work. I will return to Anthropic’s new highly euphemistic and slightly misleading description below.

There’s a bunch of research on this topic, some of which I have also linked to below. But the very best description of the general idea behind the technique is an interactive essay by James Padolsey, “How AI Text Watermarking Works”. It’s a wonderfully cogent read, and the interactive elements splendidly illustrate the main concepts. A+ work. If you have any interest in this at all, I dare say you must read — and play with — Padolsey’s piece.

But here’s my stab at a layman’s high-level summary. If you toss a coin N times and note the results, you can determine with a degree of certainty whether the coin is fair or biased. LLMs are, in their popular incarnations, non-deterministic. Ask the same question of the same model and you often get at least slightly different answers. Maybe the same meaning, but different phrasing. At each decision point for generating the next token, the model makes a choice. With these semantic watermarking techniques, they make different choices for some tokens based on word lists that could be called “green” and “red”. At each decision point, they’re a little more likely to pick a word from the green list than the red list. That doesn’t mean they never choose words from the red list. Just that they’re less likely to than they would if the adulterated marking technique weren’t in place. (Same way that a crooked 51-49 coin will still land “wrong” side up 49 times out of 100 on average.)

Words or word phrases are sorted into the green and red lists deterministically on the fly, at each “next token” generation point. So sometimes a specific word will be on the green list, and other times it will be on the red list. Someone with the secret key can determine which list a word will be on at each token generation point (which is how the watermarking is detected); those without the secret key cannot. This means there will never be a list of words that Claude prefers or eschews.

With coin flipping, the higher N is — the more times you flip — the more confident you can be that the coin is fair or biased. So too with this semantic watermarking. The more words in the text, the more accurate the analysis will be that the text was generated by a specific AI model or not. With too few coin flips, you can’t achieve any confidence at all regarding a coin’s fairness. With too few words (or tokens), there’s no way to achieve any confidence whether a string of text was AI-generated or not.

Given a string of text to examine for signs of a specific watermarking system, if there are more words tagged as green and fewer tagged as red than would otherwise be expected, the text can be flagged — with some degree of confidence — as having been generated, or merely modified, by the AI system that applies the specific secret-key watermarking system. The amount of confidence in the determination will obviously vary, significantly, based on the size of the text string and randomized weights given to words on the green and red lists. But only Anthropic will be able to determine if text was seemingly generated by Claude, and Anthropic will only be able to detect the watermarks that are applied by Claude. Claude can’t detect the hidden watermark signals generated by, say, Gemini, and Gemini can’t detect the hidden watermark signals created by Claude, because each implementation is predicated on secret keys held only by the LLM provider.

Objections to the Technical Premise

One of my fundamental problem with this is that no two synonyms carry the exact same meaning. “ He leaped at the chance ” and “ He jumped at the opportunity ” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter. I want any LLM I use to choose the very best, most precise words at every single decision point. An obvious constraint that I accept is time and computation. Within the constraint of executing inference quickly, and at a certain cost per token, I want the best words. This constraint matches human writing. I could surely write a better column by taking longer to write it. I write with a sense of how much care I should put into every word and punctuation choice I make. I take more time with certain paragraphs, sentences, or even individual word choices when my gut feeling says I should.

In other words, these are necessary trade-offs. These factors are all in my interest: speed, cost, quality. Ideally I would like perfect writing, at instantaneous generation speed, at zero cost. None of those things are possible. Computation is not free of charge (and cloud-based LLM inference with leading models is actually expensive). Inference is not instantaneous. And great writing, whether natural or artificial, can only approach perfection.

The idea that anything other than my needs should factor into the generation of text for me is patently offensive.

This isn’t just about text one might generate with the intention of passing it off as their own natural work. This isn’t even about LLM proofreading of work written by hand. Anthropic is saying that all new Claude models are going to adulterate every single bit of text longer than 200 tokens (~150 words) they generate, including everything it presents to its users to read. So even in a private conversation between a user and Claude, which will never be read by anyone other than the user, Claude will begin making word choices in the name of marking its output in statistically predictable ways rather than maximizing clarity and precision.

Even today’s so-called frontier models are already decidedly lacking in lucidity. Claude, ChatGPT, Grok, et al. are “better writers” than most humans and produce better prose than the median human. But: no shit. Most people are terrible writers. The “average person” is pretty stupid and half of all people are stupider than that. And there are many smart, interesting people who are miserable writers. So as impressive as LLMs are, the bar is low. The best writing I see come out of these models is worse than anything I would choose to read for pleasure. And now Anthropic is saying they’re going to make it worse, on purpose, for purposes that do not benefit me in any way? Even if only slightly worse?

Get fucked.

Objections to the EU Regulation

Speaking of objections, the relevant EU regulation motivating all of this, “Code of Practice on Transparency of AI-Generated Content”, is red-tape nanny-state pipe-dream nonsense. Here’s Ben Thompson’s summary from a paywalled Stratechery update this week:

  • The regulation applies to text longer than 200 tokens.
  • The provider must mandate in their terms-of-service that users not remove the watermarking.
  • The solution should be robust in terms of evading “typical processing solutions” like screen shots, scanning and OCR, copy-and-pasting, translations, etc.

Taken literally, compliant LLM terms of service must forbid users from rephrasing the output from models that comply with this regulation, because the word choices are the marks. But it’s not the European Union that is trying to impose their absurd, impractical, witch-hunt-fueling regulation on the entire world. That falls on Anthropic.

Complying with this, particularly with regard to text , is only going to create problems for honest users. Dishonest users attempting to pass off AI-generated text as their own writing (students, employees, whoever) will simply circumvent detection through non-compliant AI paraphrasing tools.

James Padolsey — whose interactive visual explanation of how these schemes work I linked to above — explains this in a post titled “Anthropic’s Weak Watermarks Appease a Weak Law” (which, if it rings a bell, I linked to in a standalone post earlier today):

The same thought that led to this law could have applied to calculators at the time of their inception, had their outputs revealed themselves through artefacts. Thankfully, a sum borne of the brain is treated no differently from one produced by a calculator. Likewise with spellcheckers. To make assistance suspect only once the tool becomes capable enough to compose a whole sentence is not a principled boundary. It is a moral premium placed on difficulty itself.

Anthropic has nevertheless chosen a blanket, model-level implementation that appears broader than the law’s minimum requirement. That may be convenient compliance engineering, but it discards distinctions the law expressly attempted to preserve. The result is a signal broad enough to implicate harmless and assistive use, yet fragile enough to be removed by a motivated person through substantial recomposition. It risks concentrating suspicion on ordinary and assistive users while remaining weakest against deliberate deception.

Padolsey is the creator of Declaude, a delightfully simple web app that allows you to “Paste in AI-flavored text and get the same content back as plain prose”. Declaude’s original purpose is cleaning the saccharine Claude personality stink from text (whether it was created by Claude or any other LLM), but, if Anthropic persists in its stated plan to begin adulterating all text Claude generates, Declaude will also serve as a copy-paste single-extra-step way to eliminates those marks. Declaude is interesting and useful already, but it exemplifies how ill-considered and futile this EU regulation is when it comes to prose.

Google SynthID

Google has a watermarking system in place that they call SynthID, which they apply to AI-generated images, video, audio, and text. I’m concerned in this article only with text. With multimedia, embedded watermarks can be metadata within files , and truly not affect the experiential quality of the work when viewed or listened to. With text, we are talking about the actual words that are chosen. From the “AI-generated text” section of Google DeepMind’s own description of SynthID:

We’ve expanded SynthID to watermarking and identifying text generated by the Gemini app and web experience. Large language models generate text one word (token) at a time. Each word is assigned a probability score, based on how likely it is to be generated next. So for a sentence like “My favorite tropical fruits are mango and…”, the word “bananas” would have a higher probability score than the word “airplanes”. SynthID adjusts these probability scores to generate a watermark. It’s not noticeable to the human eye, and doesn’t affect the quality of the output.

In a group chat, a friend of mine quoted the above, and I responded that if a chatbot wrote “My favorite tropical fruits are mango and airplanes”, I’m pretty sure I’d fucking notice. Another friend then responded with this:

AI-generated image of an airplane carved out of a pineapple or something, on a tropical beach.

Days later, that still cracks me up.

But Google’s absurd description puts the lie to their own claim that it isn’t noticeable, and it serves to show just how little regard the people behind these generated-text fingerprinting schemes have for the actual craft of writing. Of course bananas has a higher probability score than airplanes , because airplanes aren’t fruit. But what about pineapple? Should the sentence complete to “mango and bananas” or “mango and pineapple”? That’s a good question, and the only acceptable answer for why an LLM should choose bananas instead of pineapple (or coconut , or guava , or papaya...) is that it has determined that it’s the best fit for the intended meaning, tone, and sentiment of the text. Not because bananas is on the watermarking “green” list and pineapple is on the “red” list, even though pineapple might be the better fit. Google’s own supposedly jocular description of how SynthID works in fact captures how the scheme perverts the text it generates.

They’re saying you won’t notice because if it only chooses bananas over pineapple for these fingerprinting purposes, well, they’re both tropical fruits and who cares. But it’s utter nonsense that the difference is “not noticeable to the human eye”. The semantic difference between banana and pineapple is just as noticeable to the human eye as the as the taste of the two are to the human tongue.

If it did produce “My favorite tropical fruits are mango and airplanes”, it’d be incredibly stupid, but it wouldn’t be offensive because we’d all recognize that something completely off-key happened. What’s offensive is that with a system like SynthId in place, where the fingerprinting decisions are motivated by a secret key, we have no idea whether it completed to “mangos and bananas” because bananas was determined to be the best next token, or because bananas is in the “green” bucket of words. It calls every single word choice into question.

Here’s a paper published in Nature where Google’s team behind SynthID published their work, after putting it into production with Gemini (née Bard):

We analysed approximately 20 million watermarked and unwatermarked responses and computed the thumbs-up and thumbs-down rates (both as a fraction of the total number of thumbs-up and thumbs-down feedback received). We found that the thumbs-up rate for the two models differed by 0.01% (with the watermarked model being higher); and the thumbs-down rate differed by 0.02% (with the watermarked model being lower). We found both of these differences to be statistically insignificant, and well within the 95% confidence intervals.

From this experiment, we conclude that over a wide variety of real chatbot interactions, the difference in response quality and utility, as judged by humans, is negligible. Subsequently, non-distortionary SynthID-Text has been productionized and is currently watermarking responses in Gemini and Gemini Advanced. To the best of our knowledge, this evaluation represents the first systematic watermarking investigation of its kind within a large-scale production system.

To this I say:

  • Gemini/Bard’s thumbs-up/thumbs-down buttons are not a good experiment for evaluating the effect on quality. If a chatbot tells me “My favorite tropical fruits are mango and bananas” instead of “mango and pineapple”, I’m not going to give the response a thumbs down because of the fruit it chose. I’d give it a thumbs down if said “airplanes”, yes, but that’s a strawman. (The paper in Nature even uses the “My favorite tropical fruits are mango and ...” example as an illustration, but in the paper, the only four next tokens considered are, in order of probability distribution, mango, lychee, papaya, and durian. No airplanes. And, conveniently, in the paper’s example, the “winner” of the watermarking “tournament” just happens to be mango, the one that would have been selected as the best if the watermarking weren’t in place.)

  • A “difference in response quality and utility, as judged by humans” that is “negligible” does not mean imperceptible. What they really mean is that it’s only slightly worse and that everyone is either too stupid to notice or too indifferent to care.

  • It’s widely considered that Gemini is behind ChatGPT and Claude in quality. Perhaps the fact that they’ve put SynthID-text into production is one of many reasons why. I personally agree that Gemini’s prose is inferior. Maybe the use of SynthID has nothing to do with the fact that I, along with the general public consensus, consider Gemini to be a second-rate chatbot — but in that case, maybe it’s the fact that Gemini is a second-rate chatbot that makes the difference “negligible” when Google started mixing in SynthID-motivated tokens in its results. It’s a lot more likely that your restaurant customers won’t notice that you replaced your regular coffee with Folgers Crystals if your regular coffee is second-rate to start with.

Anthropic

Now, finally, back to Anthropic’s new “How Claude’s Text Watermark Works” published yesterday. I have some comments.

To summarize:

  • We use a method of watermarking that does not have any practical impact on the quality or content of Claude’s outputs;

  • The difference between watermarked and un-watermarked text will not be distinguishable to readers;

Translation: Specific words do not matter and we don’t think anyone reads anything closely.

  • Nothing is added to the text and there are no hidden characters;

This would have been worth clarifying at the outset.

  • Watermarking won’t be specific to Claude. As of August 2, the EU requires AI providers serving its market to mark AI-generated content. Other major model developers have signed the same Code of Practice and will be implementing their own watermarks.

No other AI provider has stated that they will apply such marking, adulterating all generated text, outside the EU.

Take the sentence “The weather today was cold and…”. The next word is very unlikely to be “sugary.” But it is quite likely to be “overcast” or “grey.” Under most circumstances, it doesn’t matter much to the reader which of these latter two words the model ultimately chooses — the meaning of the sentence is largely the same either way. In cases like this, the choice is settled by a random number.

Arguing that grey vs. overcast “doesn’t matter much to the reader” is the crux of my argument that this entire endeavor is a perverse adulteration of what it means to write — or to read. That it’s subtle in some ways makes it more perverse, because it’s sneaky.

In internal testing, we’ve seen no impact of watermarking on the content, level of creativity, or readability of Claude’s text. In the SynthID-Text paper, which introduced the technique we use, Google DeepMind tested this impact by serving a model that used watermarking to a portion of their Gemini traffic and comparing thumbs-up and thumbs-down ratings. They found no statistically significant differences from the unwatermarked model. And in a controlled study, human raters comparing watermarked and unwatermarked answers side-by-side saw no difference in quality.

See above for my argument that this thumbs-up/thumbs-down data is absolutely worthless in evaluating whether the SynthID-style word-bias watermarking makes text worse. By definition it must make text worse, unless the underlying LLM model’s scoring is wrong, because the nature of the watermarking algorithm requires it to sometimes increase the probability of selecting a worse word choice and decrease the probability of selecting the model’s best choice. It’s only a question of how much worse. What Google’s thumb-counting data shows is only that it isn’t so much worse as to make Gemini users click the thumbs-down button.

Watermarking doesn’t change the meaning or experience for the person reading it, but if you wanted to check after the fact whether the text was likely generated by Claude, the watermark allows you to do so.

No, it does not. Because the entire scheme is tied to secret keys held only by the AI provider, it only allows Anthropic, not “you”, to check anything.

When Claude proofreads text written by a person, what it gives back has generally only been lightly edited; because nearly all the words are the person’s, there’s very little (if anything) for the watermark to attach to. Depending on the length of the text and how heavily Claude has edited it, those changes might not be enough to make Claude’s involvement detectable. The more Claude writes, the more decisions it has to make, and the more space there is for a watermark.

Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude.

For example, once the model has written “2 + 2 =”, there is a very clear best choice for the next token (if the model is completing the sum, there isn’t an answer that’s equally as good as “4”; if it’s talking about George Orwell’s Nineteen Eighty-Four , there isn’t an answer that’s equally as good as “5”). The “nudge” of the watermark wouldn’t be applied here. For the same reason, code — which in very many cases has to be exact — has generally less watermarking than some other forms of text.

Having said that, in areas where there is an arbitrary choice between particular words or terms within the code, the watermark can be used, such as comments within code. But by definition, it will have a negligible effect on the actual code produced.

Translation: We value precision in programming code; we do not in prose.

And it is exceedingly rich to cite George Orwell’s Nineteen Eighty-Four , approvingly, in the context of justifying a text adulteration scheme premised on the notion that specific words do not matter. I mean what the actual fuck? Orwell!

Lastly, as to why they’re doing this:

We’re implementing watermarking to comply with the EU AI Act. Anthropic, along with several other major AI model providers and around 190 total signatories, signed the EU Code of Practice on Transparency of AI-Generated Content in July 2026. This requires AI system providers to use methods of “marking” AI-generated text. We’re applying watermarking globally at launch because we don’t yet have a durable way to scope it by region.

This, from a company that the Financial Times just reported is weeks away from an IPO with an intended valuation of $2 trillion, which would make it one of the 10 highest-valued companies in the world — as of today, placing it at #7, between TSMC ($2.2T) and Broadcom ($1.9T).

This leaves us to believe that one of the following must be true:

  • It’s perfectly reasonable that a technology company valued on par with Amazon and TSMC is technically incapable of complying with an EU regional law only within the EU itself.1 Not a cause for concern at all.

  • Anthropic is in over their heads, wields shockingly little control over their own tech stack, and their imminent IPO is likely to be remembered only as a new high-water mark in the manic global AI bubble.

Also, what happens if another major global market makes it unlawful for AI to secretly watermark generated text?

OpenAI

From an OpenAI support document titled “Provenance Signals (Content Credentials, SynthID) in OpenAI-Generated Content”:

Consistent with our commitments under the European Commission’s Code of Practice on Transparency of AI-generated content, our goal is to expand provenance signals to all modalities including text, so customers and developers have clear ways to meet their own transparency obligations as standards and tooling continue to mature.

There’s a lot of wiggle room in this brief statement, and it could just as well mean that OpenAI models will only adulterate text with fingerprint markers when users or developers ask for it. Or that it will only be mandatory for users in the EU. If I were at OpenAI I’d go hard on this and publicly say that ChatGPT will never watermark text it generates unless you ask it to , and that if you want tools that secretly work behind your back without telling you how they work to flag your words in ways you can’t see, go ahead and use Claude.

Further Reading

Three papers on ArXiv:

I will admit that while I’m profoundly offended by the idea of personally using tools that attempt to leave such watermarks in text they produce or touch, the mathematics behind it are fascinating.

Michael Lopp, at Rands in Repose, “RIP Claude”:

As a human who has had to wrangle with EU regulations in the past, I am abundantly clear what’s involved in the laborious bureaucratic process. I can guess what threats Anthropic is facing. However, this is a tone-deaf, clumsy, and alarming opening salvo in their watermark strategy. [...]

My writing is my work, and Anthropic’s current strategy is aggressively writer-hostile.

Jeff Gamet, “Anthropic’s Claude Watermark Is Akin to an AI Poison Pill”:

To be clear, the watermarking is embedded in pretty much any text Claude touches. Along with text Claude generates, it also applies to text it processes, such as proofreading and summarizing. I expect we’ll see too many inaccurate accusations of using Claude to write documents where the content was human-written, but AI-proofread.

The watermarking sticks with documents through copy-and-paste, too. Imagine copying text from a blog post or email only to have what you wrote tagged as potentially AI-generated. In fact, that could very well happen with this post. I personally write all of my content without AI tools, but I copied the quote at the top of this piece directly from Anthropic’s website. Does that mean what I wrote here will show as AI-generated? If they used their own models to generate or edit what I quoted, then the answer is very likely “yes.”

One of the papers published at ArXiv I cited above claims that such watermarking even persists when an article of text originally generated in English is translated into German.

Secrets are the poison here. When only Anthropic holds the secret keys that both produce the watermarking and perform the probabilistic detection of those marks, we’re all left to wonder. To wonder if what we’re reading is secretly watermarked, what we’re quoting is secretly watermarked, and whether what we ourselves are writing will be unjustly accused of being AI-generated based on secrets we don’t know and can’t see. Poisonous is exactly the right word.

Or should I say toxic? Or airplanes?