How AI text watermarking works

Join the discussion

• Hacker News
• Would OpenPangram's LoRa fine-tuning training be able to pick up on the patterns of green left behind by LLM vendors? https://huggingface.co/pangram/editlens_Llama-3.2-3B
  by thadk
• Do you need to know the full conditional probability distributions of the model to tell if it's a watermarked text, or does this work without that knowledge, i.e. without having the access to the full weights?
  by storus
• I’d be curious to play with this with some different open weights models. I’d think that a single next token would end up having similar probability distributions so given that this a probabilistic method, you’d be able to get at least some signal.

  I suppose if that did work someone would have been able to work backwards and crack their key already, so I must be missing something.

  by gibspaulding
• I think that's the part the article glosses over, probably because it's such an evidence to the author.

  The only way this works is to use the same exact model and weights right? So that you can replay the text generation as it would have been originally done, and compare output?

  And then what, if there is no match do you need to retry with all other known models that could have been used?

  Or are models sufficiently similar that they are interchangeable for this type of watermark?

  And what if a competing or open source model was used? I can't see how the watermark would work.

  And if you have access to a non-watermarked output? How can you prove they are not simply using another key? How can you be sure the text is not watermarked? From the explanations, you can't.

  by smashed
• I wonder how much output text it would take to work out the key that the watermarker is using?
  by oidar
• I suspect there are too many potential methods to reliably reverse engineer.
  by nonethewiser
• "Whoever gets to lean on how the dice land"

  I'm a native English speaker and I have no idea what this means.

  by fnord77
• The example above that paragraph shows how words might be chosen using fair dice. Watermarking works by using weighted (unfair) dice. It's saying that you can hide watermarking by weighted the dice, i.e. "lean on how the dice land."

  If you then know how the particular dice weights, you can later use that to confirm watermarked text.

  by js2
• I wonder if this is for use in future legal battles over ownership/creation/invention of new software and/or concepts.

  For example, someone inventing a new technology might use AI to assist with code prototypes/doc prep etc. Where would that leave the rights of the owner/creator/inventor. These text watermarks provide fuel for legal battles.

  Am I wrong?

  by calif123
• My guess would be to detect if their models are being distilled by other models.
  by bryzaguy
• I think the goal is not to validate if you used Claude, the goal is to insert enough information in your code to identify who you are over at Anthropic, like a covert unique hash to identify you, hidden in plain sight, but that's my own speculation and not based on anything.

  I'm having a hard time thinking of any other use case... What other reason is there to fingerprint your code?

  I assume if they arrest you for vibe coding something that violated computing laws (hacking) they can then scan your code, send it to anthropic, anthropic confirms that Claude was used on your account, on x device to build malware.

  Remember they busted a hacker because of his Windows unique install ID.

  by giancarlostoro
• I wonder how the keys and watermarking survives when you ask the model to write in the style of another author or something that deviates from its classic patterns. I once asked Claude a while back to write some short stories in the Expanse universe and style and be curious if the watermarking survived that text.
  by antilimit
• As Anthropic explains it, the watermark is at the model level. It just produces stuff that matches the watermark, prompts and instructions don't matter.
  by theshrike79
• I wonder if this will become a new revenue stream for providers. Want to know if Claude generated some text? There's a free web form you can paste into.

  Of course, you can also run your own check service if you pay some API fees, and those checks can be a lot more convenient for users since the services can check multiple sources, to whom they are paying for the privilege.

  Then someone washes the text through a local model that rewords it, the markers are lost, amd they're clear again.

  by techjamie
• It doesn't look like the providers are going to publish their randomisation key. The google version is accessible only via the gemini prompt which only actually ran the synthID tool 1 out of 2 times I tried it the other day. So at the moment the best approach would be taking the suspicious test and using claude computer use to feed it into each provider's prompt brower window "manually"...
  by ImaCake
• I can see this being real useful when they scrape the internet for content. And they don't want their own crap. Or just to see how prevalent their stuff is.
  by sroussey
• One aspect that people seemingly aren't talking about is the impact this has in the model's creativity. Because the model will nudge each word towards group A vs group B, you're losing on creativity, especially more so if the nudge isn't a gentle 55% but something like 70% or 80%. So essentially they're forcing the model to be less creative for the upside that the longer the text the easier it is to detect the watermark.

  Here's a visual representation of the watermark: https://i.imgur.com/JNUIykX.png

  This is even worse in such forms of writing like coding, where there's even less choices the model can make on what the next token should be. Plain text in code will obviously be watermarked, that includes comments. But the code itself might get watermarked by choosing certain code over others more often.

  I'm inclined to believe the models will be instructed to not watermark code, especially since it's harder to detect reliably because the shorter the body of text the harder it is to detect, but who knows what Anthropic and all the other AI labs will decide to do in the future.

  EDIT: Also for those in the comments who are naive enough to think Anthropic is doing this just because the EU said so and not because it's beneficial to them (and all other AI labs), well, you are indeed naive. Identifying code will be paramount in training future models because the more synthetic data you feed it, the more cannibalization happens, the worse the models will perform over time due to lack of good data, among other such reasons as selling AI detection services to colleges, and a plethora of other reasons.

  by lemoncookiechip
• It is worse than that, it can make text unintelligible. I have Claude explain what is happening in a PR, and the technobabble and use of rare words have to look up in a dictionary make it difficult to understand. I need another LLM to translate what the LLM is saying.

  And I pray it is because of the EU AI Act, because it is worth giving them my personal information to prove I am not in the EU so they can turn this off. Heck, I will pay more to avoid this crap.

  If it were not so noticable, I would shrug it off. But it has made things clearly worse this year.

  by sroussey
• The details will depend on the exact implemention, but I don't think this necessarily effects output quality.

  The models "natural" output is the result of a series of random numbers. The watermark works by biassing that series towards a different series of numbers. Assuming that second series is cryptographically secure psuedo-random, the even distinguishing the biased sequence from true random would be impossible with compromising the key or prng.

  As an extreme, suppose your prompt was public, and the model seeded its PRNG with a secret key instead of a genuine random seed. Such an output is not meaningfully different from one based on a true RNG, but can be trivially fingerprinted by someone who knows the keys.

  In practice, I am doubtful they have a scheme that is both practically useful and cryptographically secure. However, there is a lot of room below cryptographically secure that is still just as good for all other purposes.

  by gizmo686
• As per the link, the words in the green and red groups are calculated dynamically, so it's not like the model is going to be told "use 'unique' over 'unusual'" and suddenly writing from the model will contain the word 'unique' far more often than 'unusual'. So I'm not sure it's clear that this has an impact on creativity as such?

  That being said, I do question how this will apply to code as opposed to prose. Even data dense text (ie, if you ask Claude to evaluate what running shoe to buy, and it spits back a list of options with reviews and prices) may struggle.

  What it probably will work well at it flagging the current tsunami of entirely AI generated novels on Amazon/Kindle, which is...honestly not without value.

  > Identifying code will be paramount in training future models

  True, but note that this strictly allows providers to identify text generated by their own models. If Anthropic wants to filter out GPT generated text in their training data, they'll need to feed it through an OpenAI API, which is implausible. So it might help on the margins, but I don't think it solves the problem of model collapse.

  by Lazare
• I almost never copy & paste AI-generated text, I almost always transcribe it by hand, which in turn forces me to read what the LLM generated and gives me the opportunity to replace words as I go. This obviously doesn’t scale, especially if your impact is measured by the number of software features you implement, but for more experienced engineers (Staff, Principal, and above) who are usually evaluated on the success of company-wide initiatives, I think this is the best course of action.
  by guessmyname
• And certainly never copy/paste an assignment
  by nonethewiser
• One thing you can try is to write it by hand without an LLM! That way there will be no watermark.
  by uncivilized
• New job idea: have a human reword/summarize and manually input/transcribe AI output to remove the watermarking. They use “tools” like dictionaries and thesaurus’ in order to sufficiently change the text so that it doesn’t fit within AI distribution anymore. Humans that can write significantly “organic” text will be able to make lucrative careers out of it.
  by throwatdem12311
• So a writer

  Burn, witch

  by nonethewiser
• This reads like translate this English into Spanish into English into Spanish into English into Spanish into English and get something totally different than the original.
  by dylan604
• Full circle, the fired content writers are going to get rehired. What a time to be alive.
  by system2
• What's wild (imo) is pretty much everyone I talk to/read from (anecdata) HATES the way claude writes. I see it in the comments on Hacker News, hear about it in discussions with my colleagues, and talk about it with my non tech family. it's over the top bad. now it seems like these quirks will now be enforced in some weird way to meet the watermarking rules?
  by wpasc
• Its quite bad. I use it for code all the time and then recently used it for writing and its so bad. Went back tk ChatGPT for the first time in a long time.
  by nonethewiser
• Do we know that Claude’s quirks are related to watermarking? The article said Gemini’s web console is apparently watermarked, and it has different quirks / a different sound.
  by jdc-pub