Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Seems like an awful idea. I hope that that "watermark" will soon be discovered, reverse-engineered, and that tools to remove it will appear.by drnick1
- I hope all models adopt it.by cassianoleal
- I cannot wait for the inevitable "I've always used Claude watermarks in my writing, even before we had LLMs!" when someone gets caught using an LLM.by bramhaag
- Can someone help me understand how exactly this watermarking of text works?
Given that text is, well, text, and not some kind of binary format, I don't see how any watermarking can work unless you insert characters which are invalid under Unicode. I further don't really understand how this won't be perceivable by assistive technology (the "watermark" will just appear as either unreadable characters, or if the watermark is mixed thoroughly enough into the text, it will scramble the text to any speech synthesizer and will make it really really obvious). Thus, I don't see how this wouldn't be insanely trivial to remove. And this is before we get into things being put on the clipboard. Sure, I can press the "Copy" button at the end of each response, but what I can also do is manually select the response and copy it, or only copy partial selections, or any number of other things. How does this "watermark" (or any "watermark" technology) take into account this?
So, really, to summarize this: I see no way of this actually being technologically achievable unless we revise the very core of how computers work and encodings for textual information. So I'm very curious as to how this is actually supposed to work.
by ethin - it's a statistical way. like for example maybe in your above paragraph claude maybe writes "Thus, I don't see how this wouldn't be insanely <easy>(instead of trivial) to remove" and then also says like "And this is before we <analyze> things being put on the clipboard." or maybe the i just says the word "the" in a certain pattern or frequency.
you can then consistently like figure out if it was claude that wrote the sentence. it is easy as you noted if you just get another ai to read it and then rewrite it.
- It's important to distinguish "watermark" from "signature" (and the text/binary distinction is not relevant here):
Signatures are extra data, added out-of-band to the existing data. Out of band data, by definition, is easily detected and stripped, so the utility of a signature is that authoring one requires secret knowledge. Philosophically, the presence of a signature is a kind of authentication, a desirable thing that is hard to grant and easy to revoke (the smallest change to the data renders it invalid).
Now, watermarks: if you flip it round and say you want to glue on a piece of undesirable data - something that represents disauthentication, like a cursed black spot of written-by-LLM - then you want it to resist removal efforts. And now right away you have a hard problem because your sticky data must be in band, or else it is trivially stripped. Not only that, in fact, it has to look enough like real signal that it isn't easily filtered. And on top of that, you can't distort the real signal too much, or people will complain. So you're cornered into doing a kind of steganography - hiding small amounts of information in the entropy, biasing the signal in perceptually plausible ways that are detectable to those in the know. Cartographers add fake streets ("trap streets") to catch plaigiarists - for LLMs, watermarking might take the form of subtly odd word choices.
by dTal - Have a look around for token biasing, or green lists. It's based on a nudge to the choice of the next token (which can always be drawn from a set of possibilities which are all probable enough).
At first I thought this approach was just the "LLM flavour" of writing, but it's way more subtle, especially as the bias is applied uniquely for each token position.
by benrow - An interesting factor of this is competition.
If Claude was the only model family they could ship a change like this and users who want to cheat (or don't like watermarks for other reasons) would just have to put up with it.
In a world with many different competing models, the risk of losing customers to other providers over this is much more real.
Maybe they've looked at the numbers and the portion of people who clearly use Claude to cheat on examples etc is so tiny that losing them to other providers isn't a problem?
by simonw - Presumably the expected cost of doing it is less than the cost of getting fined by the EU for not doing it.by andai
- Either that or they want to comply with the EU AI Act when it affects them.by nprateem
- I’m more worried that this will degrade performance. I want the best results from a model, not the results that fit a constraint that’s not defined by me. Any increased cost or latency is also unacceptable.by slowin
- Scott Aaronson spoke about this in a colloquium where he said that this was mooted at OpenAI before the decision was made by Altman to not implement it for the reasons you describe.by lhd1
- There are already small models trained specifically to prevent statistical detection, e.g., https://huggingface.co/kalpeshk2011/dipper-paraphraser-xxl
I guess whoever is the policy maker is assuming that some protection is better than none and that most people will not reach for such tools.
by gajus - I have had a hunch for a while now that (in addition to these tools), Anthropic has actually leaned in to Claude's distinctive manner of writing since it makes the text more obviously AI generated and thus less susceptible to misuse.
That's not necessarily the same thing as a markov-style fingerprint but it could be a correlated factor.
by aabhay - It's pretty trivial to command it to not speak that way. That's one of the first things you should write into the prompt. What style you want it to write in. Make it use a very concise and dry academic style with no overt LLMisms, melodramatic or flowery language, or metacommentary.by noman-land
- I had a similar thought but I assumed they leaned in because it improved performance on coding or something like thatby pjm331
- We need to just stop pretending we can reliably tell if plain text is written by an LLM.
It’s just not a reasonable ask.
by jonplackett - True, but what you can do is a one-sided guarantee. If it bears the mark, it is likely generated (or someone deliberately made it look generated).
Thus, if a news article, research article, book, student paper submission, blog post, HN comment, etc, bears the mark, it could be automatically flagged as such.
It helps detect low effort slop.
---
Caveat. If you write your own creative work and send it to Claude for "cleaning up grammar", it might insert the watermark.
by JohnKemeny - An approach like C2PA is the only realistic path forward. If the trajectory we're on continues, it's probably safe to assume nearly all content will be AI generated. We need realistic ways to prove content is human generated, and without true authentication (someone willing to corroborate they created the content, and they can certify it), the whole endeavor is pointless. While private human-verifiable content will still cease to exist, at least in this way we can avoid moving into an information dark-ageby cush
- Now that the EU mandated watermarking, the point is that services (or browser extension developers) can add their own detectors to make AI-generated text obvious. It won't fix AI in print, but most of the problem is online anyway.by nunez
- All big LLMs already visibly watermark all their text with easy to detect annoying phrases and turns of speech that everyone is already sick of hearing. Why do AI companies keep making their products worse to appease anti-AI, it’s not like they’ll suddenly start supporting it if you do so. If you’re worried about European customers, just relax your firewalls to let more VPNs through, if the productivity boost is high they will use it anyway if their rules keep crippling their own modelsby mhjkl
- Is it to appease anti ai or is it a method they will use to avoid training on their own slop?by WD-42
- > if the productivity boost is high they will use it anyway if their rules keep crippling their own models
Individuals maybe, companies won't and that's where most of money is at.
by IsTom - Many good comments here. It’s somewhat common for me to voice record say a blog post of product updates, more like a ramble. Then have Claude clean it up. Then argue back and forth about certain things until it’s good, then make a final pass sometimes to change a few key words. This is incredibly different than pure ai text. Presumably it will show as ai generated here, even though I would argue it is not really. So I can’t use Claude for this usecase anymore.
I think the solution is assume everything is ai generated unless told otherwise and rely on authorship/brand as a sign of quality.
by mchusma - Perhaps. But it seems like your beef is not with the presence of watermarking, it's with what people will use that watermarking for. You're not directly harmed by that blog post being labeled as AI generated. In a hypothetical (but unfortunately likely) world where everything has passed through an AI's digestive system, nobody would care.
In the meantime, it is true that this takes something away from you. But it's something you were only recently given. Now you're not given quite as much, but readers are given a little more (or rather, there's less being taken from us!)
> This is incredibly different than pure ai text.
Ok. But it's still incredibly different from pure human text. I guess the question is which provides more value? Providing the information "this text is AI watermarked" to readers? Or allowing creators to lie and claim that AI processed text was 100% human generated? I agree that people assuming that "has AI watermark" == "is AI slop" is incorrect and causes some amount of harm, but having the watermarks also pushes back on a large amount of harm already being done.
(Personally, I'm skeptical that these watermarks will ever hold up to adversarial attacks, and they haven't claimed that they will. So I think it's the usual "casual liars will be caught, determined liars will get an additional thin veneer of respectability".)
by sfink - You can do something like "here's my website. Read it, then try your best to write in my voice. Do your best to avoid common and uncommon tell-tale AI-isms"
If you're putting the work you say in, the result won't be obviously distinguishable. Obviously, from some of the things that get posted here, that last sentence is too much for most people to bother adding to their prompt.
- I've done this before at work, and I feel true ownership of the output after this workflow. Moreso than when someone from a marketing team publishes a blog with the CEO's name as the author.by MattSayar
- Simply write your own posts if you don't want people to think they are AI generated.
- IDK. At my job it seems like it's expected you'll use it in that way.
In academia they've got their own concerns of 'purity,' (not least of which is justifying their continued existence which is in my opinion hard to do) and they are the ones who are going to want most strongly to punish anyone who uses AI.
And perhaps "journalists," who will want to trumpet the latest government's press release being [what they'll portray as] mostly AI-generated, as a headline-grabbing "gotcha." Ironically, that may even be a story that'll be written by a fully-autonomous journalist "agent" in a newsroom that's been pruned of all human journalists!
But in business it seems to me that we're all agreeing that it's a "good" use of AI to write in that way.
by xp84 - But it is not incredibley different than pure ai text tho, it is literally the same as pure ai text.
I understand you're saying since you "worked with it", it is not ai generated but if you still use the final output verbatim, the writing itself is LLM generated purely.
You want to share the output by it but also position it as not ai output. But that's fundamentally dishonest.
Furthermore, if you think your approach actually creates value and can be judged on its merit, why not disclose its ai written? If you think that will make people think your content is bad then you should see that as feedback and maybe not use AI since readers don't like it.
by altmanaltman