Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Really cool work, you get the actual traces. Looks like the vendors can all reliably fix this one though.
A harder to defend against approach here where they work backwards from the results and ask the model to generate a plausible trace: How to Steal Reasoning Without Reasoning Traces https://arxiv.org/pdf/2603.07267
by nervai - Trace Inversion is fascinating, but it’s more of an independent reconstruction that will give you some coherent-looking generated CoT; but not necessarily anywhere close or related to the underlying model’s CoT.by dannyw
- I did this with Codex's recent encryption of compaction.
Interestingly, I didn't have to drop to a dumber model, just a 2 sentence <developer> prompt auto-injected before and after compaction made all their models output the encrypted compaction data in plaintext.
The result was... interesting. There's nothing unique in there and I still don't understand why they decided to encrypt it in the first place.
by glub - Possibly something to do with other providers using the it to train their own models?by chrisss395
- Super cool that this works. I'm surprised these companies re-use the same encryption key across models!
I wonder if you can use these for attacks, like this previous paper showing that if you know how a model reasons, you can "fake its thinking" to control it? https://news.ycombinator.com/item?id=48631888
by x312 - Seriously, what does it take to encrypt per session? There are many ways to make it scalable and efficient so I am wondering if this is left like this to allow interested 3rd parties ahem unobtrusively peek what people are doing with the AI.
(Thanks for the link. That’s an interesting idea!)
by yubblegum - > I'm surprised these companies re-use the same encryption key across models
I assume switching the model in the middle of a conversation is intended behavior (very useful in coding agents, for example)
by flexagoon - Is this how the eastern labs "distill" SOTA models?
If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT.
But the real question is: Is it okay to steal from a thief's hoard?
- The reasoning blocks are not stolen/mined from the internet at large directly. They’re the result of a lot of research, time, money, and expertise into creating a reasoning model. To me the answer is quite clearly no, especially when the encrypted blocks demonstrate they want to protect it.by azinman2
- Not necessarily. There's a million ways to jailbreak any current model to show the trace and bypass all guardrails, or hijack and modify it. It's just one of them.
- >But the real question is: Is it okay to steal from a thief's hoard?
How does this relate to your previous paragraphs? LLM outputs are not copyrightable and you didn't break into Anthropic servers to steal the files from there. So how exactly is it theft? If I send an "encrypted" files to thousands of peoples and some manage to figure out how to read it I can't really accuse them of that or can I?
by qwytw - > But the real question is: Is it okay to steal
By definition it cannot be stealing since you're paying for the tokens. It may be against their ToS, depending on what you end up doing with those tokens, but it cannot be stealing. If they charge by the token, all your tokens are belong to you :)
I also find it very strange that everyone sort of accepts their ToS like no big deal. Imagine MS using the same terms for their software - you cannot use any MS software to develop competing services. Bananas! They'd be dragged through the courts like it's the 90s.
(I get why they're doing it. Distillation is unreasonably effective. But still, I find it bananas that we've kinda accepted it, to the point where people use "stealing" or "attack" or any such terms)
- "Stealing" something you already paid for (tokens), but that you can't have access to(!). And trained on the sum of human knowledge.
Training on other model outputs ought to be business as usual, stop using morally charged terms made up by future monopolists: https://thomasdullien.github.io/posts/2026-06-15-rl-economic...
by Aissen - > stop using morally charged terms made up by future monopolists
Lets not gloss over this claim. Being: “Stealing is a morally charged term made up by future monopolists.”
I strongly disagree. Stealing is not a made up term and property rights are foundational for any society. Your take is at least sensationalist if not malicious.
by nonethewiser - No, you paid for the end result. The thought process is a step in between, a function. Think about it, who should get charged if the answer you received comes from a cache? Thinking tokens are the complexity-of-the-problem cost. I mean, you may not agree but both are valid points of view.
- Liberating!
- The only person calling it stealing is the author of this article, so this is a pointless discussion. The majority of this thread is just arguing with themselves.by paxys
- Suppose you hired a consulting firm to write a report, and they delivered the report but not the internal conversations they had when developing it. You exploit a vulnerability in their phone system to get those conversations. You can argue over semantics of whether “theft” is what you did, maybe the right word is “espionage” or “spying”, but that either way we probably agree you are guilty of something? Paying for the final product didn’t entitle you to see how it was made, unless that was part of the agreement.
Anyway, you can distinguish this from the debate over copyright.
by ashdksnndck - Apparently you can do the same by simply running it without reasoning, while giving it a thinking tool...
>guys you do know you can just disable thinking, and instead give it a "deep_think" tool, and it will call it with internal CoT reasoning format right?
>gl fixing that
by Pragmata - its not exactly the same... its tool use spec asking to put thinking in inputs fields... it is a good idea but its not same.by retinaros
- I've been doing that since before reasoning was a thing baked into the models, it always performs better this way. Except for some providers/models where you just can't easily turn it off, now I just avoid them. This way I save tokens and have full control of the reasoning.by ashirviskas
- The model is finetuned to enter/leave its thinking mode using special token separators. there's no reason to assume the tool calls induce the same token distribution or produce the model's actual native reasoning traceby klntsky
- "Recovery" would be a more apt (although less catchy name). The stealing is on the provider side for not giving you access to tokens you already paid for.by sly010
- "Stealing" is a strong word to use for looking at the words produced by models built from the collective commons of the world.
And, honestly, being able to see how LLMs make decisions is critical to trust and security. I consider it a valuable feature, somewhat akin to seeing the source of software I use.
by SwellJoe - Yeah, it’s also useful for prompt tuning, debugging and understanding how a model interprets your prompt.
Also really good for identifying any contradictions in your system prompt and context.
by dannyw - You cannot steal what is not owned.
At least in the EU there is no copyright for LLM outputs, so I guess all they might do is violate the terms of service.
by niemandhier - While I feel it is morally OK for users to access the metadata of their own exchanges, there's something awry with that formulation, (not-)"stealing" is about more than just copyright or ToS.
For example, consider my browser cookies that authenticate me to HN right now. Nobody even wants to copyright them, but if you were to somehow acquire a copy I'd very much consider it "stealing."
by Terr_ - they also can't claim to be the owner of the output either because they can't claim to have had a valid license for the inputs.by cyanydeez
- If I hire you for a consulting project, do I own the screen recording of your computer while you completed it?by levocardia
- Even copyrighted information can never be "stolen". It can only copied without authorization.by Zambyte