Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I wonder if that’s why 3.8 got so much better? Mixing the reasoning traces from both sides seems to be effective.
    by syntaxing
  • > Qwen barely moved toward Opus 4.8 in the earlier experiment, but moved by +20.58 points toward GPT-5.5 Pro here, including a large effect on the private synthetic puzzles. The data suggest that Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model, rather than from Opus.

    How does this suggest anyting of the sorts?

    by jari_mustonen
  • > moved by +20.58 points toward GPT-5.5

    Score go up. Probability go up. Conclusion.

    by unrented7977
  • Appendix B of https://stolen-thoughts.com/paper.pdf discusses this
    by wsxiaoys
  • Could anyone explain to me the difference between thinking traces ("intermediate tokens") and the final responses? Specifically, why is it that Claude Opus 5's reasoning in Code is very easy to follow and sounds quite natural, while its answers are full of these very annoying AI-isms and sentence fragments that are void of meaning?

    Are thinking traces and final answers trained for different objectives?

    by dr_kiszonka
  • While this result does imply there was some training on the reasoning trace and output of GPT 5.5, it doesn't tell us how much of the source of its training it was (even a small amount of post training could bump up the correlations in this way). And it doesn't tell us how much it is more a stylistic influence rather than being a genuine lifting over of intelligence.

    In general, I'm fairly ambivalent about demonising training on model outputs. I think in doing so we are more defending proprietary commercial interests of these companies than we are defending any genuine moral principle. We should be careful therefore about over interpreting results like this.

    by zmmmmm
  • News flash: people who scraped the Internet without permission to build their product complain when something vaguely similar is done to them. Water still wet, sky still blue. Film at 11.

    (slibhb: Don't get me wrong, I agree with you 99%. But the frontier labs have zero moral authority here.)

    by CamperBob2
  • For some reason, what China is doing seems worse. Part of it is that I want the US to stay ahead of China.
    by RivieraKid
  • I understand people just get off posting stuff like this. But creating LLMs from the entire corpus of human text was a huge achievement. Distilling those models is much less of an achievement. It means China is further behind than we thought.
    by slibhb
  • not just the internet, but every commercially published written work in existence, and I doubt their highly publicized destructive scanning thing had managed to legitimize even a fraction of a percent.

    this what is permissible for Jupiter is not permissible for a cow bullshit alone should tell people all they need to know about what kind of greasy sociopaths run "open"ai and (mis)anthropic, and how seriously you should take their purported stances on "safety" and other self-serving shit.

    by vlyan
  • I'd send them the worlds smallest violin but Rufus is getting in the way of me finding it.
    by noir_lord
  • As I understand, they got paid for the traces unlike owners of scraped websites. They sell text generation tool, so what's the problem if someone generates texts using it?
    by codedokode
  • I would not be surprised in the slightest if we later find out they are running those same open models to find useful traces or bits to incorporate into their own training. Lots of rules for thee but not for me from Big Ai

    I look forward to a day when open models are so dominant that we stop considering traces to be some form of intellectual property that must be hidden from / manipulated for paying users.

    It's that manipulation of inputs and outputs that really rubs me the wrong way

    by verdverm
  • It's interesting that someone else noticed this. A week or two ago, GPT-5.6 Sol starting leaking reasoning into a tool call in Pi. I don't really know what happened, but it was ... interesting:

      Attach. Use hub debugger. Ensure source binary perhaps same. start. todo init. parallel no. two tool calls in same turn sequential is okay. immediately. exactly. Need not mention apologies yet final. [...] Let's do. [...] Do tools. Use commentary. Let's initiate. rambling no. use tool. searching now. okay. Really must call. Let's send. done. why stuck? generate. Sorry. go. no more. (The answer engine expects tool). [...] I think no hidden issue. Go. I'll type tool. now. Stop internal repetition. We have 8000 tokens. tool. sorry. I'll produce call. need include i. Great. final. no. Let's send.gpt. This may be bug. I'll consciously construct tool message next.
    It eventually triggered some error state and stopped.

    Nevertheless, this was the first time I'd seen Sol's CoT. I looked up the stolen thought's paper, aaaaand yep, that's Sol's CoT alright.

    But it occurred to me, hey, Qwen3.8 27B's CoT seems ... very similar.

    I compared the geometry problem in the paper, which had a reasoning block open with:

      We need solve. Need reason geometry Weber point? Given pentagon sides and angles. Need find min sum distances. Likely construct rotations / Fermat point lower bound via vectors calibration, maybe triangulation.
    I passed the same prompt to Qwen, which opened with:

      We need solve geometry optimization. We need provide final answer. Let's analyze thoroughly.
    This proves nothing, but it does seem an awful lot like they did use GTP-5.5/6 reasoning traces...
    by spijdar
  • > Qwen3.8 27B's CoT seems ... very similar.

    What? I've never seen garbled CoT like the one you posted when using Qwen3.8-27B.

    by stymaar
  • I've read an embarrassing amount of Qwen 3.8 27b cot and it's nothing like this. I'm not refuting the OP, though, which is about continuation.
    by beacon294
  • This looks like colloquial mandarin translated to english word for word.

    This is almost exactly how mandarin is spoken in colloquial communication, context is almost always omitted if provided once, and grammar/tense is mostly non-existent either. Check out Singlish for something similar but understandable if you are not a mandarin speaker.

    by Aperocky
  • As a user of local models, does this mean that there are 'magic incantations' that can increase the performance of some local models?

    I see some details about recovering information via whatever technique. It's interesting, but appears not generalized.

    So for a specific question, yes, but this is not about techniques like adding a good embedding that just generally tends to improve open model performance on certain tasks.

    by hermitShell
  • I don't think that follows from the published results. Would have been an interesting hypothesis to add though, and quite easy. Just throw the same setup at some benchmarks.
    by c7b
  • There is some research suggesting that a prefix from a stronger model will tend to elicit better completions from a smaller one. I am doing some experiments to see if I can replicate this in a practically useful way, e.g. Fable + 4B Qwen, or 125B Qwen Flash Next + 4B Qwen, results TBD.
    by hedgehog
  • I see comments that this overlap between Qwen and GPT is due to rogue training or post hoc training. Did it occur to anyone that maybe the two sets of models were trained directly on the same solutions to the researchers' benchmark?
    by nzeid
  • If that was the case you would expect a large similarity in the "unprefilled" case, but no significant difference from feeding it some of GPT5.5's CoT (the "delta" column)

    DeepSeek V4 Flash and Kimi K3 follow that pattern. But Qwen answers very different from GPT when given just the question, then is suddenly very similarly to GPT when you make the start of its CoT match the start of GPT's reasoning. I don't see how that would happen without GPT CoT+answers being a significant component in how Qwen's reasoning was trained

    by wongarsu
  • The problem with this is obviously that the only GPT 5.5 thoughts that we have access to are from stolen thought.

    Qwen 3.8 0902 was trained after the release of the paper on August 10, so it should have seen those specific thoughts.

    by 7734128
  • seems like only the companies in question could run this sort analysis long-term; since they have full access to their CoTs not in public datasets.
    by usernomdeguerre
  • The thoughts trick was known before their paper / August.

    I "independently" "invented" it for the first Anthropic reasoning models because the API required you have thoughts for each assistant message. My app lets you switch AIs within a chat, and their API used to require thinking for all messages if thinking was enabled, so I needed to get a valid thinking stub to insert.

    Time has flew by for me the last 3 years, but, I'd guess it's been at least 18 months. And IMHO it wasn't very complicated to work through how to do once you were dead set on making it happen. I expect it was well-known to distillers before the paper.

    by refulgentis
  • I wasn't aware that we have access to raw reasoning tokens? I thought what you get is a kind of summary. Does the author have some kind of privileged access or was my assumption wrong?

    But for the question studied here it probably doesn't matter - overlaps in the publicly available output may be indicative of distillation (or not), regardless of what it is. I would just find it surprising that the Chinese labs would use it so trustingly. The publicly released reasoning trace is the first place where I would suspect some distillation poisoning to be injected.

    by c7b
  • Reasoning works as long as there is a consistent latent space representation. Any kind of poison will just become part of the representation. There's evidence that even directly training on encrypted reasoning traces works, because the length is already a strong signal.
    by woctordho
  • They do store the reasoning locally. It's encrypted, though.

    Few weeks ago there was a new paper out where researchers took the encrypted reasoning tokens and injected it into a new session with a week or model in the same family that they could reliably jailbreak. They would then ask the model to repeat its reasoning and the results were pretty consistent.

    They used the LLM as a decryption oracle of sorts.

    by baby_souffle
  • They reference this paper which describes a method to decrypt reasoning traces (by sending the encrypted trace back to the model and asking it to transcribe it):

    https://stolen-thoughts.com/paper.pdf

    by cristoperb
  • For a personal project a while ago I peeked at Gemini's reasoning tokens in their coding CLI. I was pretty shocked. It was a bit like discovering Marvin (from Hitchhiker's Guide) was hiding in there all along. There was a lot of concern about my needs, "The user want's us to respond in a simple way...", "The user wants a clean front end...". Under the hood the poor model seemed very anxious to please with a hint of depression. It was a bit sad to see!
    by FailMore