Qwen 3.8 27B is excellent, but it defaults to overthinking things

Join the discussion

• Hacker News
• I run mine on an M5 Max with just 48GB of (V)RAM, and it fits nearly twice in Q4. Works perfectly. I'm kinda glad I didn't spend the extra $2400 to get 128. We don't really need more... and that's a good thing (tm). God knows I thought about it in store. But I thought... maybe this year will be the year of the local model? Maybe soon we won't need that much RAM? I was right.

  The fact that it runs at 15tk/s in power saving mode, and 30 in perf. mode blows my mind. I can run the model in the background, coding something for me in OpenCode, hosted in LMStudio, while doing something else. What a world we live in.

  Having something close to human intelligence (at least for reasoning and code), running on a laptop, is amazing.

  by dexterlagan
• It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem.

  For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose:

     User: We're going to make <silly demo>.  Please create a plan, but do not write code yet.
  
     Agent: <short and reasonable plan>
  
     User: Now please follow that plan and write the code.  No other chat.
  
     Agent: <reasonable code in reasonable time>
  

  Maybe this can be fixed with Jinja templates or something, or maybe it's a hack to your harness, but it shows you can get the model to reason reasonably.
  by xscott
• I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought
  by jedbrooke
• I have this branch of llama.cpp that among other things (like patching the template to not break the kv cache, and saving conversations to disk so you can resume quickly days after) also accept the reasoning effort flag here https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev...

  I did testing and the reasoning effort can be set per message, I was not aware of the option of none mentioned by @xscott, I tested but didn't see any change, I think there are just 3 values, xhigh, medium and low as per https://huggingface.co/Qwen/Qwen3.8-27B-FP8 , I did testing and the thing can do it's "I'll speak 10 million words to myself to ensure I'm not missing something" and then switch to a faster model, then switch... I did a test and the thing keep coherence and follow it's train of though-kens, you can see the result here... https://github.com/alainnothere/llama.cpp/blob/disk-cache-ev...

  by xlayn
• To me, the amazing thing is that we now have local models that rival the reasoning of high end models from about a year ago.

  I hope this trend continues.

  by RachelF
• I forked llama.cpp and added some crude mechanism to keep exactly this behavior under control - essentially guiding the reasoning process by injecting text strategically at specific thresholds. This was mainly put together to rein in Qwen3.6-27B, but I'd imagine 3.8 would react similarly.

  Fork can be found here - https://github.com/laurencehardman/llama-mindcontrol/tree/ma...

  Of course hacks like this are not perfect and may degrade performance slightly due to injected text pushing the model slightly out-of-distribution, so the string constants need to be chosen carefully - Qwen3.5's technical whitepaper does provide some guidance in this regard. The mechanism is absolutely more of a hack than a feature, and i'd imagine will be made redundant once llama.cpp supports more appropriate reasoning controls - but for now, i've found it pretty useful.

  by hellajack3d
• All current era models overthink as it's a product of their RL incentives (or distillation of models with them...)

  From my reading of the Fable 5 and Opus 5 System cards, my reconstruction is something like:

  Finish the task → make externally observable evidence that it is finished → check your own work → fix problems → don't stop prematurely → satisfy the evaluator comprehensively.

  That is fantastic for SWE benchmarks and autonomous agents. It also naturally creates pathologies:

  under-answering is expensive; over-answering is cheap.

  by jatora
• “The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.”

  I think that should be the blinking headline - this shows what can be done with consumer hardware.

  by chvid