Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Time to dust off my 128GB strix halo (literally—it’s been dusty, and it’s running a bit warm these days).

    Any idea where this model sits according toquality benchmarks? Pre-bubble MSRP on this hardware was $1400, and it draws 200-ish watts, putting it down into consumer territory.

    I’m wondering if it can replace claude for llm-friendly coding tasks.

    by hedora
  • So back in the Qwen 3.5 release, the 122B-A10B model scored slightly better than the 27B model. I'd expect this new 125B-A6B to perform similarly to the recently released 27B. Qwen3.8 27B is supposed to rival Sonnet/Opus 4.6.
    by cpburns2009
  • This is great. I have a weird system layout (192gb system ram, 8gb vram). the mixture of experts models have been nice when i can run the dense reasoning layers on the gpu (which somehow fit?!) and then the expert on the cpu.

    its worked out to to 40 tokens/seconds on their 80b-a3b model. we'll see how much of a hit this is.

    by dmead
  • Very curious to see how this compares to Deepseek v4 Flash. I have to assume they wouldn't be releasing this if it was worse.
    by Catloafdev
  • Their "next" variants are usually undercooked, but useful for the community to verify support for inference stacks. This will likely be the same.
    by NitpickLawyer
  • Why not? It's not really competing in the same size class.

    Besides, as they explicitly wrote here, the main goal for this release is not performance, rather to serve as a reference for inference runtimes about what to implement. So that later Qwen 4 can be released with zero day support.

    by natrys
  • Alibaba is giving sleepless nights to the tech giants
    by honestlyranked
  • Sounds like a line from a fairy tale
    by WithinReason
  • To be fair, Alibaba IS a tech giant, one of the biggest in fact.

    They are just giving sleepless nights to the western tech giants.

    by drannex
  • I was already rolling around the idea of a 128GB M5 Max MBP. Now this!

    A 4-bit MLX quant with 128k window should fit perfectly, in the 50-70 tok/s range.

    by pwython
  • IDK, prefill speed is a bigger concern for most wokflows, like agent coding, and I heard that this is quite low on macs?
    by irthomasthomas
  • How do you folks code at 40-50 tps? With an extremely lightweight harness (pi) and just 8k system and tools context, and ~40tps on qwen 3.8 27B 4-bit on low thinking mode, it still takes me nearly 30-45 mins for a basic coding session...

    Does it work? yeah... But I'd pick a subscription anyday...

    by rdsubhas
  • I have a 128GB M5 Max, and it sucks at this stage. 50-70 tok/s might be something...
    by sscaryterry
  • Just out of curiosity, why run "local-local" when you could just set up a Mini or Studio at home and query it over http? [edit] whole conversation about this in another thread https://news.ycombinator.com/item?id=49433413

    I’m personally considering retiring my MBP for a Studio + 15" Air whenever this MBP ages out.

    by Eric_WVGG
  • Really happy for those with 128GB+ RAM. Sitting here with my Apple M1 Max with 64GB though. Was looking forward to a Qwen3.8-35B-A3B like many others.
    by bellowsgulch
  • Have you tested Muse Glimmer in low reasoning strength?

    Token generation is slow (and prefill is) but you will likely find it solves actual problems faster than Qwen 3.6 35B-A3B.

    by dofm
  • Really looking forward to this, 27B is a struggle with a strix halo and Laguna 2.1 can do stupid things for tooling calls.
    by syntaxing
  • So, I know https://cactuscompute.com/needle is designed only to enable tool calling on tiny devices. But, I wonder if anyone has used it as a CPU-side mediator between a tool and a GPU-side local LLM making semi-natural-language tool requests...
    by corysama
  • Yeah 27B is way too slow for the Strix Halo. Laguna was better but still slow when I tried it. Qwen3.6 35B is still the best today.
    by cpburns2009
  • > We are releasing these architectural improvements ahead of time so that the community can prepare for the upcoming full family of Qwen4 models.

    That gives me hope that "full family" means it will include smaller models like 4B.

    by big-chungus4
  • What are the use-cases for a model as small as 4B?
    by culi
  • HF link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next
    by fcanesin
  • It will be interesting to see the intersection of this with inference engines like FreeToken which improve distribution of work for MoE models across CPU/RAM and GPU/VRAM.

    If all it takes for a competitive model to run locally at good speeds is a used 3090 and some DDR4, then we might be in for the year of local AI.

    https://github.com/FlashML-org/FreeToken

    by notnullorvoid
  • You can already run it locally its just not the same.

    It is still slow, a lot slower than what you are used to with claude and co.

    And as soon as you increase context size, your memory requirements jump.

    Then when it runs for 30 minutes for something claude needs 5, your device will get hot.

    And even a used 3090 is apparently now between 1-2k.

    by Zylokloto
  • Have you tried FreeToken yourself? I was hoping to find some benchmarks on their github but took a quick pass at their research paper and it seems they're showing ~2x performance on qwen 3.6 35b when compared to llama.cpp - but llama.cpp is so sprawling and has so many options I find that a difficult comparison.
    by kamranjon
  • I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful.

    OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win.

    However, the problem is that many Qwen models have almost no capacity or is so flaky you literally have to just litter your code with a blacklist/whitelist of providers. OpenRouter has some attempts to solve this, but they don't work. In fact, OpenRouter has a lot of really cool stuff that is documented, but if you read the code it's not yet implemented or isn't actually there yet, which is a shame.

    I tried to get in contact with them at OpenRouter about this and I was interested in working with them in the past, but it's difficult to get in touch with the right people and they are growing very fast. I expect being acquired by Stripe will accelerate those problems in some ways. I have no doubt they will resolve all of these issues eventually and scaling that much that quickly is really hard, so kudos to them, but the road has been pretty lame and taken some wind out of my sails.

    by ddtaylor
  • genuine question — how has fireworks or baseten or $reputable_inference_provider worked for your use cases? most production workload probably works fine with one of these and another set as fallback, at least so i think
    by runeblaze
  • Who use openrouter for serious usage? Caching would be really bad unless you use preset to pin to single provider.
    by vietvu
  • I'm confused? Can you just define some presets and call them instead? With preset you can pinpoint a lot of things, especially the providers
    by npn
  • The best solution to this for me is to self host litellm or a different router and use model aliases. For example I have a model called "coding" and when a new good model comes out I just switch the backend without needing to change the alias or the key in my projects (opencode, etc).

    I have a few of them even a smart router called "agents" which will use local models but if it thinks the request might require higher reasoning it's routing to a different model

    by geek_at