Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Still needs 32-64GB memory to run it locally. 64GB Macbook pro with an M5 chip costs more than 4k Euros in Germany. A more practical model would be a language specific (e.g Python or JVM language) and excellent at tool calling and reasoning. Maybe that way they can shrink it even more.
- Even if you had a 64GB machine: Are you willing to reserve 90% of your memory to run a LLM? With dirt cheap models like deepseek-v4-flash that will run "forever" on $10, the answer for me is clearly: no.by sparkling
- Well if you're spending thousands on API tokens already, you could just drop the same amount on a 128GB MacBook Pro and that's a one time cost.by dbbk
- There have been discussions on language specific not really being a relevant change to reduce size.by Gecko4072
- Coming from the PC games industry in the 90s and early 2000s, it was a struggle to run some of the games on release. 90%* of people wouldn't be able to play the AAA games on release (think Crysis, etc). This period of local LLMs reminds me of that time, whereby the hardware just isn't there yet. Give it time, and the prices will drop.
* total guess
by Flere-Imsaho - I'm sooo happy I pulled the trigger on upgrading and getting a new laptop (with 64 GB RAM) last summer. Feels like it was just in time before the exponential price jumps.by mihaelm
- I am running it on a single RTX 3090 (24GB VRAM).
Some folks on Reddit are having the same experience: https://www.reddit.com/r/LocalLLaMA/comments/1vkm42m/muse_gl...
It uses an order of magnitude less VRAM at longer contexts which is a huge advantage over Qwen 3.6 27B
- I think if there's going to be advantages to making smaller, more targeted models, those advantages will probably come from targeting specific domains, not from targeting specific languages.
I think that if an LLM can't abstract over the differences between Python and C++, it probably will have an even harder time abstracting over the differences between writing code that manages a webserver, and writing code that does aerodynamic simulations.
by eigenspace - Practically ~20GB with KV cache
> We quantize weights to ~4-bit, bringing the LM under 20 GB. We validated minimal to no degradation on agentic tasks under compression.
https://www.reddit.com/r/LocalLLaMA/comments/1vkgsum/introdu...
by karimf - What I think would be perfect is a model that could run on a single DGX spark and be competitive with DSV4 Flash 731. Flash is already a game changer. Hopefully meta plans on this, like the old 70b. V4 flash is smart enough for any use but slightly too big. 27b-30b isn’t intelligent enough.by Gecko4072
- DSV4 Flash 0731 already runs on RTX 4090 24GB + 128GB system RAM at a usable tok/s and quantization.by 127
- This model I think will be too slow for that on Spark, even at 4 bit quant.
It's a dense model, not MoE like e.g. Qwen 35b or Gemma 4 26B A4B. On a Spark it will be memory bandwidth limited
I haven't tried yet (working on it) but back of the napkin estimate puts it at around 15tok/s even after converting to NVFP4. Prefill would be much higher though. That 15tok/sec is pretty typical for dense models of this size:
NVFP4 Q/K/V/O and MLP projections: ~13 GB/token
BF16 attention gates: ~3 GB/token
BF16 LM head: ~2.5 GB/token
Total: ~18.9 GB/token
At 273 GB/s, that gives a bandwidth-only ceiling of about 14.5 tok/s; actual performance would be lower.
- > Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation.
The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, notifications, and newsfeeds and is constantly preparing things for you.
by jawiggins - Maybe it's my lack of imagination, but what do you imagine you'd be doing where you'd want to keep a computer busy overnight?
It seems like the purpose of humans isn't to keep machines busy. When our phone or laptop is idle, it's fine if it sleeps. And when we do want something, we'd rather not wait.
(Also, this new model seems to be designed to keep latency down, which is useful for interactive tasks.)
by skybrian - Help convince Firefox of this: https://news.ycombinator.com/item?id=46294238 Rather than develop its own AI, Firefox should develop a system to pipe your html rendered browsing history in real time so external local services can process it: https://connect.mozilla.org/t5/ideas/archive-your-browser-hi.... Firefox could be the only browser that does this.by espeed
- This is is already possible with Claude Code. I use a setup where I have one instance monitoring a local queue, I have a web app for receiving webhooks from various sources and pushing them to the queue. Plus email for things that don't have webhooks. That instance then decides what to do with each input, sometimes it can spawn additional agent to investigate/prepare, sometimes it creates a ticket assigned to me and then waits for me input. All of that just uses the monitoring tools built into CC. The dispatcher loop doesn't need to be extremely smart, so I might experiment replacing it with a local model like this.
- I've been building this for the last 6 months or so. I've basically got it working. The model is not the issue, the infra is. Keeping everything in context just isn't possible and LLMs, even Fable, don't mode switch well. To get around this I've built a database software that ingests as much digital information as possible, and annotates it, then creates timelines with resolution gradients (longer ago = less resolution) that it feeds to the LLM on every request.
Then you have your cheap little MoE or ternary model just running in a loop, with an escalation pathway before it reaches the big expensive models.
Currently it's doing things like reminding me to take allergy medication when I wake up because it's checked AQI or whatever, reminding me to stop at the market when I'm on my way to pick up the kids to get the cherry tomatoes I forgot, giving me heads up of what folks are expecting from me in certain meetings based on cross correlating email and calendar, etc.
It's honestly the single most productive tool I've found for my ADHD.
by colingauvin - It is interesting but it does look like a careful distillation of (Spark and) biggers open-weight models.
The progress compared to Qwen3.6 27B is good, not that impressive, it's a 4 months old model. (kuto to them to compare to 27B dense and not 35B MoE, it's more fair to do so). It is very probable that Qwen3.8 27B will crush Glimmer-30B on most benchmarks.
by _ache_ - I'm so excited about these two new models. Qwen 3.6 27B has been my sweet spot so I cannot wait to try 3.8. Glimmer looks really strong, I'm encouraged that Meta compared it to 3.6 in the model card! Exciting times!by pettijohn
- Still great if they want to play in this space. Having competition for the 24-32GB VRAM target is only good for the end user.by skohan
- The gguf is up and works, I don’t know if it’s them or unsloth that’s facilitated this but it’s nice because e.g. Inkling still doesn’t appear to have support in llama.cpp which makes it irrelevant to a class of user.
Unfortunately I don’t have enough experience with Qwen 27B to immediately compare, but I do it’s Qwen 3.6 35B A3. It’s much slower obviously but it seems to be way more efficient with its thinking to the point that using it might actually be faster. I find Qwen and some others rehash the same things over and over when thinking without getting anywhere, in mg limited checks here Muse is much better.
by andy99 - There's an inkling branch, go to unsloth, read - https://unsloth.ai/docs/models/inklingby segmondy
- I don't really use the Qwen 3.6 27B though I do test the variants (Bonsai, ThinkingCap).
I really like the 3.6 35B A3B for experiments, and it seems OK, but as you say, it spins round in thinking loops more than say the 26B Gemma 4 does. If Muse doesn't actually-wait itself as much it will be very interesting.
I am just downloading it to run my small tests.
by dofm - With the business model for API based LLMs looking iffy at best it seems like we’re heading back to the “server under your desk” era of IT again.by cmiles8
- more likely that hosting and delivering the models will be commoditized, much like how DO, Linode, Hertzer etc all commoditized VPSs and server hosting. And you'll end up paying for virtual hardware size (or compute resources) rather than tokensby samtp
- This release is not a meaningful improvement in any metric over 5 months old Qwen 3.6.
DS v4 Flash update maybe, but it is too big for typical Joe's desktop.
by lostmsu - That’s part of it. There’s also just a natural back-and-forth between what I call “time sharing” and “personal.” When the thing you want is expensive, you share it remotely, but as soon as costs fall, everyone wants it under their desk.by drob518
- I wouldn't say "server under your desk", necessarily; more of an "Linux getting big" era of IT.
If you want to host the model on the server under your desk, you can. If you want to build a data center on-prem to host it, you can. If you want to pay a cloud provider to host it at their data center until you figure out how to scale it without their help, you can. It's like when people were first building commercial services to support Linux-based OSes, and people were also still hacking on it on local machines.
APIs may still have their place - maybe you just want to throw your devs a known quantity with all of the management built in - but it's not going to make Sam Altman a trillionaire, which is something anyone outside of the SV echo chamber could have figured out as soon as the first real competition to OpenAI emerged.
by lenerdenator - Why do you say API based llms looking iffy at best? Do you just mean current profitability due to market pressures from some companies’ subsidized investor money?
Surely, even if you’re just using open weights models, it should theoretically be cheaper to use them in a highly optimized cloud architecture(even with vendor markups) rather than each person serving their own models from much less efficient (and more importantly, much less consistent volume) self-owned “server under your desk”?
by gdhkgdhkvff - I've been coding using the LLM server in my living room for the past few weeks, and I haven't had this much fun with tech for agesby skohan