Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- There’s a new announcement every other day wrt models. How do y’all keep track of them all same know what’s decent? Good grief!
And if it’s decent today, it’s shit in eight months! I tool hop as much as the next dev but this is a bit much.
by NetOpWibby - I'm working on solidifying local LLM work flow so I can stop tweaking the nobs and build things that arn't AI related.
There's tons of people caught up in the AI gold rush, but the value of real products for real people is out there. Even more so when you exclude the "I slapped AI onto this"
by cyanydeez - There's no need to, the 50 foot view is simply that many alternatives exist and they mostly fall into 3 meaningful weight classes with comparable performance among each class's members: too expensive to use indiscriminately, too big to run at home, and too small for complex work. As for names and faces in between, the overarching conclusion is that we're rapidly approaching commodity status and those don't really matter muchby badatnames
- I just tune out. It’s not worth knowing, following every development in the field. If something works now it will probably work in 8 months even if it’s no longer the new hype thing. Who cares.
Not using any of it is also a valid option though it doesn’t satisfy your FOMO. But nothing ever will.
by weikju - Unfortunately it crashed out 'no space left on device' while installing the python demo/quickstart.
Only problem was there is plenty of space on the device. PLENTY (not quite 750gb).
- are you pn linux? reproduced the same thing just now its pip dumping torch into /tmp while it installs, and on most distros /tmp is ram bound and capped at half your memory, so it fills up while your actual disk jsut sits there. export TMPDIR=/some/big/path before installing should fix it. adding a proper check next release so it actually tells you which folder ran out.by wincondition
- So I'd like to see a Nemotron3 Ultra converted to a 1.58bit format then have them retrain on the open dataset.by androiddrew
- So... not a new subatomic particle discovery then.by codeduck
- Damn, I was hoping this was about an actual neutrino.by pavel_lishin
- software naming claims another victimby supjeff
- I was really hoping this would have something to do with neutrinosby closetothe
- Can’t say I’m a fan of containers for this. A big chunk of local LLM gains come (imo) from the open modular nature of llama.cpp and friends. Easy to modify. Easy to experiment.
Containers are the proprietary binary blob in hardware world equivalent
by Havoc - What? How are those even related with each other? You can just as easy modify and experiment with llama.cpp in a container as outside of it, they really shouldn't impact one another. Containers don't suddenly make llama.cpp less "open modular" somehow, and I'm not sure how you'd arrive as such conclusion.
- Largely outperformed by Ternary-Bonsai-8B by their own chart, doesn't seem clear what their special sauce is here.by yborg
- There is not a single person mentioned on the website, github created 3 days ago, no real contact, everything hidden. Completely anonymous. Domain owner hidden.by secult
- hey guys, founder here. I am working completely solo right now running on grant funding, so things slipped through on the website quality, it is largely written by AI, I'll deslopify the blogs and be much clearer about methodology. Neutrino-1 is a starting point for Fermion, It's a density experiment on how much capability survives per byte. rn working on really fast and efficient TTS and realtime models to which this architecture is generalizing pretty well.
Sorry for all the confusion from reading through the blogs, should be fixed in a couple of days!
by wincondition - For me, sloppy content is worse than no content. I'd rather have 3 sentences written by a human than a wall of Claudespeak.
Do you have an RSS or a way to subscribe to the TTS work you are doing? Your newsletter button is just a mailto link
by deivid - Lots of respect for a solo founder. But you submitted this yourself, and it’s incomprehensible.by JSR_FDED
- The content on that page is too AI-generated to make sense to me; I don't understand what the model is for.by moinism
- I have the same question.
I get that it’s designed to run on a CPU, big GPU or MacBook (although the way that was phrased confused me at first).
I’m struggling with what a “decoder-only” model is good for.
by naruhodo - AI doesn't want anything, so it doesn't care whether it conveys meaning in its writing. And, apparently the developers of this project also don't care whether it conveys meaning. They just assume we'll wade through the slop? I dunno.by SwellJoe
- There's a really interesting trend of labs using "proprietary" methods to convert existing models to a compressed ternary format.
PrismML actually targeted the same Qwen 8b model and got it down to 1.75gb here: https://prismml.com/news/ternary-bonsai
I wonder how proprietary it all is though, since the BitNet b1.58 paper has been out for a couple years now: https://arxiv.org/abs/2402.17764
From the wikipedia on 1.58 bit llms: "BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an expensive process, and it would be desirable to be able to somehow convert an existing model to 1.58 bits. In 2024, HuggingFace reported a way to gradually ramp up the 1.58-bit quantization in fine-tuning an existing model down to 1.58 bits."
The section from huggingface is here: https://huggingface.co/blog/1_58_llm_extreme_quantization#fi...
I just wonder how many of these labs are basically following the huggingface recipe here and possibly tweaking it and releasing models without huge training costs.
by kamranjon - Independent testing of prismml suggest quite a capability drop off outside of their cherry picked benchmarks. I'll be curious to see what this model achieves though.by Twirrim