Birbla

Comments

Hacker News

Damn, I was hoping this was about an actual neutrino.

by pavel_lishin

I was really hoping this would have something to do with neutrinos

by closetothe

Can’t say I’m a fan of containers for this. A big chunk of local LLM gains come (imo) from the open modular nature of llama.cpp and friends. Easy to modify. Easy to experiment.

Containers are the proprietary binary blob in hardware world equivalent

by Havoc

Largely outperformed by Ternary-Bonsai-8B by their own chart, doesn't seem clear what their special sauce is here.

by yborg

There is not a single person mentioned on the website, github created 3 days ago, no real contact, everything hidden. Completely anonymous. Domain owner hidden.

by secult

hey guys, founder here. I am working completely solo right now running on grant funding, so things slipped through on the website quality, it is largely written by AI, I'll deslopify the blogs and be much clearer about methodology. Neutrino-1 is a starting point for Fermion, It's a density experiment on how much capability survives per byte. rn working on really fast and efficient TTS and realtime models to which this architecture is generalizing pretty well.

Sorry for all the confusion from reading through the blogs, should be fixed in a couple of days!

by wincondition

The content on that page is too AI-generated to make sense to me; I don't understand what the model is for.

by moinism

There's a really interesting trend of labs using "proprietary" methods to convert existing models to a compressed ternary format.

PrismML actually targeted the same Qwen 8b model and got it down to 1.75gb here: https://prismml.com/news/ternary-bonsai

I wonder how proprietary it all is though, since the BitNet b1.58 paper has been out for a couple years now: https://arxiv.org/abs/2402.17764

From the wikipedia on 1.58 bit llms: "BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an expensive process, and it would be desirable to be able to somehow convert an existing model to 1.58 bits. In 2024, HuggingFace reported a way to gradually ramp up the 1.58-bit quantization in fine-tuning an existing model down to 1.58 bits."

The section from huggingface is here: https://huggingface.co/blog/1_58_llm_extreme_quantization#fi...

I just wonder how many of these labs are basically following the huggingface recipe here and possibly tweaking it and releasing models without huge training costs.

by kamranjon

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Damn, I was hoping this was about an actual neutrino.
    by pavel_lishin
  • I was really hoping this would have something to do with neutrinos
    by closetothe
  • Can’t say I’m a fan of containers for this. A big chunk of local LLM gains come (imo) from the open modular nature of llama.cpp and friends. Easy to modify. Easy to experiment.

    Containers are the proprietary binary blob in hardware world equivalent

    by Havoc
  • Largely outperformed by Ternary-Bonsai-8B by their own chart, doesn't seem clear what their special sauce is here.
    by yborg
  • There is not a single person mentioned on the website, github created 3 days ago, no real contact, everything hidden. Completely anonymous. Domain owner hidden.
    by secult
  • hey guys, founder here. I am working completely solo right now running on grant funding, so things slipped through on the website quality, it is largely written by AI, I'll deslopify the blogs and be much clearer about methodology. Neutrino-1 is a starting point for Fermion, It's a density experiment on how much capability survives per byte. rn working on really fast and efficient TTS and realtime models to which this architecture is generalizing pretty well.

    Sorry for all the confusion from reading through the blogs, should be fixed in a couple of days!

    by wincondition
  • The content on that page is too AI-generated to make sense to me; I don't understand what the model is for.
    by moinism
  • There's a really interesting trend of labs using "proprietary" methods to convert existing models to a compressed ternary format.

    PrismML actually targeted the same Qwen 8b model and got it down to 1.75gb here: https://prismml.com/news/ternary-bonsai

    I wonder how proprietary it all is though, since the BitNet b1.58 paper has been out for a couple years now: https://arxiv.org/abs/2402.17764

    From the wikipedia on 1.58 bit llms: "BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an expensive process, and it would be desirable to be able to somehow convert an existing model to 1.58 bits. In 2024, HuggingFace reported a way to gradually ramp up the 1.58-bit quantization in fine-tuning an existing model down to 1.58 bits."

    The section from huggingface is here: https://huggingface.co/blog/1_58_llm_extreme_quantization#fi...

    I just wonder how many of these labs are basically following the huggingface recipe here and possibly tweaking it and releasing models without huge training costs.

    by kamranjon