Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • This is very cool I'm going to try to carve out some time to try building this into my MOO system ( https://codeberg.org/timbran/moor / https://timbran.org/moor.html ) as alternative command parser front end.
    by cmrdporcupine
  • Thanks, let us know how it goes!
    by HenryNdubuaku
  • Man, I love that there are still people writing new MOO servers in 2026. Any game out there already running on mooR?
    by Balinares
  • Looks like you need to open up access to https://huggingface.co/Cactus-Compute/datasets/needle-tokeni... - I get this error when trying to run the steps in your README:

    > Repository Not Found for url: http s://huggingface.co/api/datasets/Cactus-Compute/needle-tokenizer/revision/main.

    by simonw
  • Fixed now, apologies!
    by HenryNdubuaku
  • Can this be a Siri-like core? Set me a timer, tell me what’s the weather, etc. Here is transcribed text and available list of tools for the model to call, and voice the output.
    by murkt
  • That was the goal!
    by HenryNdubuaku
  • Built a browser version of this runs entirely client-side via onnxruntime-web, no server: https://huggingface.co/spaces/shreyask/needle-playground
    by shreyask
  • Great work, thanks!
    by HenryNdubuaku
  • Lovely to see the push for tiny models.

    I have been building for small (20B or less) models for quite a while. Highly focused/constrained agents, many of them running together in some kind of task orchestration mode to achieve what feels like one "agent".

    I build (privacy first) desktop apps this way and I want to get into mobile apps with similar ideas but tiny models.

    by brainless
  • Give it a go and let us know!
    by HenryNdubuaku
  • Commercial or FOSS? I've been researching the mobile side and it's very exciting!
    by deivid
  • Awesome! I just tried to set an alarm and add some groceries to the shopping list, and it outperformed Siri.
    by tomaskafka
  • Music to our ears!
    by HenryNdubuaku
  • That M versus B is way too subtle. 0.026B is my suggestion
    by kristopolous
  • Haha, we were trying to not be hand-wavy too much :)
    by HenryNdubuaku
  • I was so confused by many comments in this post but thanks to you I realized that some people are apparently reading it as 26B and that's why their comments make no sense.
    by DrammBA
  • The "M" nomenclature has been around since at least BERT and T5/FLAN. It's valid to use it even if today's LLM devs are more familiar with billion-scale models.
    by bigyabai
  • >Experiments at Cactus showed that MLPs can be completely dropped from transformer networks, as long as the model relies on external knowledge source.

    Heh, what a coincidence, just today one of my students presented research results which also confirmed this. He removed MLP from Qwen and the model still could do transformation tasks on input but lost knowledge.

    by kgeist
  • can knowledge then be queried via tool? :)
    by mahmoudimus
  • Sounds very interesting!
    by mlperson
  • Bullseye!
    by HenryNdubuaku
  • That sounds giant! Any unformatted unfiltered preliminary records of said findings?
    by lostmsu
  • > He removed MLP from Qwen and the model still could do transformation tasks on input but lost knowledge.

    But not deterministic?

    by cheekygeeky
  • How does that work? Don't you need knowledge to understand the meaning of the inputs?

    Or is it the difference between, recognizing something vs recalling it being much more difficult? (Classification vs generation?)

    by andai
  • Suggestion: publish a live demo of the "needle playground". It's small enough that it should be pretty cheap to run this on a little VPS somewhere!
    by simonw
  • I'll put this on chonklm.com!
    by bilalba
  • thanks, yeah, the problem is just handling scale, we don't have the infra ready to go, but anyone can do that. Its easy for people to run on their laptops straight up. Will try the VPS route.
    by HenryNdubuaku
  • Should be quick and easy with WebGPU, too.
    by quantumleaper
  • Are you worried about Google's response to this? Google reportedly reacts to distillation attempts "with real-time proactive defenses that can degrade student model performance". So if they detected you, they could have intentionally fed you a dumber but plausible variant of Gemini: https://cloud.google.com/blog/topics/threat-intelligence/dis...

    But also, this model is small and just focusing on the tool use. In terms of token usage, you're probably not anywhere near the people that are trying to distill the entire model.

    by varenc