Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Lovely to see the push for tiny models.

    I have been building for small (20B or less) models for quite a while. Highly focused/constrained agents, many of them running together in some kind of task orchestration mode to achieve what feels like one "agent".

    I build (privacy first) desktop apps this way and I want to get into mobile apps with similar ideas but tiny models.

    by brainless
  • Awesome! I just tried to set an alarm and add some groceries to the shopping list, and it outperformed Siri.
    by tomaskafka
  • That M versus B is way too subtle. 0.026B is my suggestion
    by kristopolous
  • >Experiments at Cactus showed that MLPs can be completely dropped from transformer networks, as long as the model relies on external knowledge source.

    Heh, what a coincidence, just today one of my students presented research results which also confirmed this. He removed MLP from Qwen and the model still could do transformation tasks on input but lost knowledge.

    by kgeist
  • Suggestion: publish a live demo of the "needle playground". It's small enough that it should be pretty cheap to run this on a little VPS somewhere!
    by simonw
  • Are you worried about Google's response to this? Google reportedly reacts to distillation attempts "with real-time proactive defenses that can degrade student model performance". So if they detected you, they could have intentionally fed you a dumber but plausible variant of Gemini: https://cloud.google.com/blog/topics/threat-intelligence/dis...

    But also, this model is small and just focusing on the tool use. In terms of token usage, you're probably not anywhere near the people that are trying to distill the entire model.

    by varenc
  • Hmm.. this might make it feasible to build something like a command line program where you can optionally just specify the arguments in natural language. Although I know people will object to including an extra 14 MB and the computation for "parsing" and it could be pretty bad if everyone started doing that.

    But it's really interesting to me that that may be possible now. You can include a fine-tuned model that understands how to use your program.

    E.g. `> toolcli what can you do` runs `toolcli --help summary`, `toolcli add tom to teamfutz group` = `toolcli --gadd teamfutz tom`

    by ilaksh
  • Do you have any examples or data on the discriminatory power of the model for tool use?

    The examples are things like "What is the weather in San Francisco", where you are only passed a tool like

      tools='[{"name":"get_weather","parameters":{"location":"string"}}]',
    I had a thing[1] over 10 years ago that could handle this kind of problem using SPARQL and knowledge graphs.

    My question is how effective is it at handling ambiguity.

    Can I send it something like a text message "lets catch up at coffee tomorrow 10:00" and a command like "save this" and have it choose a "add appointment" action from hundreds (or even tens) of possible tools?

    [1] https://github.com/nlothian/Acuitra/wiki/About

    by nl

Explore Birbla archives