Birbla

Discussion summary

Small AI models are gaining popularity for use in areas with unreliable networks and limited hardware, with developers exploring local and offline solutions. Discussions include the benefits of small models, hardware options, and potential applications like neuro-symbolic AI.

What the discussion says

  • Small models improve offline and low-bandwidth AI applications.
  • Using rented GPU instances is a cost-effective way to train models.
  • Neuro-symbolic AI can enhance small models' capabilities.
“I've been working on small local models for years with txtai.”
— dmezzetti
“Most of the model work is just spinning a spinner, not actual computation.”
— bombcar

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I think neuro-symbolic AI has a lot of potential here, since small models can handle a lot of conversational inputs, while relying on wired-in solvers for more complex symbolic math/computation needs. https://jjd.io/posts/swollm-bbh-leaderboard.html
    by jdonaldson
  • 100% agree on this.

    I've been working on small local models for years with txtai (https://github.com/neuml/txtai). I've published close to 100 models that can run local for RAG, Agents, Vector Search and more (https://huggingface.co/NeuML/collections).

    by dmezzetti
  • Where is a good place to start with training SLM these days if you don't have the compute locally?
    by monkeydust
  • > The RxScanner is a handheld spectrometer that scans a pill with infrared light, then sends the item’s molecular profile to an AI model equipped with a pharmaceutical database. In seconds, the AI identifies the medication from its molecular profile—or reports that it’s phony.

    Is every tech, including database search "AI" now?

    by prmph
  • Can't wait to be killed by my toaster because some sexy mossad agent seduced it.
    by egormakarov
  • Has anyone used the Rx Scanner mentioned in the opening?

    https://rxall.net/rxscanner/

    by bix6
  • Is anyone making LLM-in-a-box for emergency supply kits yet?

    I feel that would be handy in all sorts of situations when networks are down.

    by tim-fan
  • I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable.

    I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet).

    It's actually how organic brains work - specialized tasks are offloaded to local cortical columns. The overall coordination between these sub-brains creates emergent skills/abilities.

    by N_Lens

Explore Birbla archives