Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • "As far as we know, that is the first time a model of this class has run natively on a phone.".

    I feel like I've seen a similar statement on a lot of these streaming weight projects.

    400b model on an iPhone: https://x.com/anemll/status/2035901335984611412

    by sallymander
  • > One expectation to set honestly

    Hello Claude!

    by throwawayffffas
  • The web and connecting to other services is very important for almost all of my use cases. While I believe we are going to get better and faster models, the web index is certainly not downloadable and maintainable for 99.99% of the folks who are able to use local models. Any good solutions exist?
    by gizmodo59
  • I know relatively little about the workings of LLMs, but I keep seeing projects like this that run massive MoE models using very modest amounts of RAM, perhaps excessively so. I wonder, is there a way to make the RAM usage tunable? I have a Macbook with 32 GB of RAM, and it'd be great if I could run the same model but take advantage of the additional RAM to make it run faster.
    by CyLith
  • This looks useful, you can increase the RAM cache so if you have a Mac with 24-32GB it should speed up a lot and still run models that wouldn’t normally fit. I’m going to run some tests…
    by adrianco
  • Thank you for using TurboFieldfare as a starting point for this project and thank you for mentioning it at the README. I am glad it inspired more people to explore area of on-device AI further!
    by gitpusher42
  • I read a comment on here a few months back I wanna restate. Basically, there is a good chance that Apple is betting that the LLMs in the future will be so efficient that those that consumers will use everyday will be easily computed by the iPhone or even bigger ones on Macs. Honestly makes the most sense that we are heading that way in a few years latest.
    by AHASIC
  • I know everyone wants to crap all over these setups that are impractical, but this is how progress happens.

    People will keep plugging away at this and figure out how to avoid wearing the hard drive, how to make it run faster, custom hardware buses etc.

    Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.

    by dghlsakjg

Explore Birbla archives