Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • People are not talking enough how huge this is for robotics and IoT. Current robotics arhitectures are limited by tok/sec. How cares if its not a Fable model?

    This move undercuts NVIDIA directly.

    by trash_cat
  • I see a lot of discourse about it being fast-to-deprecation. But I see it a different way personally.

    Modern LLMs are trying to do more with less. Focus on doing the right thing the first time. Even if we squeeze dumb LLMs, the significantly faster speed means quicker iterations. So a bad decision doesn't cost the time and inference costs that it cost before. It theoretically changes the scale of errant token spend.

    I compare it to the 1 thousand monkeys on a typewriter. In this case it's 1,000 monkeys with stale training data of everything ever written and the ability to search the web.

    by dabbz
  • What I like about this, is that it significantly increases the probability of a sci-fi scenario where you're picking up a hot chip on the black market; rumor has it, Mythos 9 weights baked in...
    by mNovak
  • The demo: https://chatjimmy.ai/
    by whythismatters
  • Thinking that five or six years from now, Fable-level intelligence could be provided at 100x the current speed... makes me feel lost. I cannot imagine what the future will look like.
    by linzhangrun
  • I'm surprised there's not more discussion about potential inflection points here. When technology gets faster, it opens up whole new classes of UX that were hard to predict

    For example, faster internet didn't mean being able to view 100x as many HTML4 web pages. It brought SaaS, streaming media and interactivity.

    I'm not good at predicting, but some ideas:

    1. All information gets augmented in real time with personalised context.

    2. AI interaction seems more like find-as-you-type than a back and forth.

    3. AI produces many outputs to pick from. Either the human, AI, or another system can do the deciding.

    Even if it's last year's model, speeding up LLM inference could open up all sorts of opportunities.

    by dave1010uk
  • I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition.

    Baking models onto silicon would've been the next logical move to get a moat.

    Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

    by LarsDu88
  • I think what will happen is what happened to something like 4K video decoding before where it ends up in silicon costing almost nothing to run extremely fast on device.

    "Good enough" LLM functionality (for the use case) will be on-die or on-chip for cars, appliances, etc. This will provide speeds of chatjimmy at a battery-level power consumption.

    Probably this will also happen for software engineering. Some usb-powered AI accelerator with Kimi K3 (and in future even better) performance running at 10K+ tokens/sec under 50W of power purchasable for almost no cost. Need a better model? Buy the new hardware. Old hardware is probably still fine for a lot of other use-cases. I expect China to be a big player here, it fits their open-model and hardware-manufacturing strategy.

    by TechTechTech

Explore Birbla archives