Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • With a tps and a token price you can calculate approx. price per hour of running the model!

    $2.61/M tokens * 1,000 tok/s = $9.40/hr

    That would be pretty cheap for an 8-GPU node which would typically run around $45/hr or more. Guess this depends on how many parallel streams it can handle.

    by pants2
  • The interesting bits on how they achieved it:

    > On the model side, we applied FP4 quantization

    > introduced DFlash, an efficient speculative decoding method based on block-level masked parallel prediction

    > On the system side, TileRT perfectly adapts to the dynamic characteristics of these algorithms

    > 1000+ tokens/s output [...] using just a single standard 8-GPU commodity node

    by PhilippGille
  • 1k TPS is great, but I’m more fascinated by the amount of AI generated comments in this thread!
    by Oras
  • Like what?
    by eli
  • Comments at 1,000 TPS is a terrifying future.
    by trollbridge
  • Cerebras is trialing Kimi K2.6 at 3000t/s (invite only). I'm excited for when the fast hardware gets more mainstream for frontier models. Models designed for speed on Nvidia are nice addition that could bridge the gap.
    by scosman
  • now that's what i call a software development breakthrough/platform! thanks for the heads up!
    by michael-ax
  • Cerebras got lucky that they IPOed last month instead of now.
    by johndough
  • Source? Their website says 1000t/s https://www.cerebras.ai/blog/which-is-faster-gemini-3-5-flas...
    by btian
  • Cerebras currently does not provide any discounts for prefix caching making its use for agentic workloads sqr(n_turns) more expensive.
    by lostmsu
  • TFA mentions that until now special very expensive hardware like Cerebras was required for reaching this kind of speeds, and it emphasizes that what is novel in their results is that they have obtained over 1000 token/s for a model with over 1 T parameters by using just standard hardware, i.e. one server with 8 GPUs.
    by adrian_b
  • Neat. The frontier models have gotten pretty impressive, but they're all a bit too slow for interactive, human-in-the-loop coding. It incentivizes vibecoding and running multiple agents in parallel. A fast agent feels more like a partner.

    For a while I was running Cerebras GLM 4.7 for a bunch of tasks. Not a very smart model, but it's fantastic to be have a live prototype of a site up and be able to type "make the fonts bigger. No not that big" and see it change in real time. And MiMo 2.5 is a lot more capable than GLM 4.7.

    by eli
  • i tried glm 4.7 for agents that write code. simple scripts 200-1000 LOC. extremely bad . Had to abandon cerebras oferning, their smart models are only on enterprise plan.
    by maxdo
  • > And MiMo 2.5 is a lot more capable than GLM 4.7

    MiMo 2.5 is not the same model as MiMo 2.5 Pro.

    GLM 5.1 is z.ai's lastest iteration & is one of the popular open weight coding models.

    If you've had the chance, how does GLM 5.1 (which is now more expensive than MiMo 2.5 Pro after its recent 70% price drop) compare?

    by ignoramous
  • This will be really powerful for voice. Being able to reason makes LLM so much smarter but with voice your latency budget is so tight that you can't spare the time typically.
    by prplfsh
  • This is true for humans too. Lol
    by jeffrallen
  • I may sound like a shill, but exponential growth and all. We are going to get near instant software from prompt, multiple ones and then choose the best one.

    Discussions about choosing a library with the best syntactic sugar method naming is just as crazy as suggesting we type in assembly.

    by serpix
  • And they will all suck! I can't wait.
    by lionkor
  • Our bottleneck is going to be verification.
    by tmaly
  • > Discussions about choosing a library with the best syntactic sugar method naming is just as crazy as suggesting we type in assembly.

    I have a more hopeful take. As AIs improve and get faster we can more quickly and iteratively improve code which we may have historically avoided due to the work involved.

    I know i've made several refactors that would have otherwise been insane lifts. Not only because the work involved but because sometimes you don't know if it will work, and so you have a sort of double friction; you don't know if it will even succeed. With an AI you can just throw it at the refactor to see if it runs into a problem all while you're having a coffee break or w/e.

    In general AI is going to enable humanity to be more extreme versions of itself. For good and bad. I suspect more bad than good, though.

    by unshavedyak
  • You won't. Because 80% of the complexity is just "knowing what to build". You will get something that gives you a prototype in 1 min, then you break it, then you get a slightly better prototype one one side, but newly broken in another way, and you're going to repeat over and over.
    by oulipo2
  • I'm not sure. Engineers could still develop software the old way, you know taking months to deliver something like, let's say, Obsidian? Or Ghostty? Taking care of every single line of code, of dependencies, of good architecture. Truly the old way. And if the product is good it will succeed.
    by dakiol
  • The exponential is leading to full compute-in-memory within a few years which will be 100 times more efficient. Which means at least 10 times larger models that are much smarter in addition to extremely fast.

    It's going to skip the code entirely for small businesses and just render UIs straight from context data and prompts at interactive speeds. Kind of like Google's Genie does with games but much more accurately.

    by ilaksh
  • Anyone remember the old days when a new frontend framework came out every 3 months. That has pretty much stopped. No one cares anymore.
    by 9cb14c1ec0
  • Sounds like exponential growth of crappy software. I'm not saying that before we didn't have mass produced crap in SE, but now it will turn into explosive overflow.
    by alkyon
  • MiMo V2.5 Pro (regular speed) remains the strongest open weights agentic coding model we've tested -- it's been interesting to see how little attention it has received relative to some lower performing releases. And the "fast mode" pricing is very competitive here.

    Data at https://gertlabs.com/rankings

    by gertlabs
  • why is deepseek v4 pro a lot lower than flash? where is mimo 2.5?
    by unrvl22
  • Given that MiMo is as cheap as Deepseek ( previous discussion: https://news.ycombinator.com/item?id=48282814 ) multiplying that by 3x for ultra speed is still shockingly cheap.
    by kingstnap
  • MiMo and DeepSeek are not cheap. Anthropic and OpenAI are expensive for what they provide.
    by miroljub
  • These price and speed optimization from Chinese providers, combined with the raising prices from American ones will change the game sooner than later. Many companies are finding issues with the AI bills already.
    by amunozo