Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I am wondering if this is why they can offer their pro model at ~1/4th of the price compared to the other providers offering the same model, and if other providers will be able to do the same in a short timeframe.
    by pokot0
  • It'd presumably help a lot, but also when you use their endpoint they get more training data.
    by vidarh
  • I have been heavily using DeepSeek V4 Pro at Max for a month now and I would say it is 100x cheaper. If I pay for Claude I will hit that limit so fast I am always waiting 5 hours. Using the frontier models at Kilo I go through dollars while doing the same thing via DeepSeek it is pennies.
    by sschueller
  • Inference I estimate runs 90% plus gross margins. Just work out the math on these servers. I am pretty sure any player can price down. It wouldn't look good on an IPO prospectus.
    by sfifs
  • I see a world soon where there’s an extremely wide variety of small models for speculative decoding, unique to use cases, companies, and even individuals.
    by Jackobrien
  • You clearly didn't read the recent speculative decoding papers because it's been possible to use any model to speculate for any other model for awhile. They solved the tokenization problems that prevented this in the past.
    by Der_Einzige
  • yes, heavily constrained by sophisticated guardrails.

    this is definitely where things are going. the enormous "eat the world" models have extreme diminishing returns by comparison.

    by pydry
  • Hopefully that is the case and hardware does not get impossible to get.
    by nicce
  • Presumably this has been in production for a while, and is one of the reasons they were able to dramatically lower prices a month ago?
    by ricardobeat
  • good catch, they reduced the prices 75% seems like exactly in line with the speed/inference optimizations gains?
    by sourcecodeplz
  • Lookahead Sparse Attention should be playing a big role as well, as it dramatically slashes memory consumption.
    by _0ffh
  • Yes. Section 5 talks about real-world deployment: 5.1: "The DSpark draft models are co-deployed with the preview versions of DeepSeek-V4-Flash and DeepSeek-V4-Pro"; 5.4: "MTP-1 represents the former production setup, having been superseded by DSpark two weeks following the DeepSeek-V4-preview release."
    by chronogram
  • At this point why can't someone produce a fridge or container-sized AI appliance based on legacy chips (12nm)? I imagine this would cover 80% of corporate use cases where you need to "google-in-a-box" functionality.

    The state-of-the-art nanometer are impossible to achieve but if you have infinite solar energy during business hours does it really matter? Every company has a parking spot so this ASIC-like appliance could be as big as a shipping container.

    If it could just run recent open models for a handful of users it would be such a nobrainer to buy.

    by bflesch
  • The issue is that there are only so many fabs in the world that make memory. And if you want the good stuff, your easily going into 400 ~ 750b parameter models. That means at FP4 400 to 750GB memory.

    Did i mention there are only so many memory makers and they are all busy printing money with HBM memory?

    Intel is trying with Crescent Island, to make a 160GB GPU that uses LPDDR5X memory.

    HBM takes multiple times the resources to make vs basic DDR5 memory. So by going this route, you have more memory, with the disadvantage that its only 700GB/s. VS HBM pumping out Terrabyte numbers like its nothing.

    These cards is reasonably priced, may be good alternative to $10k 96GB Nvidia Blackwells... You give up on token generation (heavily memory dependent), for more memory to run larger models at home/office/company servers.

    The problem is, again, there are only so many memory makers and its not like the market is flooded with DDR5 memory anymore, as the big 3 moved a lot of production to HBM.

    Another approach is Sandisk making HBF ... Flash memory, like your typical NVME but designed around maximum speed. So instead of loading the models into expensive HBM memory, you use the benefits of density in Flash memory, to offload models into that. Cheaper, but slower... But it leaves your expensive HBM memory free for things like KV Cache, Active parameters, etc... So your model will be slower, but your hybrid using it. As in, faster then running a model from system memory with normal DDR memory, but not as fast as HBM.

    So yea, there is a lot in development to reduce the dependance of that resource eating HBM memory. For the wafer cost of 1GB HBM, you normally got 4GB normal memory. That is why the world supply of memory dropped. Not just the insane buying but be HBM is just very inefficient in wafer usage.

    Can we not use DDR4 production and create some kind of hybrid solution? Sure, but the big 3 moved away from DDR4 in favor of DDR5 a long time ago. We have competition from China with a mix of DDR4/DDR5, but they also need to scale up. Nobody expected to see a large part of the world production vanish into HBM...

    Even if its about DDR4 and older nodes, ironically, most companies had been moving away from DDR4. There is only so much wafer capability in the world, to the point that companies are moving to using DDR2 ... Yea, not a typo, like 2007 DDR2! for IOT devices etc, stuff that does not need fast memory. Because even DDR3 got too expensive for them.

    Its not like the old nodes are not used anymore ... Like that capacity was sitting idle. It was still in production making other stuff. The only real solution is that we need more fabs, and those take years to build. And the big 3 delayed investing in new fabs for a long time, unsure about the whole AI bubble stuff. Aka, they did not want to make a ton of fabs to end up with over capacity if the AI growth collapsed.

    by benjiro29
  • Nvidia is already selling exactly this I think, not sure when it's expected to ship
    by sixhobbits
  • See "exabox" from George Hotz: https://tinycorp.myshopify.com/products/exabox-preorder
    by scrlk
  • Title is bad, it's the first line of the abstract instead of the paper title. Speculative decoding for LLM inference was published in 2022: https://arxiv.org/abs/2211.17192

    This paper seems to be an improvement to speculative decoding but I haven't read it yet.

    by articlepan
  • Is this newer/better than the speculative decoding from 2022? https://arxiv.org/abs/2211.17192
    by xnx
  • That paper is cited in the 'introduction' and 'background' sections. This paper is improving by removing some bottlenecks.
    by alok-g
  • Seems like they focus on improving the drafter and the verification policy so speculation keeps producing net speedups rather than wasted verification work at deepseek scale.
    by tiahura
  • Nice.

    Guessing the timing isn't accidental. Demonstrated openness vs harsh regulation

    by Havoc
  • Nobody forced anthropic to go on a media blitz loudly proclaiming the dangers their new AI model. Serves them right honestly.
    by declan_roberts
  • China = Open. US = Harsh Regulation

    Strange timeline, though this only works because it’s aligned with Xi’s goals.

    by cr125rider
  • This is just one of many papers DeepSeek have released to be able to serve models at extremely cheap prices, unlike the others taking on >$100B+ of debt in building data centers for the same thing.

    > As with V4-Flash, we treat this point as an indication that DSpark sustains useful throughput under an interactivity target that the baseline cannot efficiently support. At matched system capacities, DSpark delivers 57% to 78% faster per-user generation.

    Reminds me of the flawed solution in scaling servers in 2017 that use memory-intensive technologies by adding even more servers to solve the problem. (It just increases costs.)

    Rather than doing that, think about which critical parts of your app can be written in a more performant technology.

    Fast forward to 2026, now you can see who is just throwing more money at the problem to create even more problems where as DeepSeek is giving us optimized solutions.

    I know exactly who I would pay attention to, and it is absolutely not Anthropic.

    by rvz
  • ...... are you really suggesting OpenAI and Anthropic don't have access to these techniques?
    by simianwords
  • For so long American companies have operated under the assumption that servers are cheaper than developers, and that was used to justify all sorts of inefficient practices.

    The last year has shown that’s not true anymore (even for web servers).

    by denverllc
  • I’ve been using DeepSeek v4 pro for a month now in Kilo Code and its great. Fast, reliable, large context window and cheap as… Did 1,5B tokens this month and cost me 40usd (majority cached, but still).
    by piterrro
  • Have you compared Kilo to Pi or OpenCode? Those are the two I'm most familiar with but always looking for alternatives.
    by apitman
  • Which provider? I went through 40 bucks on it on openrouter. It was not a lot of back and forth, context ended at around 300k, 15kloc output. I was using opencode, unsure if I can make the total token count visible.
    by fer
  • I've been preferring Mimo recently. Same price as deekseek, more reliable tool calling (subjectively), and has some nice qualities in terms of prose, etc.

    I've heard others say that Deepseek tends to be smarter on specific problems but that Mimo tends to more well-rounded.

    by redman25