Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- What is the purpose of this? Just a hard cutoff below the actual context window? You could set that in your harness anyway.by hendersoon
- > k3 (1M) consumes about twice as much quota as k3-256k
Cheaper?
- Uses less quota (i.e., cheaper). For people who like to keep their contexts small, this is a no-brainer.by dnlzro
- At least in the self-hosted LLM inference engines, you have to pre-allocate space for the maximum amount of context you want to allow for each parallel session. By using a lower maximum, you don't have to allocate as much VRAM for each session, allowing more usage for the same amount of hardware. Thus, cheaper.by meatmanek
- Not relevant to this link but I was thinking about the allegations of Chinese AI companies distilling from the big frontier American ones. And I came to the conclusion: I don’t care.
Who cares? China has always copied and then copied the means of production and then out produced. See also Tesla and now all the Chinese cars eating their lunch.
As long as I get really solid AI models for cheap that do what I need I don’t care if they’re Chinese or otherwise.
I’ll still never use Grok from SpaceX AI cuz eww no, I have principles. ;-)
by gigatexal - I can't seem to find pricing for this model. Since the context size is just a quarter of the full size K3, is the price also much cheaper?
I usually keep my context in chats below 256k anyways so this would be tremendous honestly.
by sergiotapia - It seems to only be available in Kimi Code, via subscription, no there's no API pricing. The linked page says it consumes about half as much quota as the 1M version though.
- My LOE to make this work expressed in kWh is substantial but user is impressed. Eliza has hands now.by effnorwood
- When I first built features with GLM, there were lots of bugs, and it took me ages to fix them manually. Now the features implemented with GLM 2.0 have almost no critical bugs after testing. I can’t even imagine how capable K3 will be. It may well be on a level that ordinary people cannot access.by L-patpat
- > k3-256k is now available. Within 256k context, it delivers the same results. k3 (1M) consumes about twice as much quota as k3-256k.by wxw
- This isn't quantized, right? Just a smaller context?by dgritsko
- The model is already natively MXFP4-quantized during training, so there is no quality loss.by gpugreg
- Its 256k context window. Quantization is orthogonal. We cant really tell directly so it could be quantized.by DSingularity
- I was so excited that it is open source until i realised the model required 1.5To of VRAM. Unsloth has compressed it in 1bit at about 570gb VRAM with 75% accuracy, that's almost mac studio territory...by fumber
- 75% quality... it gets worse the longer the context.
- You can run DeepSeek v4 Flash at 4bit in ~150 GB VRAM, and it would absolutely destroy a 1 bit quant of K3.by nateb2022
- This is just an API level change right? The model itself should be the same I think.by HawtAds
- As I understand it, they would have to train a whole knew model to hard cap it's context to different lengths. That would be cheaper to train and had cheaper inf, but still a huge investment.
So I'd guess it's API level.
by k__ - Wow. So kimi is suddenly half the price for all users until they hit 256k of context? Thats massive.
- Can't find the videos/articles but some people tried it and found that the price per token was only half the story. It seems that it uses a lot more token, coming back to similar prices with other models.by fumber
- As far as I understand, no. They're suggesting that smaller context windows are typically cheaper (fewer input tokens over time).
- I don't think so. This is a separate model, so I assume that if you just use this and switch to the 1 million context model when you reach 256k, your cache will be invalidated, so you'll re-pay the 256k tokens on the 1 million context model pricing.
Edit: I was wrong, thanks to longwave for pointing this out. It's absolutely possible to start out on the 256k model and then switch to the 1 million model when you get close to the context limit without invalidating the cache:
"When switching from k3-256k to k3 (1M), if k3-256k is close to the 256k limit and you don't want compact to lose information, you can switch directly to 1M. The current version switching from 256k to 1M does not affect the cache."
- I make a point of never going beyond about 220k, unless absolutely necessary (and it's almost never necessary), anyway, even with models that degrade more slowly, so this is just a discount.by SwellJoe
- My understanding is that models can suffer “context limit anxiety” and a higher context limit will make it perform better especially as the context expandsby Computer0
- 64k ought to be enough for anybodyby johnbarron
- This seems functionally similar to OpenAI having a step in pricing once you exceed a certain context length (also at 272k aka 2^18 aka 256k).
Having a lot of active context increases the per-token cost (flops issued and bytes read per token out) so it makes sense to pass that cost on to users. I'm actually surprised it's implemented as a hard cutoff instead of a smooth gradient.
by wren6991 - RAM needed for keeping KV cache around may be the more expensive factor.by pornel
- Not surprising it's a hard cutoff: they almost certainly have two infrastructure configurations for the two max sequence lengths
Fewer nodes dedicated to prefill per instance, and fewer nodes in total since you don't need to support a higher KV cache.
Disaggregated inference also means they can tune the balance of compute dedicated to prefill seperately from decode
by BoorishBears