Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- If the belief that open-weight/Chinese models depend significantly on distillation of the latest frontier models is correct, then presumably the gap will stabilise to the minimum time required for extraction of meaningful data (from the latest frontier model) plus finalisation of training of the latest dependent model. This gap can be minimised by increasing the process efficiency, but can't be eliminated entirely. (Attempts to hinder distillation from Anthropic/OpenAI may shift the balance too.)by mft_
- USA, a country that known for the land of freedom, is now restricting frontier models to the point where non-Americans cannot even use them.
China, a "authoritarian state" country, "the antonym of freedom", with a software industry that is especially capitalist, has produced all the competitive open-weight models.
It really is IRONIC.
Disclosure: I am Chinese, and I understand this strategy comes from being behind, using open source as an asymmetric way to compete and make up for missing compute by sharing the burden, etc. But still, very ironically.
by linzhangrun - It would be interesting to know how much of a boost the closed models companies are giving the open models.
If the closed models stop improving will the progress of open models slow?
by jacobgold - > What is notable is that a large amount of the total improvement of models has been in the coding benchmark. The coding index has gone from 15 months behind to only a month or two behind
This makes sense, right? Coding is one of the most obvious short-term uses of models, it also has a readymade market willing to pay a lot for tokens, it has a huge corpus to work with, and a significant degree of validation is built into the problem domain...
by swiftcoder - I haven’t seen it discussed anywhere that closed models can essentially cheat benchmarks right? What Anthropic or OpenAI brand as a model doesn’t necessarily have to be just weights, it can be a whole backend system that augments the model itself. With this they can score better benchmarks than an open source model that is weights alone.by cedws
- > Now is probably a good time to liquidate your pension, fly to a remote island somewhere, and live out the remaining 6 months or so of civilization in peace.
> So maybe the open source apocalypse won’t happen yet.
Sorry I wasn't at the last doomer meeting, when did we decide good open source models are a harbinger for the apocalypse?
by taffydavid - The Chinese models will not overtake the frontier US ones given the current way things are going. The US models derive their lead from incredible efforts to source more and higher quality (mostly synthetic data) via great feats (eg generating with humongous teacher models that could never feasibly serve interactive traffic). The Chinese models advance via heroic efforts to optimize models and great feats to secure more and higher quality training data from the US frontier models.
For an (Chinese) open weight model to surpass the (US lab) frontier models, this equation must flip and the Chinese labs must entirely retool from harvesting frontier model data to producing the data systems and efforts to produce novel data; as well as procuring latest generation hardware en masse for this. This does not happen easily. Also training a frontier scale model is actually not such an unimaginable feat: doing all the inference with the teacher models is where the hardware goes.
by christina97 - IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek).
The spigot can be turned off at any time.
Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.