Comments
Hacker News
LLMs are trained on public data (as well as illegally obtained data see: Anthropic 1.5B settlement). LLMs are nothing without the huge corpus of human data that powers them. There is an argument that research of this kind should be restricted to governments and regulated universities rather than opaque public companies with competing incentives. Or research should be stewarded by genuine non-profit collectives with democratic leadership. e.g. like internet standards, telecom, etc
I do not think we can trust private companies, no matter the virtue signaling they put forth into the world, to effectively regulate themselves and inform the public and scientific communities about risks. Their ongoing conflict of interest poses serious credibility risks.
by aarondong
The reason why anyone argues against local and free AI is because they want corporations to have control over the general population.
America is a corporate hellhole.
by kouru225
Just to add on to other comments: reasonable people can disagree about the degree of safety concern with near to medium term AI. But to not address the arguments at all is, in my opinion, a serious mark against the value of this article.
by MostlyStable
The only good arguments I see against open weight AI also apply to closed AI. And regardless, the box is open, nobody can stop it even if stopping it was a good thing.
by deaton
I have shipped open weight models in iOS apps and agree with this in theory. A model is just one (significant) piece of the stack though. Prompting, adjustable parameters, tooling, workflows, and interfaces are what make a product usable. Black boxes, open and commercial, sit in most of the products people trust today.
As for the backdoor section, inspecting weights is not the same as auditing training. You can examine behavior and strip guardrails with fine tuning but you cannot determine from the weights what the model was trained on or whether the data was spiked. That can be a real problem for some use cases.
Closed models are far more opaque but you are buying in to a contract and accountability in exchange for the lack of transparency.
by James333i
OpenAI exec scaremongering about "A nonliving, invisible, dangerous, and infinitely self- replicating agent escaped from a Chinese lab" mere 4 days before https://news.ycombinator.com/item?id=48997548 is hilarious
by valicord
>Much of the angst around China's models centers on "losing the AI race". But what's the goal of this race? Is it to develop the best model? To sell the most tokens? To destroy humanity first?
Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI development that, if realized, would effectively make it a superweapon. Altman has been pretty vocal about his expectation that this will eventually happen and that it is his goal to be the guy to produce it. Even if it isn't some superintelligence, the ability for a machine to do something like, say, exploit cybersecurity weaknesses, is pretty worrying for entities like governments. It's the pretense used when we saw the US government ban a US model recently.
Again, you don't have to buy that "the singularity" is a real thing, but it's not hard to see that some people think some version of this is real and it is exactly what is being referred to as the goal in an "AI race."
by nater5000
This is not "open source" AI.
Photoshop source code + OSI license = open source
Photoshop binary = open weight
Photoshop SAAS web app = closed model like GPT, Opus/Fable etc.
There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API.
If you want actual open source then you would need to look at like OLMo 3
by petcat
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- LLMs are trained on public data (as well as illegally obtained data see: Anthropic 1.5B settlement). LLMs are nothing without the huge corpus of human data that powers them. There is an argument that research of this kind should be restricted to governments and regulated universities rather than opaque public companies with competing incentives. Or research should be stewarded by genuine non-profit collectives with democratic leadership. e.g. like internet standards, telecom, etc
I do not think we can trust private companies, no matter the virtue signaling they put forth into the world, to effectively regulate themselves and inform the public and scientific communities about risks. Their ongoing conflict of interest poses serious credibility risks.
by aarondong - The reason why anyone argues against local and free AI is because they want corporations to have control over the general population.
America is a corporate hellhole.
by kouru225 - Just to add on to other comments: reasonable people can disagree about the degree of safety concern with near to medium term AI. But to not address the arguments at all is, in my opinion, a serious mark against the value of this article.by MostlyStable
- The only good arguments I see against open weight AI also apply to closed AI. And regardless, the box is open, nobody can stop it even if stopping it was a good thing.by deaton
- I have shipped open weight models in iOS apps and agree with this in theory. A model is just one (significant) piece of the stack though. Prompting, adjustable parameters, tooling, workflows, and interfaces are what make a product usable. Black boxes, open and commercial, sit in most of the products people trust today.
As for the backdoor section, inspecting weights is not the same as auditing training. You can examine behavior and strip guardrails with fine tuning but you cannot determine from the weights what the model was trained on or whether the data was spiked. That can be a real problem for some use cases.
Closed models are far more opaque but you are buying in to a contract and accountability in exchange for the lack of transparency.
by James333i - OpenAI exec scaremongering about "A nonliving, invisible, dangerous, and infinitely self- replicating agent escaped from a Chinese lab" mere 4 days before https://news.ycombinator.com/item?id=48997548 is hilariousby valicord
- >Much of the angst around China's models centers on "losing the AI race". But what's the goal of this race? Is it to develop the best model? To sell the most tokens? To destroy humanity first?
Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI development that, if realized, would effectively make it a superweapon. Altman has been pretty vocal about his expectation that this will eventually happen and that it is his goal to be the guy to produce it. Even if it isn't some superintelligence, the ability for a machine to do something like, say, exploit cybersecurity weaknesses, is pretty worrying for entities like governments. It's the pretense used when we saw the US government ban a US model recently.
Again, you don't have to buy that "the singularity" is a real thing, but it's not hard to see that some people think some version of this is real and it is exactly what is being referred to as the goal in an "AI race."
by nater5000 - This is not "open source" AI.
Photoshop source code + OSI license = open source
Photoshop binary = open weight
Photoshop SAAS web app = closed model like GPT, Opus/Fable etc.
There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API.
If you want actual open source then you would need to look at like OLMo 3
by petcat