Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- > Open-Source AI
There are no open source AI models, at least not useful ones (yet [1]). Open weight is not the same as open source. "Open weight" models are still just inscrutable binary blobs that you can (theoretically) run on your own computer instead of through a SAAS web app. The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data.
This is not open source and we should stop conflating the two things.
by petcat - It's not just copyrighted training data. Truly open source e2e model training would include scrapers, data cleaning, all pretraining scripts, posttraining scripts, exact hardware info, etc. Open weights labs will release a sanitized version to make themselves look good / not give too much away.by alansaber
- Completely agree with this, I'm sick of people conflating the two. Open-weight models should be treated no more favourably than proprietary freeware.
Sure you can run tests and benchmarks on open-weight models, but that is the extent - there is no scrutiny, no auditing for bias or copyright contamination - just a black box that you rely on for "intelligence". I'm still shocked the way people can hand over not just huge swathes of data, but also decisions of all shapes and sizes - to AI companies with no way of being able to assess how the sausage is made.
by foopod - I thought Nemotron tried to be pretty open?
- I think you would be pleasantly surprised by the content of the linked article.by Gracana
- This irks me as well. There have been perfectly good words to describe these.
The models are freeware, and some have additional restrictions in the license that make them shareware, imho!
by scoopr - I think there needs to be some nuance to this.
Without the training data and open sourcing the training code and protocol you cant reproduce the model from scratch, thats true, but because the weights and structure are free to download you can modify, inspect and make derivative works off it. Calling it an inscrutable binary blob is misleading.
I do agree that its not reproducible and its often talked about like it is.
In some ways its like most scientific research with a "methodology section", which in todays academic climate often falls quite a bit short of a detailed step-by-step to reproduce results with all the ingredients, but in principle could be used by other experts to do this again. Even without publishing the exact data, a catalogue or description of what data and how it was cleaned/processed would go a long way towards a reproducible methodology i agree.
by azath92 - K2 horizon is also fully open source i believe - https://ifm.ai/k2/
I understand your quibble with terminology, but i think the “inscrutable binary blob” thing is a bit off base. You can create finetunes and post train models using only their open weights. You can’t create derivative works like that from an inscrutable binary blob
by kennywinker - >The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data.
because the training data is full of copyrighted works.
the answer to "what went into the training data" is "everything we could get our hands on".
by vlyan - is aI a Conscious Being With Rights?: Emergence of Post-Human Collective Consciousness | Zenodo https://zenodo.org/records/20676952/latest
my fav read so far 2026
by vtemp009 - Mostly useless reading list. Very little emphasis on technical SOTA and mostly policy level waffling.
And regarding the data question the other commenters are asking — you scrape everything you can (oh look I used an em dash, wanna run me through the cover-your-ass Pangram?). Anna's archive, The Pile, the various Huggingface data sets and aggregates, Common Crawl. You pay proxy farms like Bright Data to run residential and mobile gray area proxies and VPN and CloudFlare bypasses to do more scraping. I see a lot of HN users up in arms on the front page thread about LG TVs having VPN SDKs within them etc. And then five minutes later they will go back to their frontier LLMs to print their pay cheque. Fucking hypocrites of the highest order. It's oh noes how bad this tech is in my house but we will happily benefit from it.
Then you have a data cleaning team deduplicate and clean and annotate the data (with or without help of more AI)
Certain RL specific datasets for supervised fine-tuning and RLHF like coding and git commits and chat needs to be curated by hand depending on your use case.
The level of discourse on AI has fallen tremendously on HN if 5 years into the AI revolution people are still wondering why datasets aren't being released. They aren't being released because they are a fucking snapshot of the internet for fuck's sake. There are a few "sanctuary" nations where AI data scraping is somewhat legally unenforced but the United States is not one of them so stop asking why data isn't released on a Bay Area website. Use your head for once. Too many React and YouTube influencers and the brains have been rotted.
Out of all the so called "AI engineers" here pontificating about "alignment" and "AI safety" and "Recursive Self Improvement", I wonder how many can even formulate or describe what an ELBO is. I wonder how many product managers here yapping about "recalibrating their priors" actually know what a prior is. I truly wonder why LLMs seem so magical to people when it's only a few steps removed from the same neural networks people have been using since 2015, at least architecturally (except scaled up by a few magnitudes).
Get your head out of Roko's Basilisk's agentic ass and maybe actually read the technical reports and papers for once.
(One important paper post Attention is All You Need is the DeepSeek paper where they used RL to bootstrap the "thinking" chain of thought token chains. IIRC it's the DeepSeek R2 paper. That's one of the most important papers for understanding LLMs beyond basic ML neural networks. If you need a quick way to get up to speed, read that one).
by Onavo - the fact that it's mostly about policies does not make the link list useless.by dada216
- I don't know if people actually expect the training data to be published. For me its frustration, if its not open - don't call it open-source. It's so easy to continue to use the words the same way as we did before AI.by foopod
- Seems like you have opinions on what a good reading list should be, so maybe you can post one here.by erikkri
- > I see a lot of HN users up in arms on the front page thread about LG TVs having VPN SDKs within them etc. And then five minutes later they will go back to their frontier LLMs to print their pay cheque. Fucking hypocrites of the highest order. It's oh noes how bad this tech is in my house but we will happily benefit from it.
Something something participate in society
(Or more elaborate: Disagreeing with the status quo does not make you a hypocrite for benefiting from the status quo.)
by aniviacat - How about learning the internals of LLMs, is Sebastian Raschka's content still the best in 2026?
- Ive gotten a lot of value lately using his writing to understand how parameter size, MOE and PLE variations on local models influence performance on various hardware. Its still a useful level of detail as someone who used to roll their own models for various applications pre-LLM but who now mostly uses LLMs rather than training them.
If you used to write pytorch, i think his writing is a great level of technicality to stay on top of things. And id look to them if i ever wanted to do a bit of learning/train my own.
Id love to hear if others have resources at a similar level of technicality and detial.
by azath92 - This should include "Hands-on Large Language Models" by Jay Alammar and Maarten Grootendorst.by ofrzeta
- is aI a Conscious Being With Rights?: Emergence of Post-Human Collective Consciousness | Zenodo https://zenodo.org/records/20676952/latest
my fav read so far 2026
by vtemp009