Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Not a lawyer but distillation sounds like a transformative work.
Same thing as Cliff Notes imo. In every other area of manufacturering and tech I can use a machine to build a new machine that competes with the original machine. Should Milwaukee be able to prevent DeWalt from using their drill to make a competing drill? Should Jetbrains ban Eclipse contributors from using their IDE?
by nijave - My guess is that when you sign up for either Anthropic or OpenAI, the terms of use specify you can't use their model for purpose A, B, C, D. For example, you can't use their model to try to build biological weapons, or to try to extort people, etc. Most likely there is language there that you can't use their models to train other models. It's as simple as that. You agree to those terms of use, or you don't use their models.by credit_guy
- > To him, the true AI doomer scenario is for all the immense power of frontier AI to wind up in the hands of a single powerful, proprietary provider.
Isn't this exactly what Dario wanted? He thought he knew what's best for the humanity...
by hintymad - It is. Dario the Book Burner will not good what he wants, the world sees through him.by nullbio
- Agreed! Allow US companies to innovate by creating an ecosystem of smaller, more efficient open weight models and it will be a net benefit for everyone. Distillation is a good thing.
Preventing token-consumers from developing competing products should be litigated as anti-competitive behavior.
by pton_xd - Ok, but how do the economics of this work? Based on its settlement, Anthropic paid an average of $3000 per work they scanned based on their settlement (https://tech-insider.org/au/anthropic-copyright-settlement-2...). They and OpenAI pay billions per year for a mix of experts and normal people to label or create data. Why would they continue doing this if the value of this is immediately copied by open models? If your goal is to end the economics of generating and buying data for AI (and I recognize for some people this is really the goal) then sure, but if you want AI for various subfields of interest to continue improving then it's not workable.
Back when people made arguments for software privacy, the argument was usually "big business will still pay and consumers wouldn't have paid anyways so it's ok for us to pirate" - I actually think that was fine for business software but terrible for indie games, whose market was 0% businesses.
But in the AI case, it's not like they get to keep some of the value of their investment - it all gets cloned into models that businesses and consumers alike are happy to use. If someone knows how labs could continue to fund data creation and acquisition in this model, please do share!
by zetazzed - %99 of the startups fail, they are venture backed. Nobody or no market forced them to spend like that. It's all their decisionsby kingleopold
- Surely if you hoover up every book in existence to feed into an ai model you must be extracting more than 1.5B in value. If not then it’s not a viable business.by wonnage
- > Anthropic paid an average of $3000 per work they scanned based on their settlement
Not sure you get to count breaking the law and getting in trouble in your cost-of-doing-business. That's a little too on the nose.
You're basically arguing that a criminal syndicate must be allowed to continue and we're required to make their business model make sense?
by kadoban - They can’t they’re literally fucked, and it’s not society’s problem! The whole world doesn't have to bend over to make sure a couple of lunatics who believe they are building a doomsday weapon also have a viable business modelby etdznots
- https://youtu.be/ZIaOBAjvc38
Garry Tan and Sam Altman recently did this interview together. They seemed pretty friendly with each other during it. Wonder what Sam Altman would say about Tan advocating for OpenAI’s models to be distilled.
Then again this is the same OpenAI that has gotten into legal trouble recently regarding Apple’s IP so who knows
by ViktorRay - Society as a whole has paid into this technology: through the theft of its intellectual property, through having to deal with the pillaging of so many commons (digital or otherwise) by it, through skyrocketing energy and computing device prices, and even just through ordinary investment. Democratize the technology! At the very least, don't step in legally to prevent this from happening.by gr_norm
- > To him, the true AI doomer scenario is for all the immense power of frontier AI to wind up in the hands of a single powerful, proprietary provider. “The nightmare scenario, the doomer scenario for AI is that there’s just one company,” he said. “It has the best access to capital. It has the best AI researchers. It runs away with it and suddenly there’s one company that’s monolithic. And that would be bad.
Well yes, as I think I said in a previous comment, on the current trajectory OpenAI and Anthropic will really stop releasing models due to distillation and regulatory pressures. Then, they would eat all knowledge work themselves, which would be the end of YC.
by consumer451 - Controlling what users and customers do with API calls to closed weight models feels constraining, and there’s a role government can play here to normalize the fact that access to intelligence that was trained on broad public access data should itself also be more a form of a public good than something locked away behind restrictive terms of service
I do not agree with this man all that often, but that is very concisely put.
by dofm - I think OpenAI and Anthropic will go bust, or at least be scrapped for parts in the next 5 years or so. It's clear that the extreme cost used up for training is impossible to recoup, as inference is already being subsidized.
It's also clear that, as Tan indicates, open-weight models will be (and basically already are) just as good as frontier models. It's all about the harness, baby. We will have two main forks in the road, and two new industries created:
We already saw a glimmer of this with popularity of OpenClaw—the problem is that it's janky, hard to set up, inconsistent, and very hacker-esque. Imo "AI labs" will be a dying breed because there's no real money in the actual models if they get commoditized, which they already kind of are.- AI hardware (NVidia/Cerebras/etc.), the equivalent of Intel/AMD - AI software (harnesses, assistants, etc.) the equivalent of Microsoft/Appleby dvt - If inference needs to be subsidized to be economical (idk if true), open models have the same problem.by hn993302
- It would probably be for the best if they did, and training became something that humans did collaboratively.by nullbio
- If harness is all that matters, a co-developed harness + model stack + large compute availability advantage + massive distribution advantage with data for post training will win the market.by FanaHOVA
idk where this comes from but it's laughably false.> inference is already being subsidizedthe only place where actual subsidization (below cost) might be happening are the subscriptions. even that is unlikely because to be truly below cost you either need to offer below cost of electricity which isn't happening, have potential API users using multiple subscriptions or have opportunity cost loss due to saturation.
by teravor- I think this is very possible. Plus, something I don't see talked about enough here. The VERY fragile supply chain that keeps it all going. Look at what is happening in the Middle East.
The US can no longer keep global trade secure on the high seas. What if the supply chains for GPUs get disrupted for months, a year? Then what?
I fear Google will win in the longer run.
by kilroy123 - >inference is already being subsidized.
Inference is not being subsidized and in fact has pretty high margins.
Similar-sized open weight models on openrouter are 15x cheaper per token than the big labs. This should reflect the isolated cost of inference, since 3rd party hosts have no reason to subsidize and no training costs to amortize.
Only datacenter buildout costs are being subsidized.
by Legend2440 - > He also notes that the proprietary AI labs didn’t ask permission when they vacuumed up as much human knowledge as they could to train their models.
I think this should desactivate the moral high ground from which Anthropic is trying to speak. That they would want to make distillation orderly IMHO is fair, but to make it illegal is very rich from any AI frontier lab, really.
by TheJCDenton - YC does better if its startups get open weight frontier benefits. Garry’s just advocating for his book, which is his job. Consider how much capital YC portfolio companies would have to burn until liquidity if they have to pay OpenAI and Anthropic, versus relying on open weight frontier capabilities.by toomuchtodo
- I reflected on this myself recently. Model distillation seems to be at least as fair a use as distilling a book.by sobellian
- Also, as said elsewhere: "Lab" is rich here, for outfits that, facing these giant, energy swallowing black boxes have really no clue what's going on inside.-
The moniker gives them an air of scientific, knowledgeable, tranquil, pro-social, pro bono work.-
Of course they are entitled to kill off a few mice, or pillage the commons to forward their "lab" work.-
by Bluestein - I agree. The frontier models are based on training data from tons of copyrighted work. Some of that work was obtained illegally, even. They could not exist without strip-mining the commons. The labs have no moral or ethical ownership to the end result, and others should feel free to treat any company-imposed restrictions on their use as invalid.
I don't expect Tan's position to be based on any kind of real moral high ground, but his conclusion is correct.
I love the "illicit distillation attacks" framing from the incumbents. There's nothing illicit. There's no attack. You just don't like it because it threatens your market position and business model.
by kelnos - 100% - the work came from the people, it should go back into the hands of the people.
I also think if Anthropic and OpenAI had been releasing Open models along the way, people wouldn't be nearly as suspicious of them.
by soundworlds - "Strip-mine" is not correct. The commons are all still there and you can still train on them just like the frontier labs did. Of course, it may be illegal to do so, but that's not any different than before.by throwawayk7h
- These frontier labs violate billions of terms of services across the web, that prohibit scraping / automated access / etc. Most sites have a clause, it’s basically standard boilerplate.
So why is their own ToS so special? :)
by dannyw - There is nothing illegal about training on traces from frontier models.
However the frontier labs don’t have to serve customers who are farming the service for distillation purposes. That’s their choice and they’re free to make it if they detect distillation happening.
by Aurornis