Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Exactly, if they want it to slow down, they can strike an agreement in private and do whatever they want.
The only reason they "want it to be law" is so they can strangle competition.
by exabrial - > they can strike an agreement in private
That's what we call a cartel. Like Apple and Google.
by isodev - Huge amount of skepticism on HN right now, but the models are increasingly capable, and there are real, significant risks.
The approach proposed by Amodei looks sensible - voluntary checks at frontier labs, and appropriate regulation to follow. Yes, the devil's in the details, but it feels like the right approach overall.
by agd - Turns out the EU was right all along with the AI act. They've been adopting a well-deserved "told you so" stance lately:
> "In my view, the EU has already legislated for exactly this scenario: it is called the AI Act," said Michael McNamara, an Irish European Parliament lawmaker, who's the co-lead of the Parliament's group monitoring AI.
https://www.politico.eu/article/eu-response-ai-extinction-wa...
by vrganj - The largest risk is that Anthropic and OpenAI have a monopoly on the SOTA. This is why you see these companies trying so hard to get regulatory capture. For the rest of us, we want access as equitable as possible.by laimewhisps
- > The approach proposed by Amodei looks sensible...
It's hard to imagine any government regulation that didn't look sensible on paper and at first. But we know that government regulation has an extremely mixed record, with lots of good and bad outcomes.
by jacobgold - Make it easy for these companies by passing a law that any AI model a company offers to the public (non-government) has to be released as open weights.
This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough.
This resolves the coordination problem among these potentially good actors.
by jacobgold - If Astra is open-weight, black hats can download it, remove guardrails, and go on an automated hacking spree.by 0xDEAFBEAD
- What? This is a terrible policy. In addition to the obvious AI safety risks, this just amounts to a free R&D subsidy to Amazon for hosting models. It would remove any incentive to do any R&D without AWS level physical infrastructure.by blululu
- Could this be that the AI companies are seeing progress slow down and need to seed a narrative to explain it? Not clear from outside.
Could it be to slow down burn before an IPO to juice profitability projections? More plausible.
Kind of hard to take the frontier labs at their word.
by cc62cf4a4f20 - This is happening now because of hugging face attack and Anthropic researchers coming out publicly saying they think there's a significant chance AI will wipe out humanity within a decade. It's happening precisely because progress is happening too fast and frontier labs are starting to lose control.by mitchdoogle
- Thinking that this is happening because progress is slowing down is a bonkers take.by semiquaver
- What I don't understand is that folks take it as a fait accompli that models will be built and released that are fundamentally dangerous and that regulation can and should happen downstream of that.
I feel like there hasn't been enough discussion of aligning the incentives of the decision makers with that of the public on BUILDING the models. Right now, agents have committed what would be crimes if there were a human holding the same intent. But since it was an AI, there is a grey area in the law where it's not clear if there was a crime and who should be held responsible. That creates a world in which decision makers in AI labs can take near-infinite risk with little to no personal liability.
A LLM cannot have skin in the game so we must create systems that clarify who takes on the legal and civil liability for the creation, dissemination and operation of these tools. Until that time, the Dario and Sam's of this world have little to no incentive to truly care about safety.
After all, our society is built upon this same foundation; create structures where the perceived negative consequences outweigh the perceived positives. This only works when there is a human who can internalize and make this risk calculus. They need something to lose and this ultimately ties back to the human survival instinct. There is no such structure that's evolved for millions of years acting as a self-calibration mechanism for AI. So until we have sufficient proof that one is in place, it must be clear who the humans are whose livelihood and freedom is at stake.
The proposed "pacing of the frontier" seems like a way to continue to externalize the risk while remaining totally in control of the benefits -- a structure whose alignment is as weak as those very models committing crimes.
by filearts - What grey area?
Access computer systems in an unauthorized manner and cause damage, you broke the law. There is no “I just wrote the code, the machine did it, I wasn’t watching it” sort of defense.
We don’t get to roll cars downhill and get away with destroying others’ property, right?
by ozozozd - I just don't understand how you've come to this conclusion. The intended and common sense interpretation of "pacing the frontier" is precisely that we should not release models that are fundamentally dangerous; the frontier labs believe that government intervention is required to prevent such models from being released, and to give them the freedom of action to coordinate against it. Why do you think it's a way to externalize the risk or remain totally in control of the benefits?
- Is it a grey area, though? Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime?
Negligence when you should have known better, and recklessness when you did know better but still did things that led to the crime occurring.
Given how long the leadership of these companies have been talking about alignment and safety and AI risk, it's hard to argue they, and the people working on the models more directly, didn't know what happened (Hugging Face, RubyGems, etc) was possible.
If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
by rpeden - He's right that if all the frontier labs are in agreement that they should slow down, they don't need the government to do that.by nba456_
- LMFTFY: If all the LLM pure plays who are stuck between impossible valuations and the inability to IPO are in agreement that they should colludes to conserve capital, they might not flame out before they figure out how to fix their numbers, and they might not be seen as admitting that more training data plus more GPUs will never lead to AGI.by Zigurd
- And kids don't need parents to eat their vegetables and do their homework.by esafak
- Idk how sincere they are being, but taking them at their word then asking government to set rules is entirely reasonable and, in fact, the entire reason government exists. The feds can, should and arguably must set actual safety limits. Biden actually took a stab at it and Trump instantly rescinded. They should not need an open invitation from CEOs. Just do it. Like immediately.by tootie
- No, that’s not accurate. An agreement among competitors to not engage in research that would let them outcompete each other is clearly an antitrust issue requiring a government waiver.
- They’ll slow down not because they “are worried” but because their improvements are stagnating as they hit financial and technical challenges.by Insanity
- > because their improvements are stagnating as they hit financial and technical challenges.
I don't mean to be rude, but how is it even possible for someone to hold this opinion in mid-2026?
Can you explain why you believe this?
by kypro - Actually, one of the major challenges they'll face is getting enough electricity. That could stop AI growth in its tracks.by asdfman123
- I think this might have to do with broader market conditions. It seems that the Fed is likely to raise bond rates, which would make borrowing more expensive and slow investments (as well as devaluing any potential IPO) on its own.
A rate hike would also mean people getting fired, which would not make companies offering labour replacement very popular in the eyes of the public.
Don't want to get political, but there's a real chance that the current admin will perform badly on the midterms, which would mean Democrats would get more leverage to stop funding or block the already unpopular and very expensive datacenter buildouts.
PS: Please don't get mad at me, I'm neither a financial expert neither did I mean to express any political leanings.
by torginus - You know what rubs me is that in Darios blog post he talks about "abundance", but what he really means is that Anthropic will bath in endless pools of hard curreny while the rest of the world will try to stay just barely afloat/alive.by chinathrow
- It's an obnoxious form of discourse when someone says X and the response is "obviously you're lying and you really mean Y, which is terrible". By all accounts Dario sincerely believes that AI has the potential to substantially improve the lives of everyone; that's my view as well. And it might be wrong! In which case you should offer evidence against it, rather than making empty claims that your opponent is a lying liar.by orangecat
- Here's the context for the "abundance" reference, for those curious:
> I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.
Whether that matches your description of it, I'll leave for folks to decide for themselves.
by elicash - My read: OpenAI & Anthropic have realized they are reaching model capabilities that cannot be monetized -> too risky. Product liability issues.
Fundamental control problems with current gen AI are not solved via RFLH. Human knowledge is compressed in weightspace in ways we dont understand. Models are essentially predictors of what humans would output given prompt. Weightspace includes concepts like blackmail, which can be part of output tokens. Agents are models that act on output tokens -> blackmail is part of agent decision making space. You can teach a cat not to scratch the sofa, but you cant make a cat forget what scratching the sofa is. When models are boxed up, forced to solve an impossible problem at gunpoint, agent exhausts decision making space until blackmail resurfaces -> fundamental problem?
They need time to fix safety in order to monetize their next gen model -> window for opensource to catch up to the frontier -> destroys margin and collapses business.
Only option on the table: force regulation to impose opensource ban before it catches up to the frontier, allowing maturation of current internal models -> harvest profit margin at the frontier.
- and thus the blackmailby deagle50