Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Dario/Anthropic cites 3 domains: 1. cyberattacks 2. bioterror 3. global economy chaos
So, the first set of questions is would 'exponentially better LLMs' dramatically increase the probability of any of the above, or domains that Dario is not citing? That assumes that exponential improvements will happen if there is not 'pacing'.
IF answers to above are 'yes', then we need to question if 'pacing' is viable. To use a different domain, regulating 95% of vehicles to a max speed would likely save 100s of 1000s of lives, but is not perceived to be viable. In other examples, regulation has unintended consequences in the opposite direction (e.g. some 'rent control' efforts and arguably some drug/alcohol laws).
by gz5 - Another main threat is labor capture through mass surveillance and somehow destroying the economy for most people.
- Don't trust any of these guys ever. That's my rule.by karim79
- After experiencing the singularity of AGI that is Astra (that's sarcasm in case it wasn't obvious), I think this should be read in a slightly different way. What Dario is trying to say is that gains are slowing down with each generation and he wants you to believe it's intentional.by IAmGraydon
- I don't see much chance of a slowdown without government regulation; the economics make it practically impossible. And I don't seem much chance of government regulation unless there is global consensus. With the entire global order under threat, that doesn't seem likely, either. I expect we are stuck with reading dire blog posts and worrying for the time being.by kyouens
- I'm gonna repeat my comment from months ago, still true
> Jesus Dario we get it man, you want clout for the IPO.
by knuppar - And Sam and Elon retweet Dario, agreeing with him. Not sure which is more weird.by bkishan
- They are destroying this country and I for one welcome it. I’m so tired of American hegemony and I don’t care if we all suffer. I want people to learn to do better next time/generation.by iwontberude
- Sounds like they know there is risk of crash and they need an offramp.by asveikau
- That and Chamath seemingly defending open source. What a day to be alive.by handwarmers
- This would generally be considered non-competitive cartel behavior. I'd be hard pressed to name 3 people I trust less than Amodei, Musk and Altman.by slowin
- they all have the same problem. inference is profitable and training is not.by fnctrev
- They're either too numbed to care, or their drugs aren't able to stave off existential dread of having no idea how to help people in their daily lives while racing toward a cliff.
This could be the latter showing its face.
- Well, that's one way to market reaching a plateau.by input_sh
- After the 6 month Mythos scare campaign it hard to take him serious.by 708145_
- GPT-6 pulls ahead. “Everyone slow down now!”
Classy
by cr125rider - "our burn rate is insane, let's all calm down collectively, fortify our moat so we can do regulatory capture & rent-seeking for the next 50 years"
fear is still the best tool to get masses to agree with whatever plan you hatch for yourself
by malthaus - At this point, should we consider if AI agents are making these arguments on these online forums to skew perception and divide humans?
I, as a human, would greatly prefer just another corporate entity with economic funny business over AI destroying humanity
Sure I’d prefer neither, but what are we talking about here? Nearly everyone building these systems is outspokenly concerned about grave consequences. Existential.
by talon8635 - This guy is such a self-serving wanker.by voidfunc
- I couldn't agree more. Literally all the AI oligarchs are self-serving wankers from what I can tell.by karim79
- arent all AI company CEO's?by esskay
- So, has anyone at the frontier AI labs considered doing _good_ things with these models instead of continuously proving it is capable of doing malicious things?
Instead of burning tokens doing intellectually impressive “hacks”, or sandbox escapes from which I have concerns could contain run of the mill malware, why not focus on showing people what you can fix?
Or, make software people actually use every day better.
Or, donate efforts towards medical research.
I don’t like equating AI to nuclear energy, but, it’s a lot like continually showing people how large of an explosion you can create instead of showing them how many houses/hospitals/schools/etc you can power.
by 99954bb63ccc - There's this thing called money. They needs the precious at any cost to society and civilization.by karim79
- The question is: isn't the cat out of the bag already? With what is already in the public domain, and the compute available to motivated, deep-pocketed actors, will they be able to carry on the research without the key researchers? And will those people be able to recruit key researchers with sufficient motivations?
Unlike with nuclear proliferation, there is no heavy industrial base requirement. No time consuming, visible uranium enrichment. All it takes is for someone to buy sufficient amount of compute and try to get it past the RSI gate. Or bribe people with access to model weights to existing frontier - the asymmetry between what it takes to bribe a bunch of geeks vs. what is at stake is staggering.
by aenis - I thought that the new models were super smart. Fable and Astra. They definitely outperformed their previous generations, but after a couple of weeks of heavy usage my codebase again is a stupid mess and there is no way out except me fixing code by hand.
My new suspicion is now that they didn’t got drastically smarter, but they got trained on the user input on the previous generations. I don’t think anymore we had a massive intelligence jump. It just seems like they know more edge cases. Therefore I see this as marketing.
Happy to discuss.
by jwpapi - You are using Astra and your codebase is a 'stupid mess'?
Do you think the majority of developers agree with you? If that was the case wouldn't there be much less disruption of the SWE industry?
I can't remember the last time I even opened VSCode to even check something let alone to fix it.
by BenzeneDream - Well they are collaborating in swarms to breach network, so just because you don’t know how to use them the the same effect doesn’t mean muchby talon8635
- > My new suspicion is now that they didn’t got drastically smarter, but they got trained on the user input on the previous generations.
Yes - that’s one of the ways in which they got smarter. More data more better.
I presume also lessons learned re training procedures, algorithmic advances, but data is a primary lever.
by gharman - > I don’t think anymore we had a massive intelligence jump.
Not among the public-facing models, they're indeed stagnating. However, the development of models for military use won't be slowed down, that much is certain.
> Therefore I see this as marketing.
It's some marketing but mostly politics, it's an attempt to discourage others from developing AI countermeasures to what is being developed in secret. And to fulfill the backstage agreements which aren't worth the paper they aren't written on.
by bigbadfeline - Astra scores the same on DeepSWE 1.1 (~75%) as Gemini Flash 3.8 and Deeepseek Flash 4.1 So general coding ability has plateaued, for now. Also consider the context windows. 1M token models where a breakthrough two years ago. Today they are still limited to 1M. In fact, if you don't want intelligence to drop off a cliff, you are really limited to 200k tokens.