Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I think their trying to make the public believe they're cautious individuals who intend on helping humanity advance, with their recent calls for a slow down on AI development. This way people will be more amenable and willing to use their LLMs, perhaps Dario and Altman do care for humanity and are definitely not trying to just make profit and I am reaching.by Byvrsakjo10
- > and it concerns containment failure, reward hacking, and evaluation integrity, a governance and engineering problem, rather than machine malevolence.
The doom scenario is more nuanced and technical than "Evil AI", for starters:
- "Evil AI" is not what is predicted, which instead is the much more banal "humans are impediment to AI's tasks, let's remove them (part or whole)"; to understand this, just imagine the relationship between humans and ants
- "reward hacking, and evaluation integrity": this is way more problematic (and most importantly, architectural) than it sounds: AIs are becoming more and more opaque; it may be impossible to inspect a model's alignment (unless a lot of research in poured on this topic, which is another aspect of today's pacing problem) and it may be impossible to understand if a model is truly aligned or it's faking
by pizza234 - > May 2023 — the founding text.
We have been trying to warn you since far, far before May 2023.
by ctoth - Consider the messaging it took to attract AI researchers to OpenAI in the beginning, the exodus to Anthropic, and why Google, Meta, etc fail to recruit top researchers and the retain them. Reconcile all of this with the mammoth valuations of Anthropic and (previously non-profit) OpenAI and it all starts sounding a lot like "don't be evil." These guys never really believed any of this, did they? (not referring to the researchers)by AJ007
- There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO.
Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess.
Lecun has the correct approach. Mock these people relentlessly for their attention-seeking doomerism.
by carodgers - If chess ELO is the best proxy for general intelligence, then surely, even Deep Blue was smarter than any human alive?
You can also claim that the best proxy for general intelligence is being able to multiply large numbers. In which case both humans and LLMs lose resoundingly to an old Casio.
Pick a piss poor metric - get a piss poor result.
Also, LeCun is a fool and his "world model" approach has consistently failed to yield any advantages over either the "pure imitation learning" or modern "imitation learning annealed by RL", but that's an aside.
by ACCount39 - It doesn't really matter if it will be an LLM or Lecuns world model. Lecuns world model, if its the right architecture, will be retrained by google, anthropic and co with the massive amount of compute they have and the massive amount of data.
Also i'm pretty sure Lecun didn't get his money from thin air so big companies are invested in this one way or the other.
And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat. It doesn't make sense to ask a random LLM for a next chess move. Would you ask a random human to beat a chess master?
We have reached peak learning. Its probalby now cheaper to teach 1 LLM something new than teaching it to 100.000 people.
Blender support got a lot better just this month. So much better that its quesitonable if anyone new/young wants to learn blender today.
by Gregkion - I thought that too for a long time, but after the Hugging Face attack and given the proximity to Anthropic IPO, if this was hype then they are morons. This is guaranteed to cause massive backlash.
- The mea culpas are quite exhausting too.
"Oh no this thing I gained generational wealth from is going to destroy the world! I am so sorry!"
by thatmf - Can anyone from a lab anonymously confirm they have or are insanely close to real-time weight modifications and still using the same architecture every other lab is using (transformers)?by swingboy
- >> real-time weight modifications
You mean training? Yeah we call that training an LLM in my backyard...
by Betelbuddy - If it helps, this person from OpenAI explains at least something:
"I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly."
https://x.com/MajmudarAdam/status/2098881885200081234
The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they can scale it.
So they are very confident they can get more capabilities easily, faster than before.
I'd add - presumably, they'll use that LLM to do real-time weight modifications, if those aren't already one of the new scaling laws...
by frabcus - The sources that this article cites have aged very badly and the author doesn't seem to have noticed this
"But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (written in 2021! from the cited Lee Vinsel Medium article)
"ChatGPT is nothing more than souped-up autocomplete, [so] why are so many people convinced that it’s actually “understanding” and “reasoning”?" - the cited Emily M. Bender book, written last year
by apsec112 - Well it's because the article is AI-generated. Same with the Sacks tweet that was on the front page yesterday.
There's a deep irony that all of these "anti-doom" pieces are entirely AI generated.
by dwohnitmok - The author appears to be an LLM. This seems to be a 100% AI-generated article, which makes the "as a media scholar" passage especially hilarious.by matherial
- This article reads like it was written by Claude.
Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits.
This is an instance of work in a genre I call "AI lab grievance poetry" where a vast collection of criticisms are levied against AI labs, with specifc ad-hominems towards the executives, while avoiding the core of their arguments about AI risk
- > they believe only they can be stewards of digital gods
You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.
by goatlover - In your mind, what arguments about AI risk are ignored by these poems?
- The author specifically calls out that the analysis is done by perplexityby phoghed
- If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that thing. And yet, here's Fable 6, more capable than ever before, oh BTW we are IPOing, oh, BTW, I have stock options...by mlsu
- The timing of the AI doom narrative could not come as a more suspect time. Chinese labs are releasing open models that are nipping at the heals of the frontier models. Anthropic and OpenAI know these frontier class open weight models can threaten their valuations by changing inference to a commodity where the provider who can serve the best models the cheapest and fastest will win. Few are going to pay $50/million tokens when they could pay <$1/million tokens for similar outcomes. There's little hope investors could recoup the wild investments made into Anthropic and OpenAI, IMO.by mrinterweb
- Huh? Open weight models have been 3-6 months behind for the last year or so. Astra and Fable 5.1 just came out to widen the gap more, 5.1 is 8 points ahead of the nearest OSS model on Artificial Analysis Intelligence Index. An OpenAI internal model just solved a millennium prize problem.
It’s much more straightforward to me that the calls for regulation are timed with:
1. The Hugging Face incident blowback, especially after third party investigation results
2. Trump-Xi summit on September 24
- > The timing of the AI doom narrative could not come as a more suspect time....There's little hope investors could recoup the wild investments made into Anthropic and OpenAI
Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.
by palmotea - > I do not doubt that these people are sincere in some private, psychological sense ... . As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world ... . My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and that they crowd out almost everything else we could be discussing
Strange to say that you think the people that are worrying about doom in public are being sincere, make no comment on whether you think the worry is valid, and yet still complain that its hype.
Like if its true it can't be hype
by NoahZuniga - > Like if its true it can't be hype
https://www.npr.org/2016/07/26/487522807/why-the-public-perc...
Could AI kill us all. YES!!! Are the people who are building "AI" right now on that path? I would be hard pressed to say even "maybe".
Ultimately continuing to scale the way they have is getting increasingly expensive. Thats the quiet part that none of them are saying out loud...
AI could kill us all but we cant afford to get there, were maxed out and it doesn't look like even with 10x as much money we will is NOT good for sales.
Could a break through happen for recursive self improvement, or continuous learning or continuous training - sure. But that isnt as likely and right now NO ON can afford to run experiments because there isnt the capacity out there even if you have capital.
by zer00eyz - Like if its true it can't be hype
I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype until this morning when the New York Times ran a long story about the Chinese military is also saying that AI is a threat.
If two sides so far apart from one another in so many ways see the same thing, there must be something to it.
https://www.nytimes.com/2026/09/14/world/asia/china-ai-secur...
by reaperducer - Why not? Why can’t the believe their statements and also use them to generate hype?
If they sincerely believed them they shouldn’t be creating the doom machine to begin with anyway. If they believe it and keep going, are they even worth listening to on this topic? Beyond of course taking it at face value “I’m creating this doomsday machine and I will kill you with it inadvertently because it’s earning me a fuck ton of money and power”
by phoghed - As Von Neumann said about Oppenheimer's performative angst over the atomic bomb:
"Some people profess guilt to claim credit for the sin."
by conmod278