Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- During the crypto bubble, it was common for "crypto insiders" to talk about how it was poised to transform the entire economy, on the cusp of making all finance distributed, about to revolutionize ownership, etc., etc. .
The people in the most inflated parts of bubbles don't tend to have the most clear eyed assessment of the real impact and potential of the dynamics contributing to the bubble: their perception is warped by the bubble, and they cannot help but see everything filtered thru it.
by abathologist - It sounds like they're all trying to make a case for why the government should buy them out as a new kind of manhattan project.
They know the market isn't going to buy in for their big payout.
by ChiperSoft - > In general, the more senior the employee, the more concerned they are.
In general, the more senior the employee, the more equity they have in the company.
by roarcher - > Maybe you’ve already heard about the guy who resigned from OpenAI.
While he worked at both OpenAI and Anthropic, he resigned from Anthropic. Mistaken reporting in the first few sentences, definitely a horror concept.
by spindump8930 - While the persons mentionned in the articles are indubitably most of the most well-informed people in the world. They are also the most likely to have internalized that their work is leading to superhuman intelligence/AGI. But is it really realistic ?
So they have a strong bias towards imagining the most catastrophic scenario.
by cors-fls - I may have missed it in the blog post here, and I generally agree the pace of development is being set by financial incentives and not safety or lack of harm, but has anyone in a high position stated _how_ models could cause human extinction?
I assume the obvious answer is “we plugged our military’s weapons control platforms into this model and it fired all the nukes” which is clearly enough. Is that it or is there some other path to global extinction someone is concerned about?
For the record, I’m not an AI fan person annoyed at naysayers. I actually find AI to be a monkey’s paw instead of the genie in a bottle most of the time when coding, and an intrusive feature I neither want nor use most of the rest of the time.
I ask because I see these posts and obviously an AI plugged into a network of doomsday weapons would be a huge problem, but then the author only talks about cybersecurity events and model misbehavior. Both things are valid concerns, but I would like to hear more concrete information from people who are voicing their concerns about what they foresee happening even if it is just lifted whole cloth from the movie War Games.
by mrzimmerman - One thing I definitely do not understand about this discourse is that the models that are good enough to self-replicate can’t survive on normal machines, e.g., the models can’t hide on some random server.
So, if it is as dangerous as they say it is: there is a single physical source of this danger, which is OpenAI/Anthropic servers. If it is this dangerous, they can just turn it off.
Instead, we just keep pretending that the models that attacked HF were hosted or replicating on HF hardware. Not the case! They infiltrated it, but were hosted elsewhere.
by stephantul - I’ve noticed a weird thing about the discourse around this, that it can only be a marketing thing or a true belief, as if everyone working in AI has a monolithic opinion. The tweet kicking off this article has that assumption “it’s not a marketing thing, many people truly believe it”.
It’s not an either/or. Many sincerely held beliefs can be used by cynical actors in cynical ways. It can be both a cynical marketing ploy by the C-suite and a genuine fear.
Personally, based on my experience with AI, the only dangers with the current technology is a) massive loss of employment, b) deployment by humans to control critical infrastructure that LLMs shouldn’t be in control of. Both of those are real, society changing risks that framing the issue as “once we reach the singularity everyone could die” minimizes. Obviously their concerns are possible, but >10% is a random guess. The loss of jobs and how that affects an already K shaped economy is already happening, and nothing is being done for that
by klik99