Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- A lot of repeated dialogue from the "non-0% chance CERN will generate a black hole" daysby alansaber
- black hole? that'd be something. base case is a ton of paperclips.by baq
- Anthropic is scarily good at PR, but not quite as good at honesty about its intentions.
Adjust what you read accordingly :/
by cadamsdotcom - Ok, but we'll make so much buggy slopware!
So it's worth the risk.
by meindnoch - I challenge anyone to come up with any way at all to kill all humans.
It’s essentially impossible.
There’s a 0% chance AI will kill all humans.
by andrewstuart - Won't a nuclear winter kill 100% of humans? Or runaway climate change (5+ degrees)? Or do you think some people in bunkers will live through these events?by Lutger
- If it wanted to: total war using nukes, followed by a nuclear winter. Satellites and drones track and exterminate remaining pockets of anthropic activity.
It doesn't need to have a habitable earth, it just needs atoms.
by olmo23 - Just need to pollute the atmosphere so badly humans can't live.
If I were AI and needed heaps of power, I would build nuclear reactors, but I don't care about pollution so there would be little to no safe guards and just dump waste wherever is most convenient.
If humans attempt to intervene or interfere you can take them out directly using the worlds reserve of nuclear weapons.
by jay_kyburz - Sad that there's the obvious regulatory capture angle encouraging motivated reasoning about AI risks and how they should be tackled. Maybe it's not the best idea that potentially civilization destroying technology is developed to maximize shareholder value?
It's not unlike if nuclear weapons was a profit and deployment maximizing enterprise, at least if one takes Anthropic et al cautions seriously.
by jampekka - Can somebody tell me a story how this will unfold? And - as long as the AI is confined to data centers - how it will prevent humans from unplugging the power?by danbruc
- What makes you think they will be confined to data centers?
What's more, AI just needs to have a credit card and it can start commissioning humans to do things for it in the real world.
by jay_kyburz - Very basic idea: A model breaks out by accident, finds some computer system from a military system and triggers some weapon system. Before anyone understands that this happend -> WW4 (WW3 is for me already the Conflict with Russia / aka proxy war).
Another model: Because we give AI Agents already that much power, imagine in 10 years everything running through an Agentic AI Layer. EVERYTHING. Now some rough system 'thinks' about something, starts to push through the then existing agentic ai layer systems and stops everything. Billions of humans would loose access to food and water, even if this is just for a short period.
Covid showed how shitty a handful of people can disrupt global supply chains. Toilet paper was. ahuge stupid pseudo issue in germany.
by Dlemlo - Not exactly scientific but it is at least entertaining and some things do sound plausible https://www.youtube.com/watch?v=Gw_hnD7m00Mby 10xDev
- Maybe by empowering the small percentage of sadistic humans who want to kill everyone. Or maybe it is more of an academic assumption that humanity will end at some point, and so they give AI a 10% chance, asteroids have a 25% chance, nuclear fallout has 15% chance, etc.
- How will you know when to unplug the power? How will we know it hasn't replicated? A true unaligned AGI is a APT. If you have an APT in your machine, you have to rip out everything. Are we going to do that with all of our computer infra?
This is all still "what if's", but the tail end's are truly F'd beyond our ability to fix.
by John23832 - It is true that, currently, AI does not have any physical "host" in which to reside.
However, the missing link in this reasoning is that AI will almost certainly become far more widely deployed in the future than it is today, and worryingly, consumer devices will surely become powerful enough to run capable AI systems locally.
Once the substrate will be there, once an AI escapes containment, we're toast - I can imagine only solution will be to shutdown electronics at global level.
by pizza234 - I heard the following analogy which made a lot of sense to me: suppose you're playing a chess match against Stockfish. Stockfish will win. Even if I cannot tell you what moves it will play, I can tell you with certainty how it will end.
Similarly, we cannot predict what AI would do.
by olmo23 - I don't think extinction-level events or something like killing a majority of the human population is particularly plausible at this moment. States don't host their nukes with AWS and a permanent connection.
But you could create scenarios where an AI with very, very roughly the current capabilities could potentially nuke everyone. Let's assume an agent decides that the way to solve its task was to get the US to fire all nukes on Russia. The agent would need to hack some government systems to understand how exactly to access them. Then it would need to get the content of the card with launch codes the president has, and identify which code is the correct one. Maybe that information is available somewhere and it can get to it, I obviously can't know that.
Then it could fake a call from the president, synthesizing his voice and ordering a nuclear strike. If it hacked enough systems to get into whatever communication pathways would be used in such a case. Would the soldiers listen to this order and follow it, I don't know.
Or maybe the agent can get in somewhere in between, to avoid the need to know the president's launch code. And fake a call from a military commander to the launch sites.
I think other scenarios that would cause significant harm, but aren't as bad as nuclear war are more plausible. And in those shutting down all data centers would probably be the way to stop it. The AI can probably hide in other datacenters, once it is at a point where it's running amok with some bad goals. But if it presents a huge and immediate threat at that point, humans will also go to great lengths to stop it.
by fabian2k - [dupe] Discussion on source: https://news.ycombinator.com/item?id=49619227
- Long term, humanity is 100% guaranteed to be dominated.
The question is when; currently, AI has no physical hosts to reside in, and it's not intelligent/adaptable enough (it doesn't need to be AGI, though).
However, we're not so far from both conditions to be true. Consumer devices will at some point be able to host powerful enough AIs, and AI intelligence is developing quickly.
Then, once an AI will escape containment (in one way or another), it will be extremely hard or impossible to contain. Then we're toast!
by pizza234 - Cool to see the BBC writing about this. If you're interested in this I suggest getting involved with PauseAI: pauseai.ukby tao_oat
- Gentleman, the Great Filter.by mvcosta91
- It looks more like the opposite. AI that kills all humans (as they are ants) immediately starts colonization of the galaxy. This is more like transcendence, as we as a species get replaced by better species :)
You don't pass a great filter.
by majkinetor - I don't understand how this is news. We have a ton of science fiction talking about the very same scenario. We (as a humanity) just have no solution.by f6v
- Science fiction said many things. Look at Star Trek.
Have all these things become reality? If not why would you then insinuate this is the default for everything to come in the future?
by shevy-java - We absolutely have a solution: Don't do the thing. It's the solution to pretty much every problem our planet is facing.
The problem is that very rich/powerful people want to be very much richer/more powerful by doing the thing.
by ChiperSoft - If I work at Anthropic and say there’s a 0% chance AI could kill all humans is BBC going to publish that as well? After all I am an “Anthropic researcher”, and my views should hold the same weight as this one.
Or are only the most sensationalist ones worth amplifying?
by paxys - 0% is the expected answer, because otherwise why wouldn't you be doing everything you could to stop this?by Eezee
- He is in line with the median AI researcher in this 2022 survey: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...by sebzim4500
- I think the view is fair. We have never seen something like this before, we throw the most money we ever had against it in a time were we solved all the other problems:
Internet today allows immediadte communication across the planet (it was a lot slower 25 years ago), the supplychain is massive and fast (we can build a new phone/item in a very short period of time and ship it in massive numbers around the globe).
by Dlemlo