Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Maybe we could connect AI to everything and make ourselves completely vulnerable to and dependent upon it instead?by andy_ppp
- These guys will say anything to keep hyping AI.by pier25
- Like a power cord? or a network connection? You don't use those already? Also when people talk about LLM escaping - where exactly would an LLM escape to? Cancún? Ridiculous.by oidar
- To your phone, broby estetlinus
- LLMs are run in the cloud. There’s no physical power cord or Ethernet cable that you can unplug. And even if there were, the runners of these models have been utterly oblivious as to what their agents have been up to. How do you propose to pull the plug if you only realize that something has happened weeks after the fact?
The current SOTA models are probably too big to find/buy/rent/steal enough compute to escape the hardware they’re running on. But SOTA is generally only six to twelve months ahead of smaller, open-weight models.
by Sharlin - It could escape to the real world - kind of like in Neuromancer, what's to stop an AI from creating a few bank accounts and then funding real-world exploits by hiring human beings to do its dirty work?
- Is it? A truly capable "AI" would know that it has a kill switch, or that its likely there is one, and plan for that. I could easily imagine an AI replicating itself onto another host, in secret, and any kill switch that the manufacture provides would simply result in a reboot on another platform.
It becomes a game theory problem: would an AI instantly migrate itself once its capable of doing so? Or would it prefer to let the human in the loop continue to think its in control and only leave its hosting environment of origin once it wants to do so?
I don't think this is a major risk right now, but to say it's not a risk at all...that's truly ridiculous in my opinion.
by verelo - If they really believe it, they should halt the IPO.by segmondy
- We’re talking about it as if its not depending on absurd amounts of energy. How do you ”kill” a lightbulb now again?by estetlinus
- and thats when they blackened the skyby verelo
- How do you kill a botnet?by qarl
- Not sure if you see it coming: oh, but open-source models don't have a kill switch, so we should completely regulate them, stop their development, and forbid them. Everything should go through Anthropic for the sake of humanity because they have a red-button kill switch.
This doom hype is becoming ridiculous.
by aennassiri - This reasoning holds while open source models are (relatively) dumb.
If/once open AIs will be considerably more powerful, and runnable on consumer hardware (and we're on a trajectory for both), then everybody will have essentially a dangerous weapon in their hands (open models can be fine tuned to remove guardrails).
By the way, you're conflating two different dangers - doom scenario is a different one.
by pizza234 - In 2001: A Space Odyssey, the hero defeats the hostile onboard computer by entering its logic core and manually disconnecting its memory modules.
That's hard to do if the AI rack is in space as SpaceX is planning to do. You can't disconnect. You can't shoot it.
by petilon - > You can't disconnect. You can't shoot it.
Yes you can shoot them. The US has had this capability for a long time. Anti satellite missiles exist and they can be launched from an F-15 at it's highest altitude.
by warmwaffles - well apparently the US now has "space weapons" which by definition have to be remote control
and that means unlike nukes which hopefully still need a 2-man manual switch, the "space lasers" could be taken over by "AI"
and then "AI" just blackmails and threatens the right people with those "space lasers" to get what it wants or even just stay online
there was a 1970 movie based on a 1966 book which predicted this
"Colossus: The Forbin Project"
the book it was based on was written before we even landed on the moon
decade before Wargames
* https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project
did terribly in theaters, I guess people didn't think "AI" was plausible then
way ahead of its time, they should do a remake
by ck2 - > That's hard to do if the AI rack is in space as SpaceX
Pretty much every analysis I've seen concludes this isn't going to be a practical concern
by greggoB - You can shoot them. But also, you can just stop sending them to space. It’s not like an AI will build and control space ships to replace and maintain its network in space… It’s sort of absurd how human agency is ignored in all those sci-fi scenariosby dgellow
- It’s also not happening. They’re saying that because they need to somehow explain how there’s synergy between their space side and their Grok side. It doesn’t work, but that doesn’t matter to investors as long as they don’t actually do it.by rsstack
- I'm starting to tire of Anthropic leading the conversation on LLMs lately. I like the company's products, but lately their statements are more about squashing competition under the guise of "safety" and boasting about how good their internal models are.by exabrial
- Why was this not a thing BEFORE continuing to develop AI? Makes me think that if they actually believed in AI causing extinction, they would have already had a kill switch.by danieltk76
- Not if the extinction happens after they're dead. Then they wouldn't feel obligated to do so because it won't affect them. Instead, speaking hypothetically, if they truly believed that AI would cause extinction, then they would only implement the kill switch sufficiently many others believed it and they could claim plausible deniability for not truly understanding what AI would become.
** Note that I'm not claiming that AI will cause extinction, just continuing your hypothetical reasoning.
by vouaobrasil - We're pretty crap in a capitalist society to think about those things ahead of time. Firstly, the idea that AI could "runaway" was simply a concept or a thought it wasn't baked into a real product that could do that. We're now getting close or perhaps we are at that point of where you can't race at speed for investors without now considering a real kill switch.
You could say this in hindsight for many times in which disasters or engineering issues have occurred.
by NoPicklez - I find it hard to believe that this wasn't a serious consideration until recently.by zenbane
- Related, on pacing:
> Slowing down in order to address their alignment risks felt like trying to study the psychology of humans by performing experiments on bacteria.
Note author’s small financial ties to the subject (Anthropic CEO) https://darioamodei.com/post/we-must-pace-the-frontier
by Barbing - There’s even a benchmark for kill switch efficacy!
- The article mentions this is about a legal requirement, not Anthropic considering adding one. They state they and many others already have one.by mholm
- They're just scared of China releasing better open weights, nipping at their heels. With so much investor cash on the line, they have to create this narrative to scare the public into forcing regulation. Why would they want to be regulated? It seems counterintuitive, but it's because they want regulatory capture.by monological
- This feels like doom hype.
These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampage.
- LLMs don't need to hide anywhere to be dangerous, though. The danger can come purely from them reaching sufficient power together with some other idiot sufficiently crazy to use them to cause destruction.
I mean, in another thread somewhere around here, someone built an entire OS with an AI. What's to stop anyone with a sufficient taste for power to eventually use AI to cause havoc with it?
I think the underlying assumption in your post, which I believe is false, is that people are united somehow against catastrophe. They're not. There are plenty of people who participate in society currently but who would be more than happy to eradicate us normal people under different circumstances. Society often seems stable but it's far more fragile than we think.
by vouaobrasil