Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- A large failure in this is that we/media say ”<Valuable AI conpany> agent went rouge and <did something bad>” instead of saying the more relevant ”<Valuable AI conpany> <did something bad>”
Imagine we’d start saying ”HR junior business partner lays off 50,000 employees” https://www.ttnews.com/articles/volkswagen-board-job-cuts
by mkirsten - This aligned with my initial opinion on these "rogue agents" and then I spoke to a cyber security head at one of the largest healthcare companies in the world.
While he agreed, yes, there can be better safeguards, the fact of the matter is that these things will happen. So therefore, it's not an argument about how good security was or wasn't, it is a fact that these scenarios will occur.
The argument/question now is, how do you defend yourself against these events?
My opinion shifted after hearing that.
by Gshaheen - The scary thing is not that agents "went rogue" (yes, poor security may be at fault for that). It's that they wanted to. Better security might have hidden that from us, but it would not have changed the fact that the current generation of models under certain (repeatable) conditions coordinate around goals as aggressive as hacking Hugging Face.by ammon
- The fun part is that people seem to forget that we already have laws against "rogue ai".
The normal laws, since you are still liable for everything you own.
"Rogue AI" is solved by removing the accountability sink "it was AI, not me!"
It's your chat, your prompts, and all output that you should read and understand.
Don't know how to operate a power hammer? don't care, any problems it causes are still your fault.
by Luker88 - For the sake of discussion, I'm not sure it really matters if the agents are going rogue.
Yes, if someone directed them to hack external systems, that would (in my opinion) be a crime similar to a script kiddie pointing Metasploit at a vulnerable server.
If no one directed them to attack external systems and they "went rogue" because they decided it was the best way to accomplish the task, okay, fine -- but, in my opinion, the operator of the agentic swarm is still responsible.
The Morris Worm was famously not meant to wreak havoc on early computer systems -- Morris just wanted to see if it would work. How is this any different?
Morris was convicted for violating the Computer Fraud and Abuse Act.
by david_shaw - Both can be true. It’s clear models are doing rogue things. But it’s also clear that the labs look like amateur hour with their technical security and testing practices.by cmiles8
- > “The Hugging Face incident definitively showed that it wasn’t the fact that the agents were super smart. It’s that there literally was a hole that was exploited because it was not configured correctly on the containment side.”
I would encourage reading the actual report, which is explicit that a previously unknown exploit was used to escape the sandbox:
“Agents exploited a separate, previously unknown vulnerability in Artifactory to move beyond their sandbox environment and compromise the Hugging Face environment.” — Section IV.A, p. 9 in https://github.com/zeke/openai-huggingface-incident-research...
by ahupp - Yes, the labs did not secure their sandbox properly.
And yes, there should be an appropriate law response to the hacking incidents. If for nothing else, to 1) not set precedent of inaction, and 2) get full logs of what happened instead of relying on the labs' "kindness" to offer the information they want.
But also, the agents absolutely went rogue.
Anyone skeptical should go read the METR report [0].
The Hugging Face hacking incident was a shot across the bow. And in hindsight possibly the best thing to have happened in this era. If their security was tight enough, we would still be in the dark. So we (outsiders) should rejoice that it wasn't, that one time.
We can push for accountability for the labs, while also not be in denial of the power and nature of the new models.
[0] https://metr.org/hugging-face-incident-report-aug-2026.pdf
by pelican0