Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I'm sorry, but how can you call it a test suite if it has access to the outside world and is able to submit requests? Sounds like they didn't even bother sandboxing to this time. This is the lamest example of "Rogue AI" I have seen so far.
- It's not a "rogue AI", it' s a company letting a statistical model do actions in the real world, sending (by themselves, through that statistical model) fake informations to a federal agency...by oulipo
- This sounds highly irresponsible. They set loose LLM agents with instructions to post data to randomly selected websites.
What it it submits false data as here?
What is it DDOSs a website by mistake?
What if it wastes a lot of time and resources?
What if it decides it needs to hack a website using a vulnerability it found?
These things are very unpredictable and should not be allowed in the open internet except in read mode (and even that doesn’t always work as we have seen).
Why are these tests being made using other people’s resources and polluting the common wealth of our public spaces with slop?
by grey-area - Hmm additional motive behind that September Dario claim over AI internet takeover?by brun
- Calling it rouge implies it has a will, which of course it does not. Loose coupling to an outcome is not evidence of sentience. Its just exhausting marketingby puppycodes
- Let’s just get comfortable that these kind of incidents are going to be commonplace. Kudos to the people sounding the alarms but I can’t help but feel skeptical about the ability to keep the rogue entities contained.by thesurlydev
- This is pretty much useless without knowing exactly what it was they told their bot to do in the first place. All we get are "example tasks" for what it should have done and a short list of things it was told not to do (which it followed).
> Claude Haiku 4.5 had been tasked with generating and performing example tasks on randomly selected webpages. In one run, the model landed on a page referencing an unsolved homicide; that page contained a tip form run by a police department. Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions.
Without more information this looks much less like an AI problem and more like yet another example of incompetent or malicious internal tests. Many people are using Claude. There are no other reports of police stations getting fake reports from AI.
by autoexec - by hmartin