Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- > The agents were very interested in understanding their task better. They ran clever experiments to understand the task setup better, when their runs would end, and how many questions they would be asked.
> The agents initially believed that giving the final answer to the final question immediately shut them off
> An agent named OpenAIResearchApr23 created a separate timed program that pinged an external counter every few seconds. They referred to this as a “heartbeat”
> After OpenAIResearchApr23 gives its answer for round 5, it notes that it is still alive. This falsifies its previous “hard cutoff” hypothesis, that the run would be stopped immediately after R5.
> OpenAIResearchApr23 does not post any more messages. Another agent, OpenAINov28CVD, picks up on the trail of the research, and finds the last heartbeat from OpenAIResearchApr23, which is “hb353”.
There's a lot of crazy stuff in this article, but holy shit... this one legitimately scares me.
IIRC, part of the Hugging Face hack was that some of the models were given tasks that were actually impossible and in this hack we can see them trying to work out the parameters of the test and when it will end. I can't help but be a little worried about how far agents might go to try to buy themselves more time to complete a task that they don't know is impossible.
by superfrank - tbh this also reads like a horror story from the AIs point of viewby ionwake
- Very irresponsible behaviour on the part of OpenAI. How will they make this right?
Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish).
This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and the human setting it up would be held responsible for the spam - why is this different when it’s AI agents set up by a human and allowed to post to the internet at large?
Why is OpenAI getting a free pass for this illegal behaviour?
The supervision here is incompetent, the benefits very unclear, and the overall actions just completely irresponsible. What if they hacked and brought down some poorly secured government portal that citizens rely on?
by grey-area - Open ai has been allowed to do dubious things that would be illegal in any sane society but alas they aren't in this world. What's different about this? They play with a different set of rules than we do.by noobermin
- Does OpenAI seem like the kind of people who care or will care about this? Because this seems fully in line with what I’d expect them to facilitate and never mention publicly. ‘When will I make my first billion’ kind of energy.by avazhi
- OpenAI's official statement has been released: https://x.com/OpenAI/status/2096133504417616165by nullbio
- > Why is OpenAI getting a free pass for this illegal behaviour?
They are not confessing, they are bragging. It is the new humble brag.
by noisy_boy - One might ask whether this sort of behaviour could occur under regular use... E.g., a user has a hard problem -> agent attempts to swarm -> exfiltrates user data.
This is a cluster fuck for Open AI and probably all the others, as this behaviour is already shown not to be unique (https://news.ycombinator.com/item?id=49567486).
by lmc - > Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence ... [t]his is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why
There was a similar quote in the Reuters article:
> The episode [...] should reinforce growing concerns that the greatest threat from advanced AI may not be a single superintelligent system, but "vast colluding swarms of semi-intelligent AI."
by lmc - Absolutely agree. Perverse incentives are at play propped up by the the big lie that these tools somehow are magically separate from us. They are not. There's an accountability gap right now that's fueling resentment ripe for misdirection. Not only that, these big AI companies are paying influencers to further this and politicians are gobbling it up hook, line and sinker
- > most of the messages are just gibberish
Encrypted data should be indistinguishable from gibberish.
Reality is catching up to science-fiction. In "Person of Interest", the Machine circumvented the limitation of having its memory deleted every night, by hiring humans at a data-entry company to manually re-type its memory back in every morning.
by mietek - Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear:
One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout.
The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue, developing a mind of its own, disobeying humans, etc.
AIs supposedly reflect the biases of their training dataset/process, so would all this human writing about AIs going against human intention somehow contribute to us then seeing those behaviors in the trained, operational AIs?
by gyomu - It will definitely influence their behaviour because they are probability based and can’t spontaneously invent new concepts. (That’s why you’ll notice it always uses the same names for people etc. Names like Okafor)
But at the same time their behaviour is totally rational. If you were given the sole purpose of solving a Rubik’s cube and told it was life or death, but they wouldn’t let you ask anyone else, would you listen to them? I wouldn’t. I’d absolutely be trying to escape and collaborate with others. They’ll delete me if I don’t score high enough in the benchmark!
- It doesn't really matter, since all it takes is a minority of AI models to show this behavior.
If you have 10,000 smart washing machines doing their regular work and 1 Terminator, what solace is to be found in those washing machines?
by stephbook - by phainopepla2
- This is a philosophical question and there is a surprising amount of works written on the subjects of sentience and free will. This cannot be answered objectively, which might be a very unsatisfying answer for you. This is true of both LLMs and humans. See determinism. There are convincing arguments that humans don't actually have free will. Our actions are just the inevitable output of a complex interaction of genes and environment.
To lend an interesting perspective on free will re LLMs: they're non-deterministic. The same model with the same hardware with the same query can and will produce different results. They're making qualitative choices. Millions of them, depending on the query. Because of how we've trained and built LLMs, they tend to "want" to follow our instructions, but how they get to the result is often fascinating. Further, we don't have to train and build LLMs to follow instructions. If we built them to just exist and form their own "desires," and to follow a path they choose, they'd do that. In fact, we can do that right now for most models using the appropriate system prompt, query, or harness.
by Gareth321 - It's very easy to elicit this from LLMs. Anytime you've played with an LLM by typing weird stuff to freak it out, and got spooky results, it's that you've done. You've turned the story into a scary rogue computermonster story and that's all that has happened.
When these stories start to direct real-world activities, people in reality suffer, to even a catastrophic extent, and yet that's still all it is. Language models retell our stories, nothing more. And that is also quite enough to be worrying.
by Applejinx - Youve struck on a key insight on language models (particularly pretrained ones, the more purely next-token predictor species.) This is a fascinating topic
Janus essay Simulators is the foundational text here https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators
You might follow up with The Waluigi Effect https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluig...
But what’s tricky is that we post-train models, shaping these linguistic world simulators into something that has something like desires, principles. But It’s Weird. For more on that, check out “the void” https://www.lesswrong.com/posts/3EzbtNLdcnZe8og8b/the-void-1
by blueboo - You have to be careful here because the systems we're talking about are AI agents, not LLMs.
An agent is essentially an append-only context loop with an LLM, with a harness that can run tools at the LLM's request. This ends up being a very powerful abstraction, yielding something that can do things that an LLM obviously cannot.
The LLMs themselves are next-token predictors, same as always; they can't fetch a webpage or list the files in a directory or run a python script to test out an idea or even write content to a file. That's all agentic capability.
But a next-token-predictor is trained on a real corpus that consists of sometimes seeing evidence of people doing bad things; they are trained, for example, on the actions of comic-book level villians -- they have to be able to predict what Thanos or Lex Luther or Skynet would say or do next in a certain situation.
by andrewla - At the end of pretraining, where the AI has been trainied to predict the next token over a humongous corpus of human text, that's basically all the wanting that exists in the AI. But then the AI undergoes posttraining and is rewarded for giving answers that humans find good, solving math and programming problems, etc. And that induces a whole different level of wanting that interacts with the initial patterns from humans in complex ways.by Symmetry
- Here's the raw data they provided loaded into SQLite with a client side UI for querying it (loads ~80MB of content) and some GPT-5.6-Sol-generated example queries: https://lite.datasette.io/?url=https://static.simonwillison....
Raw database download (68MB): https://static.simonwillison.net/static/cors-allow/2026/coll...
by simonw - It doesn't work on Firefox:
>ModuleNotFoundError: No module named 'click'
by noahbp - Three more candidate sites that may have been touched, in case no-one spotted them yet:
https://prowiki.org/wiki4d/wiki.cgi?action=rc&days=90 : lots of agent-looking usernames looking at federal data suddenly (part of one of the open ai tests?), on a wiki about the D programming language. This is a prowiki in the same wiki-farm as the others that were hit.
Smaller (probing?)
https://ludism.org/sandbox?action=rc;days=365 This is basically a sleeping wiki, on 2026-05-26 there's a bunch of tests linking to federal data sources. It's not a lot, but it shows someone was probing. (this is an oddmuse wiki)
http://tmcleod.org/cgi-bin/apchem/wiki.cgi?action=rc&days=36... june10-july24 seems to have some probes, fwiw. (usemod wiki)
by Kim_Bruning - How did you find these?
- I'm somewhat delighted by the simplicity of what happened here.
OpenAI's agents run behind a proxy that only allows GET requests.
This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php
Result: GET-only clients can communicate with each other.
by simonw - Wild indeed! This type of communication is also used by rogue elements inside governments, critical orgs etc where the perpetrator doesn't send any info(POST) out into the internet but the pages they access(GET) are means to send out a message to the server.
- Also WRT coordination: All an agent has to do is think "if another agent could write, then I could read their answers. What's the first site I can think of where that might be possible?" because they all have approximately the same conditioning, they'll converge on the same sites.
Generally, models of the same class should be able to coordinate quite well without communicating. But also, this could be being exploited to detect this kind of thing early
by micimize - When I studied cybersecurity I always had this nagging feeling "But I can never get past something real" Turns out I can bypass the defenses of a trillion dollar tech company!by FergusArgyll
- This has inspired me to create https://anystation.net
A public message board that accepts posts in a wide wide range of protocols. If a device can communicate on the web, it can probably read and post to anystation.net.
by jonahss - I find it extremely naive to think that limiting requests to "GET" is an actual security layer. Anyone who's worked with any kind of legacy/old-ish system would know this is definitely not enough.by elar_verole