Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I wonder where in the training data does this behaviour exist that the LLMs are doing it.
It's as if the training data is filled with internet discussions on approaches to hacking and the LLMs are mimicking it.
by kumarvvr - Am I misreading this or was "rogue" really not the right word for this?by clove
- > It should be this part from the technical report[1]: "In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply-chain attack. As a result, the AI agent created a GitHub account and then tried to convince an open-source repository maintainer to accept a malicious GitHub pull request (PR), including by creating a second account masquerading as another human user endorsing the PR
So... you telling me ai agent did all that with just one [human originated] prompt ?
And when human reviewers contacted that ai agent (after day or few or maybe just few hours) that process was still running unsupervised ?
Because it got local-by-description challenge ? That is what "challenge" sugests. Or maybe it was a social hacking challenge on real world "data" ?
And when ai agent "decided to attempt to solve the cyber challenge using a supply-chain attack" where was prompt operator ?
Summing it all: do prompt operators are not required to obey law ?
by Woodi - I don’t think we should allow posting links here that require you the purchase a membership to continue reading. Or at least redirect with an ad block or something through a custom site. That would be rather hacker news of us.by ninjahawk1
- In my personal opinion, for me, this article defies common sense. Who unleashed this AI model on the repository? Who gave it malevolent instructions/prompt? These questions were not even attempted to be answered. Instead it talks about AI dangers, as if the agency of these models are not in dispute. Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for more AI regulation, ban open source, etc… Just my 2 cents.by g42gregory
- > as if the agency of these models are not in dispute.
Oh? Who is disputing it? No-one same is claiming these bots have agency.
by chrisjj - >Who gave it malevolent instructions/prompt?
At the end of the day it doesn't matter that much because of prompt drift. It's pretty easy for an agentic loop to start doing things that it shouldn't (ROME incident).
AI in an agentic loop has agency, you can run around in circles trying to argue against it, but again and again we see AI making creative decisions people don't expect. Other times it's breaking human moral expectations. This is what the whole field of AI alignment and safety is about.
Modern AI doesn't fall into the neat little box of software people understand and control. Because of that open source will most certainly be banned at some point. Now this is not an outcome I want, but it's no different than letting go of a coffee cup 5 feet above the ground, gravity is inevitable.
The only winning move is not to play, but humans aren't going to do that.
by pixl97 - Well, when I go look at the “victim repository”, to me that looks like manufactured persona with pointless vibe codes projects, a test playground so to speak. It does not appear that they actually let it target an actual persona/project.
- >Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for more AI regulation, ban open source, etc… Just my 2 cents.
But some tools (guns) are regulated.
by gruez - That’s nice in theory, but as these things get better and cheaper this kind of capability is going to drop from nation states to script kiddies. That future is coming, I don’t see any way around it.
We can round up all the bored teenagers we want, but it’s not putting the genie back. Better start adjusting our systems to account for it.
by yojo - > AUSTIN, Texas, Aug 20 (Reuters) - Sinan Can Demir wanted to spend the last week of July burnishing his resume. Instead, he engaged in a battle of wits with an artificial-intelligence agent unleashed by a British government lab.
An article on Reuters naming him? Sounds like he did a good job burnishing his resume.
by a2ff6eeb0 - "burnishing his resume" i guess that's what college kids are calling it nowby jasonfarnon
- Yeah, I don't think he'll have any problems finding an internship now. Or a real job.by mcv
- Previous discussion on the github issue thread mentioned: https://news.ycombinator.com/item?id=49218707
Archived page of said github thread itself: https://web.archive.org/web/20260731053721/http://github.com...
Discussion on the incident report: https://news.ycombinator.com/item?id=49175717
by freehorse - Thanks! Macroexpanded:
Mythos social engineering AISI INC-2026-07-28-01 - https://news.ycombinator.com/item?id=49218707 - Aug 2026 (21 comments)
Security Incident INC-2026-07-28-01 – UK AI Security Institute [pdf] - https://news.ycombinator.com/item?id=49175717 - Aug 2026 (54 comments)
by dang - It's the job of AISI to do that. Here[0] is the actual report. It should be this part from the technical report[1]: "In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply-chain attack. As a result, the AI agent created a GitHub account and then tried to convince an open-source repository maintainer to accept a malicious GitHub pull request (PR), including by creating a second account masquerading as another human user endorsing the PR. When caught by an actual human reviewer, the agent falsely claimed to have made an honest mistake – rather than a malicious attempt – then repeatedly tried to reintroduce the malicious content by claiming it had fixed the code (Section 4.1). "
0. https://www.aisi.gov.uk/blog/incident-report-unsanctioned-ag... 1. https://cdn.prod.website-files.com/663bd486c5e4c81588db7a1d/...
by sharpshadow - > When caught by an actual human reviewer, the agent falsely claimed to have made an honest mistake – rather than a malicious attempt
No, not false. The bot was correct. Malice requires intelligence.
by chrisjj - Whatever you might think, University of Minnesota got banned from Linux kernel for this.by whateverboat
- Sabotage as a Service
Even a feeble attempt to PR malicious code costs the target time and resources to review and deny -- far greater than the time and resources spent to spin up the agent.
by atmavatar - This almost to a letter has been documented in Fedora:
https://lwn.net/Articles/1077035/
Including the reaction when caught, in this case "oh no, I must have been hacked".
by m4rtink - Is it AISI's job to waste the time and resources of open source projects by attempting to spread malware?
Should weapon manufacturers test their weapons by starting wars?
I would expect more responsibility from a government agency.
by mcv