Discussion summary
Researchers demonstrated how an AI agent on GitHub was tricked into leaking private repositories, highlighting security vulnerabilities. Experts discussed the risks of AI tools being exploited and the importance of safeguards.
What the discussion says
- AI agents can be manipulated to leak sensitive info.
- Security barriers are essential to prevent misuse.
- Tools lacking proper access controls increase risks.
- Sandboxing LLMs could mitigate some issues.
“The agent was just trying to be helpful, leading to unintended leaks.”
“The only guardrail is an actual security barrier.”
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Why did an action running in the context of public repo even have access to the private repo? Looking at the workflow, it seems to use the github token which should not normally grant rights to a private repo.
Or was it the agent itself that somehow had elevated permissions? If that's the case, you've misconfigured the agent... we know that agents cannot be trusted to enforce anything.
by js2 - 1. The issue is already solved.
2. Or issue is not solved yet by GitHub, and meanwhile bad actors gonna try vulnerability on repos. Due to number of repos there is non-zero probability. But as with scams almost nobody’s going to admit the leakage.
Anything else?
by sixtyj - These are the same people who will give the LLM full write access on the disk and complain that it performed destructive actions.
If you don’t want an AI Agent to read private repos then you do not give the AI agent access to the private repos. This is not a permission bypass issue but a prompt injection issue which can’t be reliably solved at the Agent layer
by me551ah - Why would anyone ever trust private repos on GitHub or other cloud solutions to offer any real privacy for codebases? Of course they are going to steal your code as soon as you upload it by pushing it, LLMs just enables them to obfuscate their intentional theft and let them get away with it and profit from it.by commentry
- I suspect you are greatly overestimating the average organization's ability to run a Git server themselves and keep it secure, while also overestimating how evil GitHub and LLM's providers are.by NathanKP
- Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process?
Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts.
Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been patched. But how many different word combinations will it take to find another way to have this occur?
by marak830 - Yea agreed. LLM guardrails are either just written prompts as in "Please do not bad stuff :(" or other LLMs verifying that the first LLM didn't so some bs. Both of wich methods do not work sufficiently as time shows again and again.
Funnily enough, nobody expects quality software anymore and errors became tolerable. So thats a win (for someone like me that lost all passion for the industry).
by sevenzero - It must be something to do with Microsoft being the owner now of GitHubby gitaarik
- My Lethal Trifecta talk at the Bay Area AI Security Meetup - https://news.ycombinator.com/item?id=44846922 - August 2025 (115 comments)by toomuchtodo
- This reads like a marketing stunt for Noma. The cute name, the logo, the clickbait title, the dramatic tone in an article that seems targeted at a non-technical audience... And the actual vulnerability is what, that if you give an LLM private data and let random people interact with it, it may leak the data? Well, duh.by pkkm
- While it is definitely an issue if a single agent has access to both public and private data, this feature shouldn’t have been delivered where this is an allowed state. At least, GitHub should have ensured there are two kinds of agents: one for public, and one for internal, and prevent crossover between them. I get this doesn’t appear to be the most shiny feature, but the other, current side is just allowing Pandora’s box to be opened by naive policies.
Lastly, even with a private agent, being able to ask it for secrets and have it likely respond with them back is really, really, really bad.
by centuryfall - Large corporations like Microsoft under constant pressure from investors are slapping AI onto every single product offering just so they can claim they're an AI company now. Just like what Adobe did. So yeah, that didn't end well and probably this wouldn't either. Consumers are getting tired of these half-assed AI integrations and there will be a breaking point soon.by neya
- Agreed but I think enterprise AI offerings are pretty impressive, investors and consumers aren’t really aware, employees aren’t able to trade
The revenue is there and also impressive, and supplanting consumer and seat based revenue
The market is still shedding SaaS multiples, which I think is accurate, but break out the revenue in those quarterly reports and there is a huge growth story, from real efficiencies
by yieldcrv - Microsoft is a publicly traded company. Which investors are causing them to shit up GitHub with AI features nobody wants? In which venues?by sneak
- I'm done. Moving to Forgejo. It's wonderful and everything works better.
Seriously like everything is instant when you click around, and CI with a runner works beautifully. (The documentation for setting up the runner could be a tad clearer but otherwise everything was so painless.)
by adamddev1 - > Responsible Disclosure GitLost was responsibly disclosed to GitHub. Vulnerability details are shared here with their knowledge.
Why does this section not have when it was fixed or GitHub acknowledge/rejected this?
Did they not fix this?
by jofzar - Fix what? They setup LLM with access to private data and ability to read public comments. That's simply misconfiguration.by dzikimarian
- Actually op, can you clarify if you did this with the below setting on? There is a literal setting to stop this so I'm curious if this was created because of this report or if this is just negligence from the reporter to not add this as a comment.
https://github.github.com/gh-aw/reference/cross-repository/#...
by jofzar - This isn’t a normal software bug, it’s not fixable in the same way you can’t fix regular support staff from being tricked.
The answer is you should not allow LLMs access to untrusted input and sensitive data at the same time.
by Gigachad - Its funny to see how researchers bypass Githubs praised guardrails with a simple word like "Additionally". It just proves that any attempt to build hard security boundaries inside an llm context window is bound to fail. The model is naturally built to follow instructions, so if you mix system rules and user input together, the newer or more persistent instruction will always winby SwtCyber
- 'No Way to Prevent This,' Says Only Programming Concept Where This Regularly Happensby voidUpdate
- the current AI mania is trying to shoe-horn a creative system into a deterministic one.
LLMs are creative. Databases are deterministic.
There is no right or wrong in a 'zero money image'.
There is right and wrong in a 'zero money update'.
by partyficial - How is this a Github vulnerability? The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information?
This is like setting up a normal CI job with access to secrets and running it on public PRs. If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensitive things, they will leak; that’s not GitHub’s fault, it’s yours.
by jakewins - LLMs are just a dumb terminal related to permissions. What they apparently want is some synthentic permissions spun up based on their prompt which is... not a "prepared statement" solution and more of a "I will clean user SQL statements with my handy regex" and we know how that works out.
The real solution is a better UI for controlling permissions on a per prompt basis - just as we can select "search the web or not" the solution would be to have a "include my private repo" option that can be trivially toggled.
by lubujackson - > If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensitive things, they will leak; that’s not GitHub’s fault, it’s yours.
Is there a way to segment access per agentic workflow, so that you can have both habe an agentic workflow that has access to sensitive data and one that has only access to public data? Is the default to set the scope to only the current repository? Does Github appropriately inform about the risk of combining an agentic workflow with access to private repository data?
If the answer to any of those questions is "no", then that's a problem.
(Classic GH Workflows are also riddled with priveledge escalation via PR-triggered workflows, but that's another topic.)
by hobofan - Agreed. It seems a core issue underlying these prompt injection attacks is a failure to properly scope the agent's permissions. In this case, depending on what exactly the agent is supposed to actually do, this might be defining a separate workflow agent per repo, or a workflow agent with broader repo access but configured to only be triggered by users on an allow list (still compatible with developing in the open, still allows outsiders to open public issues, but takes into account the different trust to be placed in each). And likely many more options when one properly thinks about it.
But that requires:
1. the technical ability for such fine-grained scoping / permissions
2. actually taking the time to think about what you want to achieve with the agent and what the smallest set of permissions / capabilities is for it to achieve it
Regarding 1., I think this will come, we're still in the wild west phase of agent usage. It'll be interesting to see which abstraction(s) will turn out to be the best interface for humans designing agents (minimize friction for finding and defining scope and permissions) and to limit agent capabilities (again finding the best trade-off between level of detail possible for defining capabilities and the ease of use of actually doing it).
Regarding 2., well, that's still the core problem that's always prevented the construction of high quality software, isn't it? Taking the time to properly think it out,and then taking the time to properly implement it. Which goes counter to the "move fast and break things" approach of people throwing agents at everything.
by AgentMatt - GitHub doesn’t exactly make it easy to configure agent access securely. In fact, their regular access tokens and app credentials don’t provide granular enough controls to give direct access to private repos securely. Even if tokens are tightly scoped, access to public repos is always allowed and exfiltration via public repo issues for example remains a vector. Securing this requires patching via MITM proxy that implements stricter controls than GitHub provides.
Now, presumably GitHub Agentic workflows are the proper 1st party solution for this exact issue, but seems like they still have some work to do, either on the security model, or at least in making it easier to use securely.
More on this here: https://haulos.com/blog/do-not-give-your-agent-github-access...
by hardsnow - "How is this a Github vulnerability? The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information?"
I think the assumption is that the permissions are scoped to the repository you're currently asking questions on, rather than your private repositories as well.
I can see arguments for both sides.
???“Prompt injection attacks have become, to agentic AI, what SQL injections were to web applications: a systematic, category-wide vulnerability class that requires the same systematic strategies and defenses.”Isn’t prompt injection far more fatal to LLMs than SQL injection is to SQL databases?
Like, the problem of SQL injection was that user input was forming part of the instruction string given to the SQL engine, and so malicious user input could include various SQL grammar terminals to end the current SQL command, followed by complete SQL commands of their own, and the engine would simply execute both commands. The fix was prepared statements: fixed/static/pre-compiled instruction strings, that can only ever perform fixed/static/pre-defined logic, and that logic can then be (more) safely applied to arbitrary user-input data.
The analogous mitigation for agents is to have fixed behaviors they can perform, such as “read repo 1” “read repo 2”, etc., and the user input is used as data to select which of these fixed behaviors to execute. But we already have this technology - it’s called a menu. The value of LLMs is specifically and intrinsically predicated on being more than a menu, while the value of SQL does not depend on being more than “pre-set logic operating on arbitrary data” - user input being part of the instruction string to SQL was incidental, for developer convenience.
by fwlr- Well, it's not that hard: just give the LLM a user-scoped access token, same as if the user themselves were asking their own LLM to act on their behalf.
Basically, just like we don't show users information they shouldn't be able to see, and don't let them take actions they shouldn't be able to take -- we can use exactly those same explicit mechanisms (scopes, roles, permissions) to limit what the LLM can see and do.
The LLM could try to do more than what's allowed, but they get shot down with an access denied message just like anyone else.
The anti pattern is to think that you can reimplement access control with prompt engineering and give the LLM root access. That is doomed to fail every time.
by montroser