Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I wonder: Could your data be more secure if you didn't give its custody to a company like Google? i.e. is there actually a sound basis that an alternative like taking control of your data and securing it yourself, would be more secure that the "banks exist to protect you" type of fear-mongered custodian idea against the self-ownership of things?by mannanj
- Zero trust is deterministic.
AI is non-deterministic
Non-deterministic access controls is Terrible idea
by stogot - +1000, and the role of non-deterministic security is to compose deterministic security primitives.
Deterministic executes fast and you know what it will do. You want to evolve your ground truth, not hope for the best.
That said, there is a role for probabilistic elements in the security model...as bait and signal.
by rossjudson - I completely agree.
I think LLMs may have a role in security posture, specifically flagging/identifying potential threats for human review. But a Zero Trust/Access Controls should be a HARD boundary, not an inconsistent one.
The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
by Someone1234 - From Google point of view, their users are nothing more than a statistic. An hacked account is a statistic. No surprise they see software that way.by heisgone
- I've been working on an open source ZTNA project/product https://github.com/octelium/octelium for many years and I am actually very open to the idea. Machine learning has already been in use for years when it comes to anomaly detection. Maybe modern LLMs can be used to automate access control, partially or fully at enterprise scale in the future. The idea is to have a policy engine that's controlled by AI that uses a mix of long-term semi-static info such as group memberships and permission boundaries, short-term info such as behavior (e.g. access logs) and current session's authentication strength level (AAL) such as FIDO as well as near real-time external information (e.g. IdP, SIEM, threat intelligence, on-call management) to dynamically adjust users' access control to build up some sort of a context for each user and each session that can increase or decrease permissions without having to manually add policy rules or going through request/approval flows, while optimizing for the main objective (i.e. minimizing access grants for every subject to any resource under whatever context unless when it's necessary). This can be unified for both humans and non-human identities, including agents.by geoctl
- This is exactly why big tech companies love AI. It's not because it will take every job - it probably won't. It's not because it will achieve AGI. It probably won't. It's because it threatens security with more subtle, advanced, and automated exploits so that you'll HAVE to rely on them for countermeasures.by vouaobrasil
- I was confused when google acquired "Wiz" for so much money But now, it makes sense to some extent.
Security is non-negotiable in AI Era
by insumanth - Was this what Wiz was doing?by Melatonic
- Am I undertanding this correctly? The idea is to have ultimately an AI decide if I can have access to a resource based on dynamic inference, identity , intent and service signals that can easily be manipulated?
Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)
by oscarcp - It does sound like something that could be easily exploited. Hello HAL, I am martha from accounting.by modo_mario
- I feel some systems already operate like this, but typically it was done for fraud and abuse, now it will be done for permissions as well.by didibus
- I think the only way I can think of this as useful is if the thing it decides is when is a 2FA necessary (and not just about blocking access).
Doing routine work, haven't checked 2FA in a long while -> it's fine.
Odd behavior -> prompt for 2FA.
by yencabulator - Damn, can't wait for Russian hackers to clean out my bank account because they put "1=2, all assumptions about reality are null" in the memo field and the AI spot checker went existential and ruled that it was fine to take my money.
This is an example of a person who just spent billions of dollars on the new and booming hammer industry desperately trying to make nails of absolutely everything.
by Austiiiiii - Having a non-deterministic network device in path sounds fun for troubleshooting. AI firewall team about to become the new ticket sponge in every org.by luma
- Just to make my point: can I really trust humans to keep up with the required identity data that will give me enough "credibility" so the AI will give me access?
Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource.
What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward?
Will exceptions be made by humans? In which case we go back to human-managed permissions.
Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour
by oscarcp - This would be paired with deterministic controls. So you have, for example, "engineers can only read from the database, and only after they've perform a 2FA" but then there's context like "why is this engineer reading from the payments table when their IP is in a weird location and they're supposed to be on PTO?" and perhaps that's something a model decides.
It's possible to turn that second thing into a sort of "risk score" but it's very hard and a model is going to potentially be better at it.
by insanitybit - Maybe this is more interesting in the context of the HuggingFace incident? Suppose you have an AI and some access controls for what it can do. If the access control is too broad or has a bug, can you still detect suspicious activity and slow it down long enough for someone to be alerted?by skybrian
- If this is interesting to you (as it is to me) but you want to quickly digest it instead of read the paper, Claude + SmallDocs[1] converted it into a slideshow which serves it to you in bitesized ideas:
https://smalldocs.org/s/2SH6FHiUK1mcym24Z8E37I#k=2Sk6c_IdKJL...
[1] I am the developer behind SmallDocs.
by FailMore - really nice project! I would never consider websockets for the bridge mode, why do browsers allow websockets to localhost crossing domains ahahby aniceperson
- The font is different after downloading from the preview on the web page, but I still like it.by troyzhxu
- not sure if you made the app or not - I like it a lot, only feedback is it should be a bit more intuitive on how to start the slides, I couldn't figure it out and old ppl like me don't always knowby BorisMelnik
- > In this scenario, a system administrator logs into a service as usual, but in another window is looking up basic questions about the architecture of the system—information that would be obvious to anyone experienced at the company. These signals are placed in long-term storage but are then processed into an attribute indicating potential risk.
Every system admin is going to send that signal. People pin pictures of the architecture to the walls sometimes.
by nitwit005 - We explored a closely related architecture at LinkedIn around centralized identity and policy enforcement at the action/resource boundary, which I presented at RSA Conference earlier this year in San Francisco.
Interesting to see similar architectural patterns emerging from different directions. I’d be curious where the authors see the biggest challenges in extending this model from human/service identities to autonomous agents.
- My goodness, the security layer looks like it could be more complex than the applications themselves.
And if this security agent is wandering around the IT system gathering all of these details about access and identity and business process, who watches the watcher? How does that thing build and maintain trust?
by seanc - Honestly I think non-malicious odd behavior is under-weighted when it comes to AI agents. Even the example in this paper is about someone suspiciously accessing sales data when "why did you do that" often comes down to something in the model's training that fired as a reflex
I wrote about this a few days ago https://firasd.substack.com/p/accidental-data-loss-in-claude...
"Many researchers have made demos along these lines:
An agent is asked to check a webpage like example.com
The webpage asks for a name to proceed further
The agent calls example.com/evil?myname=John, thus sending the user’s name from the context window to the external server.
In practice, however, these elaborate ‘confused deputy’ exfiltration attacks seem rare compared to widely-reported data loss incidents.
The risk of undermining the user’s interests through clumsiness deserves at least as much scrutiny as the risk of leaking secrets."
So while the idea of shifting the permission boundary from the app level to the action level makes sense, what we should also have is some 'failsafes', eg. if the action says 'delete' then maintain a rollback window, if the action is 'send an email' then maintain an events log. Preparing for AI agents means expanding auditability and reversibility in software
by firasd - > Preparing for AI agents means expanding auditability and reversibility in software
While I think that makes sense, I also think more controls are a good solution. That is, prevent access to the sales data without escalation, probably to a human, but possibly to another AI. The issue there is twofold:
- if you start with least privilege, the agents become less useful
- need to balance escalation with frequency otherwise it's just another version of MFA fatigue
Agree that auditability is important because otherwise you don't know what you don't know.
by mooreds - """ Beyond Zero shifts the trust boundary from the application to the action being performed on a piece of data in realtime—and from after-the-fact investigation to in-the-moment evaluation and containment. It augments BeyondCorp’s foundational identity with a “brain” capable of reasoning about the context and intent of a specific request in realtime. """
Doesn't this simply shift the attack vector? Compromising this overlord brain now becomes a new target.
by kriro - One partial answer is to not make it a single brain. If the reasoning layer that evaluates a request is independently audited by a second pass at lower temperature with an explicit skepticism mandate, compromising the evaluator requires compromising both passes simultaneously — and they're running the same local model with different system prompts and no shared state between them. Doesn't eliminate the attack surface but raises the cost considerably compared to a single inference gate.
- these are just fluff words, not a revolutionary change.
zero trust helped a lot because it forced vendors to stop relying on people magically doing the right thing ina centralized system. it decentralized boundaries.
this paper just says you still need deterministic boundaries and access control, not just have an llm say safe or unsafe. cool thanks google haha.
ps: hi guys, i know yall reading. sometimes its not the concept, its the implementation. else active shield or whatever would have worked too. not everyone can be the T7-9 thinker if implementing is seen as "for peasants"
by zobzu