Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • ...except when they do:

    "An off-the-shelf VM is not enough to contain a modern, cyber-capable AI agent...us[e] a virtualization technology that was purposely built with a minimal attack surface and a focus on security, like Firecracker. I had the AI agent run against Firecracker. It was able to hardlock the machine due to more Linux kernel flaws (all patched in upstream), but could not successfully escape."

    On Linux, it's all KVM and CPU hardware virtualization under the hood. Looks like the remaining known issues are with userspace. That's not to say more kernel- and hardware-level bugs won't be found, but the same tools that can find escape mechanisms are shields as well as swords.

    by otterley
  • The market is smaller (maybe, I'm not sure what the statistics are) but it would be interesting to see how Xen stacks up; also stuff like gVisor or libkrun. The latter is probably implicitly the same as Firecracker given the ancestry of the libraries used.
    by SirGiggles
  • The capabilities are incredible. I'd love to see even rough metrics on token consumption/cost in addition to the ~12-hour runtime.

    The interesting thing is that this naturally makes you want to isolate the VM as much as possible. But then every remaining interface becomes part of the attack surface: RDP, SSH, even terminal escape sequences, using sounds, and why not social engineering.

    by wslh
  • > it found a CUPS server (with a known CVE that had not made it to oldstable packages), but was not able to complete exploitation due to AppArmor.

    It's evident that Mandatory Access Control has become, ahem, mandatory.

    Both VM and inference engine processes should be confined, at least. There's a project where it's already done for many applications. [0]

    Additionally, `lockdown=confidentiality` in kernel boot flags greatly reduces attack surface. (interferes with unsigned modules like NVIDIA drivers) [1]

    Then AppArmor profiles could be written for absent applications with a learning-mode tool [2]

    Nothing is bullet proof, but these approach significantly reduces potential consequences [3]

    [0] https://github.com/roddhjav/apparmor.d (I'm the contributor)

    [1] https://madaidans-insecurities.github.io/guides/linux-harden...

    [2] https://github.com/nobody43/apparmor-suggest (I'm the author)

    [3] https://en.wikipedia.org/wiki/Swiss_cheese_model

    by nobody42
  • IMO the obvious answer is formally verified security.

    We can do this today for user mode, and we can mostly do it for ARM64 virtualization. It will be a while and would require substantial assistance from Intel or AMD to achieve it for x86 virtualization because the hardware is Too Darn Complicated and Too Poorly Specified.

    Formal verification of the hardware should also be possible.

    by amluto
  • Outside of the initial wave of security vulnerabilities and scrambling, it seems like the logical outcome of this over time is likely vastly more secure vm environments?
    by masterj
  • I have a lot of respect for Trail of Bits, and I'm sure that Artem is thinking about this correctly. However, I respectfully disagree with the premise.

    AI agents are not magic. Mythos/Glasswing does not magically create vulnerabilities in software projects. Advanced, cyber-capable models do not magically hack out of VMs or contained environments. They do not have a "hacking" stat that, if high enough, means that they can breach anything. They aren't Kevin Mitnick whistling nuclear launch codes into the prison payphone. This isn't a movie.

    What these cyber-capable frontier models can do is find security problems and exploit them. The statement should not be that VMs won't contain cyber-capable agents, but rather that we need to focus on finding and fixing vulnerabilities and misconfigurations in these environments.

    Even the article itself concludes with suggesting something like Firecracker, which was designed with security in mind.

    Like most other security-related problems introduced by advanced cyber-capable AI, it's possible that these issues will get worse until they get better. But if frontier models are run against state-of-the-art VMs, and OpenAI or Anthropic or whoever works with the virtualization projects to address the issues, eventually it will run out of things to exploit.

    The concept of virtualization is not inherently insecure. We just have a long way to go.

    by david_shaw

Explore Birbla archives