Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- "The task asked for information about a specific person who had published a blog post and the agent was provided with a set of biographical details and clues from the person’s public blog post."
Who initiated the task? An OpenAI researcher or a user?
by freitasm - What DNS service did the agent discover that allowed it to execute arbitrary llm queries? And how?by itintheory
- So the LLM was able to look up a basic way to reroute things to get to their destination (likely well available and trained in the corpus) and it's surprising?
What's surprising is the surprise the security testers are explaining.
By setting an outcome to reach an endpoint, and to find all possible ways there, would this not be in the realm of possibility if an agent is reasonably in control of a vps?
Having the vps locked within a network layer it can't see or get out of is pretty common practice when setting up IaaS / PaaS.. sans-llm.
Maybe I'm missing something here, what confuses me is how something so relatively simple can get such prominent coverage, it's hard to imagine this kind of ability is still relatively new or surprising to folks working at the major models, unless they aren't hiring for network experience?
by j45 - It's pretty impressive how thoroughly incompetent is OpenAI designing secure systems. But surely this is circumscribed to agent security. In no way are all my chat logs in some Russian forum.
BRB, I'm going to delete something before it also ends in Chinese forums.
- When exactly did we forget how to make literally anything that can perform a computation but (physically, hardware-level) not have the ability connect to the Internet?
With these companies spending the kind of money they are, if they actually mean what they say about the security risks, they should be expected to figure out those kinds of precautions and take them.
And build Faraday cages too, just in case of a hardware supply chain compromise.
by zahlman - Most interesting here:
> We therefore stopped the affected training run and have subsequently decided to pause all other training, evaluation, and inference with tool-use (defined broadly) for our most capable models until we have both validated that the gap is resolved and performed additional red-teaming of the system. When training restarts, we will begin a fresh run with additional alignment improvements, including more comprehensive misalignment interventions. We will not resume training this particular model, even though the existing reward signal already correctly penalized this behavior.
by garo-pro - Why are we blocking agent access to normal tools without telling them “hey this access is beyond the intended scope of this task”. If I woke up one day and couldn’t reach google.com, I too would start fiddling with tricks to restore access.by rao-v
- > The monitoring system detected this incident, but our retrospective review identified other cases of external DNS access that it did not flag at the expected severity. These included queries that returned a static notice that an external service had shut down. The monitor sometimes treated the failure to obtain useful information as evidence that the attempt to access the internet had failed.
This seems to say, "we are using entirely unreliable AI tools to monitor our AI tools."
by jsrozner