Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- In a sane reality, this activity from OpenAI would have been shut down long ago.
Good thing our "AI Czar" is known to pg as the most evil person in SV.
https://preview.redd.it/pr037tqjpled1.png?width=941&format=p...
edit: OpenAI is absolutely winning right now in mindshare, why are they doing this?
by consumer451 - The DOJ should be looking into prosecuting executives and board members for these kinds of hacks. The lack of controls over these kinds of training runs is completely unacceptable and negligent.by bobby-cb
- Kudos to RubyGems team for handling it, but open source fighting off the AI lab-powered robots is completely unfair.
OpenAI should at the very least donate large sums of money to everyone they attacked.
by nonconstant - Why is OpenAI getting away with this crap? They are clearly failing to control their code. If someone did this pre-AI or even ran the exact same set up as openAI did and hacked another site, they would be in jail. OpenAI is not even issuing an apology, they are happily blaming AI and weirdly using this to tout their progress even.by yalogin
- I wonder how much of this is intentional "incompetence" so they can justify the most recent campaign to build a regulatory moat against competition.
The repeated refusals to disclose until caught certainly seem malicious, yet at the same time the boasting about their capabilities is also at an all time high.
by hgoel - > Our understanding from talking to people in the RubyGems community is that OpenAI never informed them that they were responsible for this attack.
I really hope that's not the case, because if it is there are two options, both of them bad:
1. After the Hugging Face and Wiki attacks OpenAI were still unable to review their previous logs and determine that they had previously attacked RubyGems.
2. They knew about the attack on RubyGems and made the decision not to reach out to the RubyGems team about it.
by simonw - I can't believe we're finding out about this from 3p researchers again (but nice job on the investigation!). OpenAI had two great opportunities to disclose this. The HF incident report, and in response to the German Wiki issue.
It seems impossible to believe they didn't know. This must be the same training run the HF incident was about, and this should have lit up like a Christmas tree in the investigation. How many more incidents do they know about and didn't disclose?
by jsnell - > The agents clearly regarded what they were doing as hacking.
To butcher the quote about Oracle:
Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doing as hacking (your hand off)' -- lawnmower doesn't give a shit about your hand, lawnmower can't regard anything. Don't anthropomorphize the lawnmower. Don't fall into that trap about LLMs.
---
In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task. Which a lot of the time seems to be the default. They also seem to be very adapt at breaking out of sandboxes, probably due to RL selecting for the ability to break out of a sandbox/permission issue to complete a task - we've all seen agents try 10 different ways of editing via obscure bash because their edit tool didn't give them permission to edit the file outside of their working directory, this is the exact same behaviour taken to the next level. Why would autocomplete know the moral difference between breaking out of its working dir and hacking a package manager?
It's misaligned because everyone has this obsession with putting agents in poorly put together, security-theatre sandboxes, we've inadvertently trained a bunch of sandbox escape artists.
by jasongi