Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- It’s only a matter of time until one of these causes real damage to the wrong party and OpenAI finds itself drowning in years of litigation for settlement amounts they can’t possibly ever pay in their current financial state.
On the present trajectory we’re 24-36 months away from another company inheriting the smoking wreckage of OpenAI as scraps handed over as compensation for damages.
by cmiles8 - The Australian Government needs to sue OpenAI to make a point.
At this point, it feels like it is out of control and it is just a matter of time before something much more significant happens. Imagine they hack into FAA to disrupt air travel, utility companies for water/gas, a nuclear power station (ala Stuxnet), financial/banking systems, stock exchanges, etc?
by arlattimore - I'm a little confused as to why these agents are capable of escaping containment. Can someone who has more understanding (or a better guess) of the harness they are running with shed some light?
To establish the premise: as someone who has a fairly good understanding of the token completion mechanics of an LLM, these agents are completion token calls in a loop, producing a "do this now" request which the harness then runs with some standard "call this function" code.
If these agents are enabled with explicit network enabled tools, its trivial to monitor their inputs/outputs. If they are not, you can still lock down network egress on a machine. If _some_ network egress is necessary you can still do network traffic monitoring. I don't see how they couldn't implement some level of monitoring where big red lights start flashing when, say, their eval system was contacting a domain/IP located in Australia, and further categorize that domain as government owned. This all seems very doable - am I mistaken?
And you're telling me all of these companies are failing to do this? Is my understanding naive in some way? This is assuming some good faith of course, I can easily speculate as to the political and corporate incentive. But it seems to me quite risky/negligent.
Currently, my conclusion is that its just (silly until proven wildly dangerous) negligence with the small side effect of being potentially good for business. And potentially company Foobook is then incentivized to get in on the news cycle for marketing purposes and basically guarantees an agent will do something of the sort by running some harness that allows the behavior quite trivially.
My naiveté extends to why there is such concern with "losing control of agents" when the above measures seem so doable. It might take a law but it seems doable.
by bplatta - Freaking out about "hacking" any government website seems like hysteria at best, they are usually poorly secured and even children regularly "hack" them. I recall one "hack" accusation turned out to be that some clicked view source and all the data was there. So we'd need more details on that.
At the same time these companies should be blamed and held directly responsible. Openai's agent didn't hack. Openai hacked. An open ai employee or group of them was negligent and greedy and ran a process which breached a government website. This would be totally unacceptable from any non-AI company, it's like writing malware and running it, then blaming the malware and not the author.
- Someone is always paying for the tokens (Agents running at OpenAI directly use their own models without paying directly, but even then it is not like inference is free). And someone is running the prompts. If they prompt agents and launch them and don't check what they are doing, then the agent is just following the prompt. Not checking what it is doing is negligence. If they checked what it was doing, they could have just pulled the plug. There is nothing rogue there. If it cooperated with other agents running outside of OpenAI, then the blame might be shifted to whoever runs these agents.
But there isn't any agent out there that was autonomosly miracly launched by a word prediction engine. All it can do by itself is getting and input and giving an output.
by mier85 - The thing I hate the most about tech, and I feel completely impotent to change minds on this, is the "fake life" tolerances it is afforded.
Airbnb is not regulated like a hotel because it's tech. Crypto isn't betting because it's tech. Now even breaching state data is ignored.
Can you imagine walking out of a ministry with a stolen cabinet? You'd get shot for doing this physically and people wouldn't bat an eye.
by torben-friis - If a bull escapes a field and causes damage in the village, the farmer pays for the damages and is liable. It's been like that for hundreds of years and I don't see how this is any different?by port3000
- > the breach took place on 18 June - Open AI informed the government with an email to a general address on 10 September
So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.
Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?
Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
by vintagedave