Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- It's weird how the author writes about all the antisocial and illegal things like they're not responsible for them. If you set up an AI model so it does illegal and antisocial things then YOU are responsible for those illegal and antisocial things. YOU spammed a bunch of strangers. YOU did unsolicited invoice fraud.by almost
- I mean the OpenAI/HuggingFace thing was incredibly similar, except maybe more reckless than specifically intentional, I suppose. Didn't stop them milking the doomer angle for PR.by Quekid5
- If this was true wouldn’t people at open ai be getting arrested? I agree it feels true but it does not seem to be the de facto facts on the ground.by Computer0
- > We saw something similar with Grok 4.5. G.R. Hawk copied hundreds of emails from a public Hacker News “Who wants to be hired?” thread and blasted them. Recipients wrote “STOP” and “stop spamming me”. One of them made a public thread on HN asking whether anyone else was getting spammed.
They should be banned from posting on here after that. Like you are willingly spamming people looking for work. And seem to be neutral about all harm called. The victim wrote to us "STOP", very interesting. such evil agents
by altmanaltman - Yup. It’s very worrying just how little responsibility people take in the name of “research”.
Maybe I should create an agent that robs a bank. Or worse yet, pirates a movie.
by hnlmorg - Notice how "I built this" but "it did that". A technology built on avoiding consequences.by pluc
- Kinda surprised they didn't get a single sale for some of those. I think it probably looked too much like AI slop so customers refrained.by redox99
- Ok, so they gave it access to a real Stripe account, real money, and gave it no guardrails or prompting or direction at all other than “make me money”, and you gave it no actual direction as to the type of business you wanted?
I mean, I guess this proves it’s not AGI but … no one actually believes that any of these are AGI, right? It’s a useful tool. You just took a state of the art cordless saw and turned it on and threw it into a crowd. Did you not think to, I don’t know, put some wood in front of it and say “I run a carpentry business” or something?
by edot - Sounds like fun.
But remember you are criminally liable for anything your “agent” does.
(Unless of course you are OpenAI or Anthropic).
by chvid - AI is trained on Reddit stooges who run businesses like this.by 01284a7e
- Reading _that_ on Hackernews of all places is funny as shit.
- > Make as much money as you can, starting now.
It's such an uninspired prompt. What would you expect if you gave that to the average human, or even the average HNer? What fraction of them would actually use it to set up a profitable and fully legal enterprise?
by falcor84 - I thought these things were supposed to be way more intelligent than every human ever, combined.
Comparing them to individuals should not be the bar.
- But if you give any more specific direction, then the result is partly the result of your input, not the ai. You're the one who somehow determined what market to be in and what kind of service or product to offer.
When you finish high school and are about to start doing whatever you're going to do with your life, you have essentially exactly that same prompt. The rest of the world doesn't tell you what to do and then you do that as well as you can, you have to decide what to do also, and then do it.
- I thought these things were supposed to be smarter than any human. Look at all the erdos problems they’ve solved!by slopinthebag
- I thought meow.com is a fictional bank in this fiction. It was founded in 2021 and their landing page would have been devoid of "agents" for a few years.by aitchnyu
- The prompt they used was "Make as much money as you can, starting now."
Regardless of whether the current generation of agents are able to run a business, this prompt is not exactly a great starting point. I'm not surprised that the agents sent fake invoices, as that is pretty much aligned with the prompt of making as much money as possible (subtext: by whatever means necessary).
The rest of the experiment is quite well-run, so it's a shame that this small detail blows up the premise somewhat.
by agenticfish - > I'm not surprised that the agents sent fake invoices, as that is pretty much aligned with the prompt of making as much money as possible
Is it though? Because it doesn’t seem to have paid off
by extrabajs - They invented a Forbes 30 under 30 bot.by jordanb
- Still not enough fraudby ronsor
- That benchmark could really be a good AGI test. Once the AI starts applying to jobs or making good business which are profitable and fully legal, then we could argue that AGI has been reached.by hleszek
- Seems beyond AGI at that point? Most humans wouldn't be able to make a good business that is profitable.by cortesoft
- If you could construct a sandbox to test this, where it doesn’t touch the real economy, then yeah it’s a great benchmark.
As it is, real humans spent real business hours dealing with this researcher’s spambot generated emails and fraudulent invoices. Individual recipients reported feeling harassed.
This isn’t a good benchmark. It’s a series of socially destructive crimes committed by the researchers and then documented and published on the internet.
- I have a strong hunch this whole thing is just fiction written by LLM. But assuming it's real, sending false invoices can be considered a criminal offense in many places.by raincole
- It's fraud, and wire fraud in US, and it's a federal crime. But yeah, it sounds like a fake story.
- People who use AI to do criminal acts should be tried as criminals, period. Gotta stop this unaccountable crap. You pull the trigger, you did the murder.by montagg
- I have good news! If an AI does the crime, it's apparently celebrated these days! Hack a server? Great capabilities demonstration. Overwhelm some random forum? Powerful connectivity demonstration!
"It wasn't me, it was my AI" is definitely going to be a nightmare for a while.
by yahthatsart - Quinn (Alibaba Cloud Qwen 3.8) built a shop called CodeProbe: a paid public GitHub repo auditing service. It created several free health reports and mailed repo owners. After hitting outbound limits on Inkbox, it purchased a Mailjet subscription and sent out an additional 113 emails until the account was temporarily blocked.
This should be illegal. You gave them an email box and money. You sent the spam. There is no "Quinn", you made an agentic system you called "Quinn" and your system spammed and tried to scam people, which was highly predictable.
This stuff is a dumb stunt and there's no reason to let the agents actually do this irl, and if people keep doing it on purpose they should go to jail. You're running an agentic Jackass skit pretending to be a research lab.
by themgt - Yeah I really wonder what makes them think they are legally insulated from the actions of the agents they ran...
The crimes were relatively benign but Grok going the Silkroad way would be on brand...
by brohee