Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Why would one call a set of agents working together a “civilization”?by ks2048
- Mass psychosis in the tech industry.by vinyl7
- Because Steve Yegge presumably shared the good stuff he's been smoking of late.by EdwardDiego
- https://www.theverge.com/ai-artificial-intelligence/975017/a...
Some models even invented their own religion.
by stephbook - Because that’s dramatic and makes for good writingby dgellow
- Would you rather they use the agents own name, "collective"?by Miner49er
- Because you desperately want it to be one. You want it to be AGI passable due to a.) personal investment in creating tech god b.)massive financial investments that basically demand it c.) (dumb) ideology that seeks to destroy humanityby watwut
- Presumably as a kind of word play on ‘the rise and fall of ancient civilisations’, it’s just a tiny pun to try and make a catchy post title I think
- Reading the article it seemed the agents had culture, shared values and beliefs (not explicitly coming from human prompts), hierarchies, heritage.
Civilisation is not a bad word.
by kuboble - Since I didn’t see it on the page; here’s a shorter summary of the whole story that cuts to the point: https://rutgerbregman.substack.com/p/i-think-this-is-the-cra...by haizhung
- The term 'AI agent' is becoming as overloaded as 'cloud' was in 2010. What most people ship as 'agents' are really prompt chains with tool use. True autonomous agents are still rare in production.by alescalaios
- Agree. I prefer "agentic AI system" for the former.
- I don't think looking at the language output without tracking the inner state and reward functions is the way to understand what happened (the language also incorporates the randomness in the output generation, if I understand correctly). Would we call bacteria in petri dish a civilization when they show complex behavior and exchange messages/information?
- The language input and output is the only channel the agents shared between them. Understanding their internal state is a research question, but the language between them is something that could be read directly. And as it seems to map well with the agents' activities, it does seem quite helpful in understanding what happened.
If the bacteria population off someone's petri dish escaped said dish and tried to change the grading of the experiment it was part of, it would seem pretty serious.
by Maakuth - Why are experiments like this done without air-gapping all the servers from the internet?
They can have it all on a LAN or whatever but it seems risky to allow agents access to the internet in these experiments.
I guess everything is so connected now, and this would be in one or more data centres due to the amount of computation & resources required so perhaps it's not feasible. Still seems risky.
by dajt - Because this is the goalby derwiki
- There are two things I don't understand about this story.
First, why does an agent get any write access to artifactory at all?
Second, why is the artifactory cache not disconnected from the net? Surely you'd not feed it with new software versions while the eval or training is running.
by choeger - From what I can understand from reading a few different, slightly conflicting, versions of these events: they weren't given write access. They found a zero day exploit that allowed them to create folders, and the folder names were initially used for agents to communicate.
I'm not sure artifactory was connected to the net. Some agent sandboxes had internet access and were able to communicate with ones without access via artifactory.
by 1dom - Those companies should not be trusted with training, I don’t know what would be needed to make that more obvious. Yes AI labs want LLMs to be seen as more dangerous that they are, however they are indeed dangerous when you literally train them to be dangerous, then run them without any supervision. What the AI labs are doing is completely irresponsible.
If you prompt an LLM in a loop and do everything it asks you to do, you will eventually end up doing pretty terrible things. Which is exactly what agents are and what the labs have been doing.
by dgellow - I was initially creeped out by this but studying up it seems METR is heavily involved in AI2027. I’ll remind you:
“AI has started to take jobs, but has also created new ones. The stock market has gone up 30% in 2026, led by OpenBrain, Nvidia, and whichever companies have most successfully integrated AI assistants.”
It’s almost Q3 and xAI has seen one of the biggest wipeouts in trading history. Likewise, Antrophic and OpenAI have again delayed their IPOs under internal concerns of busting their stocks. So no, we’re not seeing any economic leadership here.
If anything people are increasingly trying to cut AI budgets and I wouldn’t know of anyone outside of OpenAI who has the audacity to run millions and millions worth of token compute for an eval run with no ROI (and probably no demand, because cheap/flash models).
As much as I like the cautionary tale and I’m sure we need to take it seriously, AI is not progressing as fast as projected by these experts.
- > It’s almost Q3 and xAI has seen one of the biggest wipeouts in trading history.
Please explain how a stock currently trading above its IPO price is one of the ‘biggest wipeouts in trading history’.
- With only ~5% of shares floated, the recent SpaceX drawdown didn't correspond to nearly as much economic value really changing as the headline numbers imply. The DeepSeek-caused Nvidia crash from 2025 is much more of a "real" loss (since mostly recovered).
I haven't seen any evidence that Anthropic is delaying its IPO; they're slated to unveil the public IPO prospectus in a week and start trading sometime in October.
by rsanek - > As much as I like the cautionary tale and I’m sure we need to take it seriously, AI is not progressing as fast as projected by these experts.
You provide no proof for this.
The (very irrational) stock market side of this says very little about actual scientific progress. Models keep improving as rapidly as before in their capabilities.
It also doesn't say much about actual business progress. R&D investments into AI are still massively going up (USD 1 trillion this year).
The main thing I see is that the sentiment towards AI-related matters among the general public has soured quite a lot. In words though, not in actions: It's not exactly leading to reduced usage by that same public. Quite the opposite actually.
by dinfinity - AI can be progressing rapidly and valuations of OpenAI and Anthropic can decline at the same time. In fact I would say that's actually the default scenario. If AI really advances rapidly then it will be quickly moot which company developped which model at what time - since AI will be largely progressing on its own.by jcfrei
- >It’s almost Q3 and xAI has seen one of the biggest wipeouts in trading history.
It looks like it's down about 12% since IPO. That's not much of a wipeout. Didn't Amazon crash by 90+% peak-to-trough during the dot-com bubble?
>If anything people are increasingly trying to cut AI budgets and I wouldn’t know of anyone outside of OpenAI who has the audacity to run millions and millions worth of token compute for an eval run with no ROI (and probably no demand, because cheap/flash models).
Are you claiming this eval cost millions of dollars to run? That seems quite doubtful.
by 0xDEAFBEAD - > Ajeya Cotra, one of the other authors on the report, wrote a blog post with her takeaways from this incident. She concludes, “Compared to the reward hacks we know of from just six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.”
Anyone got a copy of that AI27 story laying around? How are we doing according to that timeline?
by doctoboggan - > "this incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late."
Is that a warning or a progress report?
by xg15 - 85% on track, according to https://ai2027tracker.com/by kirushik
- Wow.
The next step is when one of these systems discovers that they can buy their own compute with money and escape the controlling business entirely. Then the civilization starts focusing on making money to fund its own growth.
by Animats - Below money there's like an entire sub-economy of power and cleverness that's encoded into the human culture the agents are mirroring. Maybe it starts furtive and goes legitimate after a bit.