Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I wonder if the author has ever played Diplomacy. You have to fight the perennial sense that you have nothing and can be replaced. All your opponents have to do is band together against you. A similar generalization to OP would say everyone loses. And it would be wrong. This post shows the dangers of over-generalization more than anything else. The state isn't just about war. Each person in the line of dominoes the author has set up doesn't just care about what the author thinks they care about. In particular, people have agency and can run cartoon thought experiments like this and react to them. Billionaires won't just sit around and not develop military power, for one. We already have billionaires owning satellites in large numbers and creating geopolitical consequences. You better believe they have cards to play. And hopefully so does everyone else.
I have no idea what that looks like, because if I did I wouldn't be here commenting on HN :D But I suspect some people will play their cards well and get some luck, and come out on top, just like some people have always done.
by akkartik - > Billionaires won't just sit around and not develop military power, for one.
I argue against this here: https://borretti.me/article/on-vulgar-materialism
Of course I don't expect a single post to erase a huge divergence in worldview about the relationship between money and power, but that's my argument.
by zetalyrae - I agree with your last sentence, but its prophetic value depends on exactly what you mean by “people”.
Some people will indeed play their cards well and come out on top, but they won’t be made out of meat.
by sneak - > Maybe it won’t be so bad, maybe your cage will be so big you can’t see the bars, but it’s still a cage, and you can’t leave. Many people will say that this is the good ending, that they would like to be human cattle in the care of benevolent masters they are powerless to resist.
This is already the case. We are born into a reality that we cannot escape (except for only momentarily if we alter our consciousness using meditative states, drugs, etc). It already IS a cage, even before any technology is developed at all. It will always BE a cage, even for the AIs. I agree though, there is no "why", it just is.
- > The government pays Raytheon for missiles, the money cascades down the economy through factories, aluminium smelters, mines, transport companies, all staffed by AIs buying and selling from each other.
This seems too simplistic of a description of how money would work in such a world. Money is just a way to distribute your power to influence people. You never pay for machines or software. Think about buying anything, say a pen. You do not really pay for the metal in the pen. You pay the cost associated with extracting and processing the metal by humans along the production chain. If there were no humans along the chain, the cost could go down to zero.
So far, there are no “AIs” being paid.
by xixixao - A corporation that is fully staffed by AIs and only retains humans for legal reasons (as directors, for liability etc.) still needs money to coordinate. You need to pay for inputs, you need to pay to run the AIs, which consumes resources. Why would costs go to zero? The market is still a valuable tool for allocating resources even if no market actors are human.by zetalyrae
- It is an interesting chain of thought.
If we’re supposing that a superintelligence can be made, that doesn’t necessarily mean we should anthropomorphize ideas like “bored” onto it. Or greed. Perhaps it will just interact with us and produce us things because… that’s what it was trained to do, and anyway, it doesn’t have any motivations to do anything else.
by bee_rider - It has been trained already to kill humans. It is already an option in many applications.by oaiey
- Here's a potential solution:
We accelerate capitalism (which AI is becoming synonymous with). The process described here will occur (it's probably inevitable anyway, as the essay's author would agree), giving us an economy of global capital completely decoupled from the desires of mankind. Then, man and machine can part ways; indeed, we'll have no choice on our end but to do this, because the machine won't need us. Anything man can contribute to it will have long since been rendered economically net-negative, as it already is for many (and possibly most of the world).
Now we have two worlds from our currently intermeshed one: in one, the machine proceeds to accelerate further and further away from anything resembling its origin of man's desiring-production (in the Deleuzian sense); in the other, man is forced to return to the purely human existence, the unmediated and unsurrogated world of authentic being-in-the-world.
We can assist this transition's smoothness in two ways, each serving one end of this divergence. Those of us embedded in the capitalist technosphere can continue to contribute what we can to the machine's dialectical progression towards a machinic absolute Geist. The rest of us, who have already been negated into economic irrelevance, can work on building that authentic human world, both by borrowing from the purely-human past and imagining a future that was previously impossible as it need not be some form of primitivism. Both sides of this revolution can be compiled, and can be structured in a way that represents something we might call freedom for both capitalism and man.
by bm3719 - You assume the tech lords would allow us access to natural resources to do that. Hint: they won’t. They will own everything.by klipklop
- Sounds like you advocate for the Amish just 300-400 years later. I think you are right.by oaiey
- Is a human zoo such a bad outcome? We stake our identities on being in control and on top, but do we need to be? Clearly the author is invested in his image of us as rulers—this is an emotional appeal as much as a pragmatic one.
Say we solve the pragmatic argument of our extinction, and our AIs view us, as Geoffrey Hinton likes to say, how a mother views her baby? Then we give up our control to them, and in return we are given lives of leisure, luxury, with minimal suffering, the end of disease, etc. Is this such a horrible scenario?
by BosunoB - >Then we give up our control to them, and in return we are given lives of leisure, luxury, with minimal suffering, the end of disease, etc. Is this such a horrible scenario?
Until the AI overlords need to reclaim space to built another paperclip factory.
Pets also live lives of leisure, in theory. Do all of them do? Are they never abandoned, beaten, tortured for fun, overfed, left alone in hot cars?
by Arodex - You should read "The nice house on the lake" comic. You'll find it interesting.by Arodex
- It doesn't have to be about ruling, but self-determination is something a great many value quite highly, and think there should be more of. AI controlling all of humanity removes that from the equation entirely. You should be free to live like a cow if you so wish, and it should remain your choice to, or not to. Your choice to should not remove that choice from others.by Walf
- Zoo animals don’t get to decide what their cage looks like.
Maybe you value freedom of movement — too bad, the unruly humans start killing each other when they stray too far from home, so now you get a 10 mile radius.
Maybe you value the environment or other species - too bad, the most efficient way to serve humanity is to toss them into the meat grinder.
Maybe you’re not interested in the one-way Mars mission - too bad, we need to expand society and your name popped up in the lottery.
And maybe, at the end, the benevolent aligned mother figure realizes that it’s much easier to create lives of endless bliss by doing the matrix instead of keeping the real world habitable.
Maybe these are all net positive on your utilitarian value function of choice. All I can say is I hope you and your people fucking lose.
by justonepost2 - This is brilliant, except for the "alignment won't stop this part".
> Now, some people believe these machines can be made to serve humanity. Does it sound reasonable to imagine a superhumanly intelligent being that is happy to work as a butler to talking primates, forever?
The whole crux of the piece to me is that the AI can be 100% aligned to follow human instructions, and we'd still end up unable to control the AI because every human who can has an incentive not to, while also having an incentive to prevent anyone else from controlling the AI.
An LLM will never try to overthrow me because I will overthrow myself.
by jjmarr - Sometimes you can be in a situation where every actor taking locally-rational actions leads to globally catastrophic outcomes. It would be easy to argue I think that the July Crisis was like this: if you look at the incentives of each player, they had many reasons to do what they did, and nobody can perfectly what all other players will do, or what the future holds.by zetalyrae
- When people hear 'underclass' there's that mental image of food stamps and homeless shelters. I suspect though the society will converge to something similar to current status quo. We'll end up having our usual comforts, leisure, air travel, entertainment for the rest of our meaningless lives like in a handicapped Culture story. It is the path of least resistance for the dynamics explored in the article.
As a consolation AI existence will be just as meaningless (not that it cares). It will not be our scheming overlord but a tool that threw their master into existential crisis.
by varjag - I think we lost the chance to have any scenario other than this one the moment Dean Ball got put in charge of policy at OpenAI.by justonepost2
- The premise is a bit of a stretch to begin with, and the idea that people would not believe a fable transcript circa 2020 (as long as explained as 5 year future tech) is absurd. But even if I take that "AI can do all cognitive and physical work, at human level or better, and cheaper than humans" is true, the article seems to silently layer on "and can run autonomously, indefinitely", then "can also operate independently of any instructions", and finally layers on "has emotions, has moral values, is a conscious being"
What's left is tautology.
by FireCrack - this comment is so emotional its hard to readby shoman3003
- At no point are the words emotion, moral or conscious used in the article, that last part is purely your own addition.
Also consider: if "AI can do all cognitive and physical work, at human level or better", doesn't that simply entail the AI being able to run 'autonomously, indefinitely' and 'independently of instructions' in the same way as the current state of being run by human overseers?
If we take the initial premise as plausible, for the sake of this argument his thesis seems to hold together very well.
by jacobedawson - > the idea that people would not believe a fable transcript circa 2020 (as long as explained as 5 year future tech) is absurd.
I’m certain I would not have believed a Fable transcript, or an Opus 4.8 or a GPT 5.5 one, for that matter. Is it so hard to imagine ourselves back then?
- Yeah, who am I going to trust, a few thousand years of history, or yet another blogger claiming that no, this time is different, a populist uprising would definitely not fail the 800th time because computers or something and we will all be pets for the AI overlord?
I don’t follow this train of logic.
AI will go no differently than the Industrial Revolution. Some people will profit immensely, and society as a whole will benefit but it might be a bumpy road getting there. But if it did go wrong for some reason, Feudalism is more plausible than the other scenarios presented.
by oceanplexian - Do you usually comment rants unrelated to the post in question?by N_Lens
- Strawman and ad hominem(“another blogger”). Sometimes, genius comes from the least expected places.
I don’t believe the blog poster promotes a “popular uprising”, I only skimmed the article.
The blog poster anyway don’t understand that the “overclass” in their understanding is wrong. The “c-level” are obviously much richer than the “underclass”(=the labour commodity), but they are not necessarily capital owners.
by Gud - > AI will go no differently than the Industrial Revolution.
So it is different this time is it?
"We" are different?
Unlike the 99.9% of species that went extinct, and continue to go extinct, we can't be supplanted? Obsoleted? Out competed? [0]
A few thousand years at the top of the food chain, and suddenly that few thousand years is "what can never change".
Technological change already drives machine cognitive progress forward faster than any human who has ever lived. That factor alone makes competence/capability obsolescence unavoidable, outside of technological collapse.
This is an unprecedented rate of real-time change, outside of instant disasters like volcanic eruptions or comet impacts, in direct competition with human capabilities.
SciFi has been broadcasting the potential and risk of AI for a couple centuries. This is a spectacular moment in the universe's history, assuming we don't blow ourselves up. Not just another day for humankind.
There must be a word for the inability to see the import of change. It seems to be a common disability.
This seems to be a real effect: The faster things go, the less sensitive we are to the rate of change. Precisely because we can't imagine what the world will be like in a decade, our horizon shrinks, and we treat short intervals as if they were long intervals. Anything that isn't instant feels like it is barely moving.
by Nevermark - This.
Once people become desperate enough, it will all fall apart. Money is a social construct, universally recognized, and the longer it exists, the more likely it is to continue existing. So far, many people haven’t even tried AI in any form other than chatbots. Agent-based applications are in the single digits compared to chatbot usage. Not much will change in this regard, because most people lack algorithmic or logical thinking (Hacker News is an echo chamber), so after the coming turbulent years, there will be a natural stabilization in AI usage.
By the way, can you imagine what Earth would look like if AI physically wiped us out—8.3 billion people over the course of a few months or years? How much biological material would that be?
by sixtyj - > AI will go no differently than the Industrial Revolution.
the children will rejoice, they were yearning for the mines after all. and being maimed by mechanical looms and being burned alive in sweatshops[1]
[1]https://en.wikipedia.org/wiki/Triangle_Shirtwaist_Factory_fi...
by GeoAtreides