Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- As exciting as it sounds, but
"As a constraint, a GA designer should aim at a system which costs, as of mid-2026, >$1,000⧸month"
will make this an elite tool for the privileged. I don't even disagree with the premise that people are shocked if something costs no matter how much value it delivers, nor do I suggest they should make it cheaper. It is just the realization that AI will accelerate the widening of the gap between the poor and the rich even more and there is probably nothing we can do about it.
by weinzierl - You’re assuming this $1000/mo will buy you something useful.by ks2048
- 2026: $1,000/month
2027: $100/month
2028: $10/month
by wmf - Obviously the costs will come down over time. And quickly.by jey
- The future isn't evenly distributed.by greyface-
- It's a fascinating proposition but I see a couple of issues with it:
1. Most of us do not have the volume of training material that gwern has 2. Most people likely would not need this functionality (as I understand it to be)
For #1 I'm sure there are ways to wring data out of metadata (e.g., youtube history log), and I'm guessing email and IMs would be a start. But a clone of yourself -- that would be a significant amount of extrapolation)
For #2, there's obviously people that would love this functionality (myself included). But I think its safe to assume that most of the population would be satisfied with having a capable digital personal assistant that knew your needs and wants.
by pstuart - Only, if he had instead fallen in love with the version of this idea in which a community acts as the principal, rather than an individual.
(I would like to be known to the agent serving my family, that serving my friends, my team at work, the PTA at my kids' school.)
by jephs - I don't think that GA would be incapable of that, from what I know of LLMs and agents. The part where you make the GA uphold your set of values could, instead of coming from an individual, come from a collective. Price could be shared too!
(Oh wait, is this the birth of AI mayors?)
by Mithriil - you would love bruce sterling's short story "maneki neko", and karl schroeder's "degrees of freedom".by zem
- Seems there is more details here https://gwern.net/guardian-angelby boltzmann_
- > The big AI labs are building a single mind for everyone
Reminds me of Pluribus
(I just started watching it on Apple TV so maybe this is a late realization for me)
by malshe - I understood Pluribus as a metaphor for what AI is to us today / what it is becoming. Really great show
- Pluribus is a collective intelligence/virus which forcefully (leads to hundreds of millions of deaths) injects itself into biospheres and then blissfully experiences itself without, of course, hurting itself in any way. Then it might eventually start starving to death because of that prime directive.
That is positively utopian compared to AI where 99.9% of humanity gets to experience a hall of uncanny valley mirrors until they are shipped off to the scrapyard or get artificially augmented with some AI mindfuck and start thinking “you’re absolutely right!” themselves. (I mean that’s the dystopian version. But if you are comparing AI to that show then presumably you think that AI is at least a little bit dystopian.)
The virus in Pluribus can even be talked with and reasoned with like a distinct entity that is real and has its own thoughts. Oh boy, much better than a million pastiches of a million echoes of the past where there is no original thought to be traced back to after 2022.
The motivation for the virus is also completely different to the motivation of the AI Overlords.
But it’s that pretty typical lack of popular imagination. Some individual/collective narrative is presented. The flawed but autonomous individual rebelling against the flawless but enslaved collective. But dang it, I’m an American and I value my freedoms. Released forty years ago and people would rave about how it aptly critiqued the Soviet Union.
by keybored - I watched it maybe 12 months ago and had a similar 'parallel to AI' realisation.
A singular voice for all of humanity.
Gives me the creepy-shivers.
by BLKNSLVR - The AI writing thats ubiquitous these days - on my company email and slack, on random quotations in the newspaper, blogspam, even narrated by youtubers on camera (!) - is exactly like the Pluribus hive mind. Everywhere on the internet you encounter the same entity.
https://bsky.app/profile/did:plc:ig5qohlny4csxs2lu3dyvhp5/po...
by duckfruit - > Reminds me of Pluribus - I just started watching it
It is astoundingly good. A contender for the best show I've ever seen (and I saw the 1st run of Star Trek TOS).
by WarOnPrivacy - Their profile specifically says they will not acknowledge follow requests, but they only share posts with followers. As such, not sure what the value of the top level link is.by jvanderbot
- Yeah I clicked through and just see hundreds of "this account limits who can view their posts" messages.by geerlingguy
- And Twitter has a lovely
> Something went wrong Try reloading. If the problem persists, please try again later.
Modern social media :(
by Havoc - LLM psychosis claims another victim...
A reminder for anyone reading this: talk to people. Real humans[1]. They will remind you there's more to life than what ChatGPT can offer you. They might even remind you, for all their stupidity and flaws, what intelligence looks like as compared to a program that predicts tokens. Forums like these always get philosophical in high-minded discussions about intelligence, but there's a useful legal principle that grounds us in the real world: "I know it when I see it". A real conversation with a real person looks nothing like one with the so-called superintelligent machine gods, so it'll probaby do your mental health some good to remember what that's like.
[1] Nobody in Sillicon Valley or big tech counts as a real human. Talk to an actual normal person.
- Did you even read Gwern's Guardian Angels post? It literally describes the concern pointed out here, that the large consumer models are not aligned with users' interests. If anything, I would argue that Gwern has maximal AI Lucidity, given everything going on.
- I understand the point that the physical world is still our (human's) anchor, but the virtual world still "exists" anyway, as in: it impacts the physical world.
Whether some humans are "Real humans", anchored in the physical world, and some are "LLM psychosis victims" is, in itself, not problematic I think: they each deal with parts of reality that are divided, but still affect one another.
Of course, a society more grounded in the physical would feel less dystopian... But ultimately, I think that GA is an intellectual's approach to fighting *for* humans in this numerical world.
by Mithriil - Talking to LLMs or reading books is much more effective in updating my priors. Talking to people is mostly ego, signaling, misunderstandings and axiological differences.
- yeah. The GA post reads like a less addled version of Yegge, but the same genre.by davidgerard
- I totally understand the sentiment, and I generally agree. Something so strange I'm noticing is how AI-pilled regular people are becoming. My sister-in-law is a totally non-technical person, but everything is chatgpt this, chatgpt that. My wife, my kids, various friends — they're all talking about generative AI with way too much regularity. On the bright side, they're disclosing when information comes from AI, but a year ago I would never hear about AI from them. I don't like it.
- And this is true of everyone else, whether lawyers or writers or researchers: increasingly, you are the bottleneck to be optimized away. As long as human workers cannot be removed from the loop, the AI tools are complements, but as soon as they can be, there’s no reason to keep them, and trillions of reasons to substitute AI for them. (And once human workers are no longer irreplaceable, where does their power or relevance come from?)
There are so many false assumptions and leaps of logic here, I don’t know where to start.
The idea of a more user security-conscious LLM is a good idea. I don’t know why it needs to be wrapped in novel-lengths of LOTR quotes and Sci-fi speculation.
That said, it’s very interesting to juxtapose this post with others by writers that are interested not in “maximizing product output” but in actually expressing themselves.
This post is like the ultimate expression of the idea that the only valuable thing writing has to offer is information, and not personal experience, poetry, advice, humor, etc. - which I suppose is fitting for someone that has written anonymously for over a decade.
It’s a shame that he didn’t reach the opposite conclusion, like some other writers have – the way to respond to LLMs is to be more human, not less.
by keiferski - “a more user security-conscious LLM is a good idea. I don’t know why it needs to be wrapped in novel-lengths of LOTR quotes and Sci-fi speculation” <-> “This post is like the ultimate expression of the idea that the only valuable thing writing has to offer is information, and not personal experience, poetry, advice, humor, etc“
- > What would it take for LLMs to make me 100× more productive? Without this, I am doomed to irrelevance.
Are you, though? You will only be "doomed" if your place your value system squarely on "productivity". Then what is to differentiate you from a machine?
Edit: I'm not sure how I can feel confident about a proposal that puts so much value on "productivity". How can you reconcile this with "self-actualization"? (Don't get me wrong, I like my LLM-based productivity gains as the next person, but I care more about wisdom than becoming "100x more productive".)* * *Edit 2: "the goal of GA is to preserve individual human cognitive liberty and flourishing" — so the proposal is to do that by overlaying a software bot that continuously mimics your "self"?
by kashyapc - One of the example Gwern brings [1] is a GA as an agent for public discourse and political life, to bring your "set of values" to have an effect in solving society's issues, when you yourself don't have the time or energy to.
I don't think it solves or reconciles with self-actualization, but, from a pragmatical point of view, it offers the promise of extending causally your set of views and philosophies to politics.
Gwern also mentions that it allows to keep "human values" (if the GA truly upholds yours) in the loop, in processes that are to eventually be automatized beyond human's reach.
[1] https://gwern.net/guardian-angel#use-cases-politics-politics
by Mithriil - Agreed, re: placing value squarely on productivity.
The thing is, we're all eventually irrelevant. That's the way the universe works. Every single one of us, after enough time. Claiming we're "doomed" to it is just ego talking.
by jjulius