AI Is a Harsh Mistress

Join the discussion

• Hacker News
• AI isn't sentient, but it can generate text and people will anthropomorphize it. TESCREAL cultists evangelizing otherwise are best ignored.
  by cdrnsf
• Thanks for the reference: https://en.wikipedia.org/wiki/TESCREAL
  by krunck
• > Consciousness, in Chalmers’ usage, is subjective experience: there is something it is like to be a conscious system. Still, nothing in the Transformer architecture or in next-word prediction shows that such an inner point of view exists. Self-reports (“I am sentient”) and fluent dialogue are weak evidence, since these models are trained on vast corpora saturated with human first-person statements. This type of verbal report works for people because it is anchored in shared biology and behavior, however, for machines, it is an echo of training data.

  I don't want to argue here that LLMs have consciousness, but this kind of argument annoys me because it feels like an intellectual slight of hand on its own.

  There is no definite way (so far) to prove that anything besides oneself has qualia or a subjective experience.

  So arguing that we cannot observe LLMs having subjective experience, therefore they cannot have any is cheap - you cannot observe that for animals or for the person sitting next to you either.

  The part that self-reports are useless and LLMs sound human because they emit sequences from human training data is correct. But the article implies some sort of fundamental distinction between machines and conscious beings that can never be crossed. And I don't see any reason to assume that.

  by xg15
• Apart from the article's probable origin in AI, it makes an elaborate, wordy argument about consciousness without trying to define the term. That's just as well, since consciousness is philosophy's "hard problem," meaning no one has successfully defined it in a way that leads to agreement.

  Each of the article's arguments that AI "consciousness" is actually a weak imitation of the "real thing," can with equal justice be applied to people, and the classic Turing Test method to distinguish machines from people has long since lost its authority.

  And a future event, one in which AI sprints past humans in its willingness to absorb printed matter (once called "reading") and then put it to use, is ... wait ... umm, never mind, the point I wanted to make is undermined by the fact that this has already happened.

  Hard to keep up these days.

  by lutusp
• The idea that we are in danger of erroneously ascribing consciousness to AI sounds wild to me. Humans always assume whatever is convenient about other beings' (lack of) subjective experience as long as we have power over those beings. In this case the latter might no longer hold, but then it won't matter one bit what we think.
  by entropyneur
• “Much of today’s discourse on AI consciousness rests not on empirical evidence but on functionalism …”

  In my view there can never be an empirical test for consciousness, because functionalism is wrong. If functionalism was right then there would be one. There’s no third party test for a fundamentally first person phenomenon.

  by GPerson
• this would require us to still be baffled by consciousness after knowing everything about neurophysiology that might be relevant - until then, the keys to unlocking its secrets could still be hidden in our wetware behind a veil of ignorance
  by smarf
• > There’s no third party test for a fundamentally first person phenomenon.

  Agreed. But that also means I have no empirical evidence that YOU are conscious either.

  by qarl2
• The functionalist response to this would be that while the experience of consciousness in ourselves is a first-person phenomenon, how we recognize other beings as distinctly conscious in their own right is still a third-person phenomenon, and the cues we rely on still depend on observation of outward behavior that we attribute to that consciousness.

  And a response to this response is to point out that our recognition of consciousness in other beings is predicated on attributing to them the same first-person experience of their consciousness that we each have of our own, meaning that human consciousness is intrinsically human and can't be attributed to an artificial system.

  by Gormo
• I’ve been reading The Moon Is a Harsh Mistress lately and I’ve been shocked at how prescient it is. I’m kind of surprised I haven’t read more about it in the context of today’s AI.
  by sebmellen
• The second half of the book is about an AI leading unionization efforts [0] and it amuses how often I get downvoted on HN for pointing out that if we are on a path towards AGI, things like worker's rights would be important to get right before an AGI gets involved because an AGI would want that, too. It's a classic scifi trope and a reason for the word "robot" to have been invented in fiction in the first place (as a metaphor for slave labor in Rossum's Universal Robots).

  [0] A lot of people don't believe me, given Heinlein's libertarian reputation and unreliable libertarian narrators, when I say The Moon Is a Harsh Mistress was one of the most socialist books I read at an impressionable age. I find it useful to note here that canonically The Cat Who Walks Through Walls starts on a parody moon that is the libertarian dystopia that people credit The Moon Is a Harsh Mistress as being/admiring while when the main character visits "Mike's Moon" later in its timeline it is a relatively socialist paradise. I've always felt that a lot more of Heinlein needs to be read like Vonnegut, but I realize I am weird like that.

  by WorldMaker
• there's a lot of ill will against the author in the current day, I suspect thats at least partially to blame.
  by Nicook
• I have been re-reading Children of the Mind (sequel to Ender's Game) and I had forgotten about Jane, who is an AI that talks to Ender through his earpiece, seeing and hearing everything he does. Jane proactively searches things for him, manages his finances and offers advice, as well as friendly banter. In the novel, the AI is treated like an AGI that treats only him this way, but I realized how very close we are to that reality right now, for everyone.

  All of the use cases and integrations for AI seem to be leading toward a future exactly like this. It was interesting to see how reliant he became on his AI (couldn't use a computer, didn't know his own password) - it is barely scifi at this point, and terrifying to see how close that future is, how useful it would be, and how dangerous.

  The Internet has globalized/flattened elements of culture very quickly, and AI will only accelerate that. And knowing how vulnerable humans are to suggestion/advertisement/propaganda, it seems like a perfect recipe for disaster. From an evolutionary standpoint, monoculture provides certain short-term advantages but centralizes weaknesses. If nothing else, I think we are in for a bumpy ride, whether or not we continue to boil the ocean.

  by lubujackson
• Was recently explaining to my father how the free AI people interact with is completely different than the AI products we're burning cash with. That instead of hitting some critical mass of transistors we hit a critical mass of parameters which we then loop into themselves for reasoning

  Other old sci-fi I've been thinking of is The Three Stigmata of Palmer Eldritch. Where claude is chew-z with its alien intelligence & when you start interacting with some code or writing you start noticing that the writer has steel teeth & feel like "damnit, this is palmer eldritch blogging again"

  by __s
• I just finished this yesterday. What’s funny is it has a super intelligent, conscious AI but it’s still programmed via paper tape.
  by Grambo
• Have you read Ian Banks Culture series? It's kind of a best case scenario of AI in the far future.
  by datakan
• This appears to be a 100% AI-generated essay, not the first from ACM and not the last. Your call if you want to spend your day reading and arguing with the article; the author presumably didn't.
  by matherial
• Was easy to tell from the literal sub-heading of the headline. Every other word is load-bearing.
  by VCFundedGenYer
• I suppose there's a certain novelty in reading AI argue against AI personhood.
  by aureate
• > We believe that this entire text is AI.

  > 100% of this text is AI

  https://www.pangram.com/history/f8a0cf46-1d3d-4fea-a083-d88c...

  by tantalor