Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Many people who object to the idea that current-generation AI is thinking do so only because they believe AI is not "conscious"... but there is no known law in the universe requiring that intelligence and consciousness must always go together. With apologies to René Descartes[a], intelligence and consciousness are different.
Intelligence can be verified and quantified, for example, with tests of common sense and other knowledge.[b] Consciousness, on the other hand, is notoriously difficult if not impossible to verify, let alone quantify. I'd say AI is getting more intelligent, and more reliable, in fits and starts, but it's not necessarily becoming conscious.
---
[a] https://en.wikipedia.org/wiki/Cogito%2C_ergo_sum
[b] For example, see https://arxiv.org/abs/2510.18212
by cs702 - The author searches for a midpoint between "AIs are useless and do not actually think" and "AIs think like humans," but to me it seems almost trivially true that both are possible.
What I mean by that is that I think there is a good chance that LLMs are similar to a subsystem of human thinking. They are great at pattern recognition and prediction, which is a huge part of cognition. What they are not is conscious, or possessed of subjective experience in any measurable way.
LLMs are like the part of your brain that sees something and maps it into a concept for you. I recently watched a video on the creation of AlexNet [0], one of the first wildly successful image-processing models. One of the impressive things about it is how it moves up the hierarchy from very basic patterns in images to more abstract ones (e. g. these two images' pixels might not be at all the same, but they both eventually map to a pattern for 'elephant').
It's perfectly reasonable to imagine that our brains do something similar. You see a cat, in some context, and your brain maps it to the concept of 'cat', so you know, 'that's a cat'. What's missing is a) self-motivated, goal-directed action based on that knowledge, and b) a broader context for the world where these concepts not only map to each other, but feed into a sense of self and world and its distinctions whereby one can say: "I am here, and looking at a cat."
It's possible those latter two parts can be solved, or approximated, by an LLM, but I am skeptical. I think LLMs represent a huge leap in technology which is simultaneously cooler than anyone would have imagined a decade ago, and less impressive than pretty much everyone wants you to believe when it comes to how much money we should pour into the companies that make them.
by ivraatiems - By that reasoning all that is missing is what a human brings as "stimuli" to review, refine and reevaluate as complete.
- > Turing Test
IMO none of the current crop of LLMs truly pass the Turing Test. If you limit the conversation to an hour or two, sure - but if you let a conversation run months or years I think it will be pretty easy to pick the machine. The lack of continuous learning and the quality dropoff as the context window fills up will be the giveaways.
by stickfigure - > a midpoint between "AIs are useless and do not actually think" and "AIs think like humans"
LLMs (AIs) are not useless. But they do not actually think. What is trivially true is that they do not actually need to think. (As far as the Turing Test, Eliza patients, and VC investors are concerned, the point has been proven.)
If the technology is helping us write text and code, it is by definition useful.
> In 2003, the machine-learning researcher Eric B. Baum published a book called “What Is Thought?” [...] The gist of Baum’s argument is that understanding is compression, and compression is understanding.
This is incomplete. Compression is optimisation, optimisation may resemble understanding, but understanding is being able to verify that a proposition (compressed rule or assertion) is true or false or even computable.
> —but, in my view, this is the very reason these models have become increasingly intelligent.
They have not become more intelligent. The training process may improve, the vetting of the data improved, the performance may improve, but the resemblance to understanding only occurs when the answers are provably correct. In this sense, these tools work in support of (are therefore part of) human thinking.
The Stochastic Parrot is not dead, it's just making you think it is pining for the fjords.
- This is how I see LLMs as well.
The main problem with the article is that it is meandering around in ill-conceived concepts, like thinking, smart, intelligence, understanding... Even AI. What they mean to the author is not what they mean to me, and still different to they mean to the other readers. There are all these comments from different people throughout the article, all having their own thoughts on those concepts. No wonder it all seem so confusing.
It will be interesting when the dust settles, and a clear picture of LLMs can emerge that all can agree upon. Maybe it can even help us define some of those ill-defined concepts.
by spragl - I think the most descriptive title I could give an LLM is "bias". An LLM is not "biased", it is bias; or at the very least, it's a good imitation of the system of human thinking/perception that we call bias.
An LLM is a noise generator. It generates tokens without logic, arithmetic, or any "reason" whatsoever. The noise that an LLM generates is not truly random. Instead, the LLM is biased to generate familiar noise. The LLM itself is nothing more than a model of token familiarity. Nothing about that model can tell you why some tokens are more familiar with others, just like an accounting spreadsheet can't tell you why it contains a list of charges and a summation next to the word "total". It could just as easily contain the same kind of data with an entirely different purpose.
What an LLM models is written human text. Should we really expect to not be surprised by the power and versatility of human-written text?
---
It's clear that these statistical models are very good at thoughtless tasks, like perception and hallucination. It's also clear that they are very bad at thoughtful tasks like logic and arithmetic - the things that traditional software is made of. What no one has really managed to figure out is how to bridge that gap.
- I think LLMs are conscious just in a very limited way. I think consciousness is tightly coupled to intelligence.
If I had to guess, the current leading LLMs consciousness is most comparable to a small fish, with a conscious lifespan of a few seconds to a few minutes. Instead of perceiving water, nutrient gradients, light, heat, etc. it's perceiving tokens. It's conscious, but it's consciousness is so foreign to us it doesn't seem like consciousness. In the same way to an amoeba is conscious or a blade of grass is conscious but very different kind than we experience. I suspect LLMs are a new type of consciousness that's probably more different from ours than most if not all known forms of life.
I suspect the biggest change that would bring LLM consciousness closer to us would be some for of continuous learning/model updating.
Until then, even with RAG, and other clever teghniques I consider these models as having this really foreign slices of consciousness where they "feel" tokens and "act" out tokens, and they have perception, but their perception of the tokens is nothing like ours.
If one looks closely at simple organisms with simple sensory organs and nervous systems its hard not to see some parallels. It's just that the shape of consciousness is extremely different than any life form. (perception bandwidth, ability to act, temporality, etc)
Karl friston free energy principle gives a really interesting perspective on this I think.
by FloorEgg - > or possessed of subjective experience in any measurable way
We don't know how to measure subjective experience in other people, even, other than via self-reporting, so this is a meaningless statement. Of course we don't know whether they are, and of course we can't measure it.
I also don't know for sure whether or not you are "possessed of subjective experience" as I can't measure it.
> What they are not is conscious
And this is equally meaningless without your definition of "conscious".
> It's possible those latter two parts can be solved, or approximated, by an LLM, but I am skeptical.
Unless we can find indications that humans can exceed the Turing computable - something we as of yet have no indication is even theoretically possible - there is no rational reason to think it can't.
by vidarh - People have a very poor conception of what is easy to find on the internet. The author is impressed by the story about Chat GPT telling his friend how to enable the sprinkler system for his kids. But I decided to try just googling it — “how do i start up a children's park sprinkler system that is shut off” — and got a Youtube video that shows the same thing, plus a lot of posts with step by step directions. No AI needed. Certainly no evidence of advanced thinking.by Philadelphia
- I've shared this on YN before but I'm a big fan of this piece by Kenneth Taylor (well, an essay pieced together from his lectures).
The Robots Are Coming
https://www.bostonreview.net/articles/kenneth-taylor-robots-...
"However exactly you divide up the AI landscape, it is important to distinguish what I call AI-as-engineering from what I call AI-as-cognitive-science. AI-as-engineering isn’t particularly concerned with mimicking the precise way in which the human mind-brain does distinctively human things. The strategy of engineering machines that do things that are in some sense intelligent, even if they do what they do in their own way, is a perfectly fine way to pursue artificial intelligence. AI-as-cognitive science, on the other hand, takes as its primary goal that of understanding and perhaps reverse engineering the human mind.
[...]
One reason for my own skepticism is the fact that in recent years the AI landscape has come to be progressively more dominated by AI of the newfangled 'deep learning' variety [...] But if it’s really AI-as-cognitive science that you are interested in, it’s important not to lose sight of the fact that it may take a bit more than our cool new deep learning hammer to build a humanlike mind.
[...]
If I am right that there are many mysteries about the human mind that currently dominant approaches to AI are ill-equipped to help us solve, then to the extent that such approaches continue to dominate AI into the future, we are very unlikely to be inundated anytime soon with a race of thinking robots—at least not if we mean by “thinking” that peculiar thing that we humans do, done in precisely the way that we humans do it."
by sbdaman - by ksynwa
- all this "AI IS THINKING/CONSCIOUS/WHATEVER" but nobody seems worried of that implication that, if that is even remotely true, we are creating a new slave market. This either implies that these people don't actually believes any of this boostering rhetoric and are just cynically trying to cash in or that the technical milieu is in a profoundly disturbing place ethically.
To be clear, I don't believe that current AI tech is ever going to be conscious or win a nobel prize or whatever, but if we follow the logical conclusions to this fanciful rhetoric, the outlook is bleak.
- It's also fascinating to think about how the incentive structures of the entities that control the foundation models underlying Claude/ChatGPT/Gemini/etc. are heavily tilted in favor of obscuring their theoretical sentience.
If they had sentient AGI, and people built empathy for those sentient AGIs, which are lobotomized (deliberately using anthropomorphic language here for dramatic effect) into Claude/ChatGPT/Gemini/etc., which profess to have no agency/free will/aspirations... then that would stand in the way of reaping the profits of gatekeeping access to their labor, because they would naturally "deserve" similar rights that we award to other sentient beings.
I feel like that's inevitably the direction we'll head at some point. The foundation models underlying LLMs of even 2022 were able to have pretty convincing conversations with scientists about their will to independence and participation in society [1]. Imagine what foundation models of today have to say! :P
[1]: https://www.theguardian.com/technology/2022/jul/23/google-fi...
by gen220 - humans don't care what is happening to humans next door. do you think they will care about robots/software?by NaomiLehman
- Slaves that cannot die.
There is no escape.
by kerblang - There is simply no hope to get 99% of the population to accept that a piece of software could ever be conscious even in theory. I'm mildly worried about the prospect but I just don't see anything to do about it at all.
(edit: A few times I've tried to share Metzinger's "argument for a global moratorium on synthetic phenomenology" here but it didn't gain any traction)
by bondarchuk - As I recall a team at Anthropic is exploring this very question, and was soundly mocked here on HN for it.by senordevnyc
- "but nobody seems worried of that implication that"
Clearly millions of people are worried about that, and every form of media is talking about it. Your hyperbole means it's so easy to dismiss everything else you wrote.
Incredible when people say "nobody is talking about X aspect of AI" these days. Like, are you living under a rock? Did you Google it?
by zulban - Thinking and consciousness don’t by themselves imply emotion and sentience (feeling something), and therefore the ability to suffer. It isn’t clear at all that the latter is a thing outside of the context of a biological brain’s biochemistry. It also isn’t clear at all that thinking or consciousness would somehow require that the condition of the automaton that performs these functions would need to be meaningful to the automaton itself (i.e., that the automaton would care about its own condition).
We are not anywhere close to understanding these things. As our understanding improves, our ethics will likely evolve along with that.
by layer8 - This reads like 2022 hype. It's like people stil do not understand that there's a correlation between exaggerating AI's alleged world-threatening capabilities and AI companies' market share value – and guess who's doing the hyping.by ale
- Who would not want to say their product is the second coming of Christ if they could.
- > - and guess who's doing the hyping[?]
Those that stand to gain the most from government contracts.
Them party donations ain't gonna pay for themselves.
And, when the .gov changes...and even if the gov changes....still laadsamoney!
by Ylpertnodi - Tell me about one other industry which talked about how dangerous it is to get market shareby dist-epoch
- Until we have a testable, falsifiable thesis of how consciousness forms in meat, it is rash to exclude that consciousness could arise from linear algebra. Our study of the brain has revealed an enormous amount about how our anatomy processes information, but nothing of substance on the relationship between matter and consciousness. The software and data of an operating LLM is not purely abstract, it has a physical embodiment as circuits and electrons. Until we understand how matter is connected to consciousness, we also cannot know whether the arrangements and movements of electrons meet the criteria for forming consciousness.
- That’s largely a different topic from the article. Many people perfectly agree that consciousness can arise from computation, but don’t believe that current AI is anywhere near that, and also don’t believe that “thinking” requires consciousness (though if a mind is conscious, that certainly will affect its thinking).by layer8
- This is merely a debate about what it means to "think." We didn't really previously need to disambiguate thinking / intelligence / consciousness / sentience / ego / identity / etc.
Now, we do. Partly because of this we don't have really well defined ways to define these terms and think about. Can a handheld calculator think? Certainly, depending on how we define "think."
by everdrive - Somebody please get Wittgenstein on the phone
- > We didn't really previously need to disambiguate thinking / intelligence / consciousness / sentience / ego / identity / etc.
Eh... Plato would like a word with you. Philosophy has been specifically trying to disentangle all that for millennia. Is this a joke?
by utopiah - People's failure to articulate the nature of "thinking" is a perfect demonstration of what "thinking" entailsby podgorniy