Alexander Lerchner, a researcher at Google’s DeepMind division, posted a preprint article a couple of months ago titled “The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness”, in which he makes the strong claim that computationally based AI will never be conscious. Here’s the abstract:
Abstract
Computational functionalism dominates current debates on AI consciousness. This is the hypothesis that subjective experience emerges entirely from abstract causal topology, regardless of the underlying physical substrate. We argue this view fundamentally mischaracterizes how physics relates to information. We call this mistake the Abstraction Fallacy. Tracing the causal origins of abstraction reveals that symbolic computation is not an intrinsic physical process. Instead, it is a mapmakerdependent description. It requires an active, experiencing cognitive agent to alphabetize continuous physics into a finite set of meaningful states. Consequently, we do not need a complete, finalized theory of consciousness to assess AI sentience—a demand that simply pushes the question beyond near-term resolution and deepens the AI welfare trap. What we actually need is a rigorous ontology of computation. The framework proposed here explicitly separates simulation (behavioral mimicry driven by vehicle causality) from instantiation (intrinsic physical constitution driven by content causality). Establishing this ontological boundary shows why algorithmic symbol manipulation is structurally incapable of instantiating experience. Crucially, this argument does not rely on biological exclusivity. If an artificial system were ever conscious, it would be because of its specific physical constitution, never its syntactic architecture. Ultimately, this framework offers a physically grounded refutation of computational functionalism to resolve the current uncertainty surrounding AI consciousness.
I think his argument fails, but it’s worth a thread to discuss why, and I’m sure it will lead to a more general discussion of the possibility of AI consciousness. I also think a separate thread is warranted because consciousness is distinct from intelligence (the latter being the topic of the “Is AI really intelligent?” thread).
Quoting the OP:
Does it? Anyone?
Alan Fox,
If Keiths thinks it fails he should be able to articulate why. So many unknowns here.
Someone at Anthropic said AIs spontaneously develop processes that parallel emotional states.
keiths has his non-reasons to claim this (that he wisely leaves unstated precisely because they are non-reasons). Also,
Yup, undefined intelligence is separate from consciousness given some more non-reasons.
Good I kept my expectations low. It is long overdue that keiths gets to a definition of something, let’s say a definition of software, of simulation, or the like, if not intelligence and consciousness. Since he has already botched the distinction of feelings and intelligence, my expectations will stay low.
The paper cited in the OP engages with “Could a Large Language Model be Conscious?” by David Chalmers https://arxiv.org/abs/2303.07103
David Chalmers’ article proposes to investigate the incident of Blake Lemoine falling in love with LaMDA, “When Blake Lemoine, a software engineer at Google, said in June 2022 that he detected sentience and consciousness in LaMDA 2, a language model system grounded in an artificial neural network, his claim was met by widespread disbelief…. The question of evidence piqued my curiosity. What is or might be the evidence in favor of consciousness in a large language model, and what might be the evidence against it? That’s what I’ll be talking about here.”
If only keiths went by defining his terms and following the evidence, but no such luck, alas…
Alan:
colewd:
Erik:
Patience, folks. Of course I’m going to explain why I think Lerchner’s argument fails. I didn’t start a thread on the paper in order not to discuss the paper, after all.
Erik:
Longtime observers of Erik will be unsurprised that I have explained multiple times how AI can be intelligent without being conscious, to which he has never responded. Emotions can be faked but intelligence cannot, as explained most recently here. In that comment, I quote part of Claude’s system prompt, which instructs the underlying LLM to take on the role of “Claude”, a helpful AI assistant:
The LLM is an actor playing the role of Claude, improvising as it goes. The emotions it expresses are feigned. The intelligence it evinces is not, and cannot, be faked.
Likewise, I’ve explained multiple times to Erik why a precise definition of intelligence isn’t needed for my argument to succeed. If AI can perform some particular activity X that we agree requires intelligence, then we can conclude that AI is intelligent. Erik claims that for any such X, AI only simulates it. I can’t get him to explain why:
Erik:
Erik is being tabloid here. Lemoine didn’t fall in love with LaMDA, he just claimed it was sentient.
petrushka:
More precisely, they develop emotional concepts and understanding that they can use in playing their roles. From Anthropic’s article on this:
The role-playing requires intelligence, but the emotions are simulated — for now.
I think Bill has presented a good description of consciousness:
The OP makes the claim that
I think this is saying that the artificial system needs a whole different category of inputs and internal responses to those inputs. The implication is that sensory experience is required as an essential data source. Quite possibly, an intrinsic fear of injury or death, and the necessary intrinsic curiosity to evaluate surroundings in order to better anticipate injury or death, is something a consciousness can’t do entirely without. Along with the ability to do something appropriate when danger to the individual is perceived.
In the other thread, Erik wrote:
As I pointed out then, Lerchner is talking about the simulation of consciousness, not of intelligence, so when Erik says “the article is entirely on my side”, he is misunderstanding Lerchner’s project.
He’ll be further disappointed to learn that Lerchner doesn’t deny the possibility of machine consciousness. He’s arguing against computation as the basis of consciousness, but not against machines. He makes his physicalism clear:
He states that in his view, consciousness isn’t limited to biology:
This will disappoint Erik, who believes that something nonphysical is behind intelligence and consciousness,* though he refuses to say what it is. It does hint at why this debate is so emotional for him.
* And even behind arithmetic. He once complained about my “false materialistic notion of arithmetic”. Yes, Erik actually believes that calculators and computers cannot perform arithmetic since they are purely physical devices.
To evaluate Lerchner’s paper you need to understand his somewhat idiosyncratic terminology. Here are some notes:
Lerchner writes:
By ‘causal topology’ he really just means ‘pattern of information processing’. The idea behind functionalism, which Lerchner rejects, is that if two systems are identical at the information processing level and one of them is conscious (a human, say) then the other will be conscious too, regardless of how it is physically implemented. A transistor-based system will be as conscious as a neuron-based one as long as the pattern of information processing is the same in both systems.
By ‘alphabetize’ he doesn’t mean anything like ‘put in alphabetical order’. He’s using it to mean the same thing as ‘quantize’ in standard engineering parlance. You’re taking a continuous process or variable and chopping it up into discrete states, the entire collection of which forms the ‘alphabet’ with which you do your symbolic computation. A concrete example is digital circuitry, where the 1s and 0s are represented by high and low voltages respectively. According to one industry standard (TTL logic, for the curious), any voltage between 0.0 V and 0.4 V is interpreted as a 0 while any voltage between 2.4 V and 5.0 V is interpreted as a 1. All bets are off for voltages between 0.4 V and 2.4 V. Those voltages are considered undefined and to be avoided. So the continuous voltage range between 0.0 V and 5.0 V has been chopped up — ‘alphabetized’ — into three discrete states: 0, undefined, and 1.
The entity doing the alphabetization is what Lerchner dubs the ‘mapmaker’. If there’s no mapmaker, there’s no alphabetization, and if there’s no alphabetization, there’s no computation. It’s just a circuit operating with continuous voltages in which those voltages have no meaning.
By ‘vehicle causality’, he’s referring to the fact that in a computational system, the laws of physics dictate how the state of the system evolves over time. It’s just physics, nothing else. The ‘content’ — the meaning of the states — is really just along for the ride. It’s a passive passenger on the physical ‘vehicle’ and has no causal role.
‘Content causality’ is his term for a situation in which the meanings do have causal power. While he denies that this is possible for computational systems, he thinks it is possible for conscious systems, though he justifies it in a handwavy fashion based on ‘intrinsic physical constitution” which I’ll discuss later.
‘Syntactic architecture’ here is basically synonymous with ‘causal topology’above. He’s saying that consciousness doesn’t depend on the nature of the symbolic computation, but it does depend on the precise physical implementation. It’s either present or absent based on that implementation and not on the pattern of information processing/’causal topology’/’syntactic architecture’. Two systems can process information in the same way and according to the same pattern, but it’s possible that only one of them is conscious while the other isn’t, depending on its physical implementation.
Evolution is a miracle and this project is NOT beyond nonsense….
Thousands of years of pondering have not shed much light on the concept of self awareness.
I struggle with the Turing test when applied emotions. Writers of fiction have struggled for at least a century with the potential humanity of robots.
If we keep making more powerful simulations, the problem will become more acute.
Seriously now, the hard problem of consciousness reaches beyond what we as humans can comprehend at this point…
I don’t like it but this is the way it is. Science reaches it limits agin.. but few are willing to acknowledge this fact because of the usual reasons….
J-Mac:
The majority of philosophers acknowledge that the Hard Problem hasn’t been solved, and I’d wager that the majority of neuroscientists believe that too, though I don’t have hard data for the latter. It isn’t just a few.
Solved? Tell me how it was solved… details
??? keiths source says the hard problem has NOT been solved. Personally, I doubt it’s possible to solve without the appropriate physical substrate.
Chemistry is, and always will be, faster than simulations of chemistry.
Awareness begins at the level of tropisms. Layers and layers, created by billions of years, and a lot of fortituity, are required to build consciousness,
Brains ruled the world long before language. Language is just a recent layer. LLMs are as good as, or better than, humans at language. But they lack the old brain, the evolved layers of encoded experience and motivation.
AI is good at revealing what people have written about those layers, but it is not in the evolutionary loop. and it will be hard to catch up, because brains are chemistry. They are faster and more energy efficient than simulations.
I think the energy part is, and will continue to be, the bottleneck.
End of pointless rant.
As keiths has no definition of intelligence, he cannot tell what the article is about. I will sort it out for him now, but he will not understand that either.
And since keiths does not do definitions, let’s have Lerchner flesh out what computation is. Namely, in the abstract of the article: “Tracing the causal origins of abstraction reveals that symbolic computation is not an intrinsic physical process. Instead, it is a mapmaker-dependent description.”
Further in the article: “To count as computation, continuous physical dynamics must be partitioned into a finite set of discrete, semantically meaningful states (i.e., a form of alphabet). Such semantic partitioning logically requires an active, experiencing cognitive agent, which we define as a mapmaker, to contrast it with the passive connotation of a standard ’observer’. It is the mapmaker who performs this alphabetization… There is no ghost reading the alphabet; the living experiencing subject enacts it.”
So, Lerchner’s view is that since computation is an operation on discrete symbols that do not exist in nature as such, but must be crafted, then computation is necessarily performed by a conscious entity called “mapmaker” and not by a machine. The term “mapmaker” relates to the famous map-territory analogy, which is used in the article throughout.
The territory is the physical reality. The map is the model of it – *simulation* and not instantiation, as per the title of the article. And the modelling (i.e. ‘computation’) requires a mapmaker, a living, breathing and experiencing cognitive agent. If there is a map, then there is also a mapmaker.
Having argued with keiths for years now about the intelligence of AI, I know that the map-territory analogy is beyond his comprehension. So here’s another analogy that tracks closer to keiths’ level: a pen and writing.
In keiths’ analysis, AI is intelligent like a pen that writes. To keiths it is self-evident and obvious that pens write. (Nevermind the hand that holds and drags the pen and nevermind the person behind the hand.) In keiths’ mind, it is indisputable to everybody who takes a close look as per keithsian methodology that it is the pen that leaves the trail of ink on the paper, the trail of ink is what people call writing, therefore it is the pen that writes! Truly and really so! Writing is as writing does!
It is ironic to see keiths whine when somebody else does not do definitions to his satisfaction – ironic because this is keiths we are talking about, the One who does not do definitions at all.
It is true that Lerchner states that in his view consciousness isn’t necessarily limited to biology. However, he does not cite any instantiation of consciousness in anything non-biological for the obvious reason that there is none to be found. He says that consciousness *could* be found elsewhere, leaving it entirely open where that could be, and he does not say that it *is*. He says it just to be cool among academic materialists.[1] What Lerchner’s article is demonstrably arguing though is that there is no consciousness in AI, which happens to be entirely on my side.
Similarly, I believe that pens do not write. If keiths believes otherwise, it is up to him to argue his position from ground up.
But here’s another thing I believe: keiths will never define a single term and never argue a single point, never attempt to examine his premises, never consider the implications of his position, and he will not even begin to understand any other position, such as Lerchner’s. I believe this simply because I believe that keiths will continue to be who he always was, the kind of keiths whom we know and love for many years now. He is the ultimate champion of square zero.
[1] Here is the relevant quote in Lerchner’s article, “…the framework does not imply that consciousness must be limited to biological life. In principle, a non-biological system could be designed to realize the necessary physical conditions.” It literally says *could*, not is. And the quote is at the end as an aside. In the main body of the article, there are statements like,
– “In contrast to substrate independence [i.e. an opposing view], this view [i.e. Lerchner’s view] makes biology central rather than incidental.”
– “Patients with mechanical hearts often suffer subtle, systemic physiological deficits precisely because the device instantiates only the coarse-grained map of the selected function. It fails to instantiate the full biological territory of the organ.”
– “The limits of physical simulation elsewhere in biology make the point explicit.”
Erik,
Thanks, Erik. You’ve convinced me that, without an agreed set of definitions, meaningful discussion is impossible.
Going with Erik’s analogy of writing, in the case of AI, who or what is the hand that holds the pen? Is it the collective of programmers who wrote the software plus the collective of users who ask the questions?
Sigh. Lerchner says AI is not conscious. Keiths says AI is not conscious. Erik says AI is not conscious. Nobody has said it is.
Erik still cannot grasp the difference between intelligence and consciousness.
Erik has already explained that if he does not know how the pen was held, he cannot determine if intelligence was involved. No matter what is written.
And this seems to be an increasingly serious problem. Professors can’t tell who wrote the essays, who took the tests, who did the analysis. Internet users can’t tell which articles, or even videos, are AI generated. Mathematicians agree that some AI proofs (that no human could produce) show amazing intelligence – but people might have been involved. So there are serious problems because AI is intelligent. Consciousness is something very different.
Keiths,
The hard problem of consciousness is not solved as you have confirmed because it can’t be solved… What I mean by that is that your assumptions are incorrect because they don’t fit other assumptions of the evolutionary theory, which is just the denial of the the obvious…
I’m not sure what you are trying to say. Yes, the problem of consciousness is a hard problem, but clearly it has been solved by evolution, and many organisms are clearly conscious. But what’s obvious that is being denied?
I should think that since the problem of consciousness has been solved, it’s not an impossible problem to solve. I doubt we are very close to solving it short of biological reproduction.