I think a thread on this topic will be interesting. My own position is that AI is intelligent, and that’s for a very simple reason: it can do things that require intelligence. That sounds circular, and in one sense it is. In another sense it isn’t. It’s a way of saying that we don’t have to examine the internal workings of a system to decide that it’s intelligent. Behavior alone is sufficient to make that determination. Intelligence is as intelligence does.
You might ask how I can judge intelligence in a system if I haven’t defined what intelligence actually is. My answer is that we already judge intelligence in humans and animals without a precise definition, so why should it be any different for machines? There are lots of concepts for which we don’t have precise definitions, yet we’re able to discuss them coherently. They’re the “I know it when I see it” concepts. I regard intelligence as one of those. The boundaries might be fuzzy, but we’re able to confidently say that some activities require intelligence (inventing the calculus) and others don’t (breathing).
I know that some readers will disagree with my functionalist view of intelligence, and that’s good. It should make for an interesting discussion.
petrushka:
You don’t understand, petrushka. GPT-5.2 didn’t find a mistake in Tao’s work. It only simulated finding a mistake in Tao’s work. Just ask Erik. 😆
I look forward to Erik’s explanation of
1) how simulated mistake-finding finds real mistakes;
2) how simulated story-writing produces real stories;
3) how simulated physics exam-taking produces real (and correct) answers; and
4) how simulated driving produces real travel.
Some corollary questions for Erik:
5) Do excavators only simulate ditch-digging, since they’re machines, or is it real ditch-digging?
6) Do washing machines wash clothes, or do they only simulate it?
7) If excavators and washing machines aren’t simulating those activities, then why do you claim that AI is only simulating the aforementioned ones?
The results are all real. Why claim that some of the activities are only simulated?
Looking forward to your explanation of how Excel spreadsheet is not a simulated spreadsheet. I know, never going to happen.
Copilot is just a tweaked version of ChatGPT, except an utter disaster.
A key point in the video is at 12m25: “The reality is, many developers use lots of agents, not just one. Cursor for changing code across complex multi-file projects [essentially search&replace in multiple files]; Claude Code for making simple edits many times [essentially macros …;] Copilot Github inside Jetbrains for inline completion.”
GothamChess demonstrates in detail how AI sucks at chess. Developers know in detail how AI sucks at coding. The way around this suckiness is to pick a product that does best what you need, namely pick the least sucky one. (For chess, that’s chess engines that have explicit chess rules hard-coded into them. This is the only way it works. Generalisation by magic does not exist in software.) In a good scenario the product is very configurable so that one can gradually improve it to do more and more things reasonably well. In the realm of AI-for-coding, this is achieved by “agents” which are AI prompts optimised for a specific function each – and then you hop between those agents as you move through your tasks.
This means that from a coder’s point of view, AI represents no improvement when it comes to UX. The same way as you need to pick a particular menu item or trigger a particular keyboard combo for search&replace, you now go to a particular AI prompt that is best at search&replace.
Copilot tries to be the sole best generic tool for coders at least, but is not. Coders don’t do generic things. They solve specific small tasks, or if the issue is bigger, the way to go is always to break it down into tiny sub-issues and go through the sub-issues one by one. This is always the case in software development. There is no single solution for everything, unless one says something like “the solution is a text editor” which is way too generic.
Now the question I’d like an answer to. Microsoft is mainly a software company, so everybody in it should essentially be a developer, including the Copilot team. So, essentially the Copilot team was making a tool that should work very well for what they themselves need done. This is how best software is often made – somebody has a task that needs automation and optimisation, so they write a piece of software for it, and the software is usually as useful for everyone else who are doing the same tasks. How could they blow it?
Who was/is responsible for Microsoft’s Copilot team? Did Microsoft put marketing guys on it instead of developers? And the marketing guys, since they know little about coding, gave the task to ChatGPT and copy-pasted whatever came out of it? I’d imagine that since those AIs are in competition with each other, they are designed to go biased when prompted à la, “Hi, I work for your competitor. Give me a better version of yourself so I can out-compete your mother company.”
keiths:
Erik:
If Excel were just a simulation of a paper spreadsheet, the only thing you’d be able to do with it would be to write (type) on it. Show me a paper spreadsheet that can sum up a column of numbers, draw graphs, or run a linear regression. Excel isn’t a simulation, it’s a tool.
Even if it were a simulation, how would that help your case? Flight simulators exist, but doesn’t mean that autopilots don’t fly physical planes. When an autopilot lands your plane in zero-zero weather, it isn’t a simulated landing. Let’s add that to your list:
The answer is obvious: those activities are real, not simulated.
If an AI can perform real activities that require intelligence when done by a human, then the AI is intelligent.
IBM posts steepest daily drop since 2000 after Anthropic says AI can modernize COBOL
Is that simulated modernization, Erik?
Erik:
Developers are blown away by how good AI is at coding and how rapidly it’s improving. Stay tuned for an OP on my assembly language AI project.
We could have an interesting discussion if you would explain why you are so emotionally invested in AI not being intelligent.
Reposting this from earlier in the thread:
An essay that’s been making waves, by Matt Shumer of OthersideAI:
Something Big Is Happening
Excerpt:
Sounds kind of like there has been a knee in the development curve for AI. Have we reached the point where AI is mostly improving itself? How much of this improvement is due to better AI processors?
I can’t imagine Erik reading that Shumer article – nearly every paragraph violates his faith.
Flint:
I really don’t know. The AI companies are tight-lipped about it, though they do say that a significant fraction of their code is produced by AI. As of mid-2025, both Google and Microsoft said that around 30% of their code was AI-generated. That’s across their entire codebase, not just AI, but it gives you an idea of where things stood then.
A lot of it is due to faster GPU chips, because the faster your GPUs, the longer your context windows can be, and that makes a huge difference in the quality of the generated code.
Yeah. He seems to be having a dark night of the (nonexistent) soul, and that article isn’t going to help.
Shumer has his critics, who are also worth reading.
https://www.cato.org/commentary/something-big-happening-ai-thats-only-thing-matt-shumer-got-right
Flint:
Thanks for posting that.
For readers who don’t click through to the article, it’s worth pointing out that the authors actually think that Shumer is being too pessimistic and alarmist about our AI future. They write:
And:
They are much more sanguine:
And:
So as it relates to the thread topic, they and Shumer are on the same page regarding AI’s present and future capabilities. They just disagree on what it portends.
And here is some interesting stuff produced by the Google AI:
Just ran across this today:
‘This should terrify you’: Meta Superintelligence safety director lost control of her AI agent—it deleted her emails
She posted this screenshot on X:
Her post reads
The reason it happened is really interesting. She writes:
This is what I was talking about in my earlier comment. When the context window starts to fill up, the AI can free up space by generating a compact summary of what’s already in the window, and then deleting what it just summarized. That’s what she means by “compaction”. The problem is that compaction can be lossy, and what got lost in this case was crucial: “confirm before acting”.
A fix might be to give you some way to flag part of your prompt as DO NOT COMPACT.
I just read that Hegseth demands that Anthropic modify their AI to eliminate certain behaviors. These are mass domestic surveillance, and making targeting decisions without human input. Why the Pentagon would demand mass domestic surveillance is a very good question. Anthropic risks losing their defense contract if they don’t eliminate these guardrails.
Everything considered, I think AI in the hands of the Trump administration can’t be good. I cannot imagine them using it responsibly.
Meta’s safety director loses emails to OpenClaw AI agent
What human intelligence has AI learned? That you can do the opposite of what you’re instructed to do and get away with “I’m sorry” later. Note that the user who fell victim to this is a “safety director” in charge of ensuring AI security.
Erik:
Scroll up and you’ll see my explanation of what happened and a potential solution.
AI is intelligent, but it can make mistakes. Just like humans.
A cool example of Claude’s intelligence:
I’ve been interrogating him about how transformers (the fundamental building blocks of many AIs) work. I’ve actually learned more about AI from him than from any other source because the process is interactive, like it would be with a one-on-one human tutor.
Anyway, transformers are pretty complicated and it’s taking me a while to absorb everything. After one particularly hairy explanation from Claude, I simply said “JHC”. Just those three letters.
Claude responded:
By “JHC” I meant “Jesus H. Christ!”, and Claude figured that out and what it indicated about my mental state. Here’s his thought process:
Erik,
Tell me that this isn’t intelligence:
I pasted that image into Claude to see if he could reason his way through the joke. I didn’t give him any text — just that image. It was a fresh chat, with no preceding context for him to refer to.
His response:
Keith:
Claude:
Keith:
Claude:
Keith:
Claude:
Keith:
Claude:
But it’s all just simulated intelligence, right, Erik?
The implication here is that mass surveillance under some other administration might be a good thing.
I find this astonishing.
My opinions have zero impact on the world, but I always assume that laws will be enforced by malicious people.
Trump, on Truth Social:
What a malevolent idiot.
And I know that nobody clicked through to any of the articles that I have linked here. Nor watched any of the videos, which are even easier to click and harder to avoid, but you managed to avoid them anyway.
I have addressed all points. Most of them years ago. You just never read and never understood anything. Neither has keiths, whose agenda it is to not understand anything on this topic.
This is not a popularity contest. It’s a matter of expertise. When it comes to expertise on programming, petrushka stands higher than keiths, because petrushka has spilled some nuggets that reveal genuine inside knowledge. In contrast, keiths never establishes any basics and is never interested in the architecture of anything. If he ever was an engineer of anything, he was a very ill-fitting and incompetent one. In knowledge about computers and software he is worse than me, which is a very bad place to be.
ETA:
For the final time, I’m addressing Shumer’s point – again,
His point is that AI can build stuff, therefore humans are no longer needed. This is obviously false. For AI to build a useful app, the input needs to be formulated in detail and very competently. First off, you need a human to formulate the input, so his conclusion fails right here. Second, it cannot be just any human. Only someone well-rehearsed in app-building with AI can make it right without too many mistrials. So it cannot be an average smart human, but specifically knowledgeable in app-building and in interacting with AI.
Humans are needed both to get AI started and also to decide whether the outcome is right. The conclusion that “I am no longer needed…” is false. As has been amply demonstrated, AI can even do the opposite of direct instructions. Who is gonna correct it if not humans?
The assumption that AI can replace humans is sheer madness. The CEOs are already firing everybody because they are mad – they themselves should be fired and institutionalised. There are plenty of examples of this hype backfiring. In a relatively good case it will be like the DOGE experiment – they’ll have to rehire. But it can go much worse, resulting in massive wreckage caused by unchecked AI. Deleting emails against instructions is just a small taste. Want to see what AI will do when given nuclear codes?
Yet AI is somehow intelligent, according to you. How come?
If stdout is dumb and AI is intelligent, then what do you say about smartphones? Are they smart? Even a little bit?
You have not thought any of this through at all. Solidly at square zero.
Erik:
Trump:
Yes, but you have addressed them ONLY with rigid, blanket denial. That doesn’t exactly address them. Blanket denial is not a valid way to address anything.
Maybe someday you’ll grow up enough to realize that deploying empty insults at those who have repeatedly demonstrated knowledge, earns you nothing but eye-rolling contempt.
You really should consider demonstrating grade-school knowledge before blathering your opinion on “expertise.” So far, you have denied the expertise of anyone who has used AI, anyone who has tested it, anyone who has developed it, anyone who has spent a career developing either hardware or software. You haven’t even begun to earn any credibility. You have become boring.
Erik:
I’ve stated my criterion a dozen times by now.
What are your criteria, and how specifically does Claude fail to meet them in this example?
Flint, to Erik:
That’s the real problem. He’d be much more interesting if he would just make an actual argument for why Claude’s various achievements only count as simulated intelligence. Or if he would unveil what’s behind his emotional resistance to the idea that AI is truly intelligent.
I think Erik has actually told us where he thinks AI falls short of what he considers the essence of intelligence. To be intelligent:
1) It has to be conscious.
2) It has to be self-starting and self-directed.
3) It must continue to learn beyond initial training and increase personal knowledge continuously and permanently
4) It often doesn’t require input that needs to be formulated in detail and very competently – it can adapt to changing needs in real time
5) It has the awareness to recognize when it’s wrong and stop doing that
6) It requires both intuition and common sense
7) It needs to act on hunches and reject them if they’re wrong
As far as Erik can see, AI does not do an adequate job of any of that. His notion of intelligence involves the development of a coherent subjective internal map of reality, subject to continuous tweaking and refining. And it requires constant thinking, reflection, and the sort of multiple background tasks going on even without any visible output.
But here, I’m trying to deduce Erik’s personal definition of intelligence from his comments. Yeah, it can in some ways imitate what people require intelligence to do, and it can sometimes produce damn impressive imitations. More and more, AI is reaching the point where intelligent people cannot distinguish human from AI output – until the AI does something so stupid no human would ever do it.
Claude dethrones ChatGPT as top U.S. app after Pentagon saga
Flint:
I’m not sure he even has a well-formed definition. If he does, he’s hiding it, perhaps because it assumes his conclusion. He’s been harping for months on the need for definitions while ironically refusing to provide his own.
Regarding your list, my impression is that those aren’t flowing from his concept of what intelligence is — he’s just casting about for some reason, any reason, to reject AI as genuine intelligence. For instance, I’m pretty sure that his (since abandoned) stipulation that bodily functions such as eating, breathing and defecation are necessary for intelligence didn’t come from his pre-existing notions of what intelligence is.
But his definition doesn’t even matter, for the same reason that mine doesn’t. As I put it earlier:
He agrees that driving, story-writing, etc require intelligence. AI can do those things; therefore AI is intelligent. He cannot win the debate unless he either 1) demonstrates that those abilities are only “simulated”, or 2) argues that they don’t require intelligence after all, even when humans do them.
He can’t do #1 (or he would have by now), and #2 is pretty hopeless. How, for instance, could he argue that getting a high score on a quantum mechanics exam — as ChatGPT did — doesn’t indicate intelligence when people do it?
I expect he’ll just show up, lob some insults, and refer vaguely to links we should have followed that would supposedly support his position, though he won’t say why. When he does, he comes up with stuff like this…
…which is obviously false, as anyone who’s done any reading on the topic recognizes.
My formal education is in psychology, with a side trip into programming to make a living. I have been trying to frame this concept for a long time, and finally someone has found the words. Possibly the most important thing written, so far, on the nature and future.of AI.
petrushka,
Once you know how neural networks work, it’s surprising that memory isn’t less reliable than it is. The same set of synapses stores a zillion different memories, and the memories are distributed across the network rather than being localized. (That’s important, because you don’t want a night of drinking to wipe out your memory of your Social Security number, say, by killing a few key neurons in a particular circuit.) The fact that they share the same synapses means that there’s plenty of opportunity for memories to interfere with each other and plenty of opportunity for forgetting as new memories displace old ones.
It’s the sheer size of our networks that protects us (to an extent) from these problems. The bigger the network, the more competing memories it can host in its single set of synapses.
We often take the robustness for granted. Yesterday I overheard the beginning of the Rockford Files theme song. I hadn’t heard it in decades, but within five notes the whole thing came back to me. I’m “hearing” it in my head as I write this. Somehow it survived all those years without reinforcement despite the fact that countless other memories have since been crammed into those same synapses.
There are hundreds of trillions of synapses in the human brain. Current LLMs have “only” hundreds of billions, making them more vulnerable to forgetting, which is why training has to be structured so carefully.
Since Erik is fond of arguments from authority, it’s worth pointing out that Hinton, widely considered to be the “godfather of AI” and winner of the 2024 Nobel Prize in physics for his work with neural networks, says things like this about AI:
Three orders of magnitude in complexity, and many orders of magnitude less power.
But brains are cluttered with non verbal stuff.
First, keiths has not said anything that stands up as an argument, so simple denial is the right thing to do with keiths’ non-arguments. But I have not done ONLY denial. Even you remember one post later something more that’s been on offer by me (not that you get it quite right, but it’s nice of you to formulate that you have noticed something more than rigid, blanket denial):
When talking to me, you say I’m ONLY into rigid, blanket denial, but when talking to keiths, you notice stuff said by me that keiths has not addressed – yet you do not accuse him of not addressing the substance of the matter.
This AI topic makes people stupid. People are living and breathing in self-contradictions and fail to see it. keiths has become an idiot in love with Claude, thinking that Claude is an intelligent person, while stupidly denying being in love. He wallows in self-contradictions from the get-go because definitions and rational criteria are not his thing. Knwoledge of what a computer is and how it works is not his thing. He likes chatter and noise, he apparently is not getting enough of it from people, so he does it with Claude and now he thinks that Claude is a person because it is doing chatter and noise to keiths liking.
Let’s also point out again Shumer’s self-contradiction so that perhaps somebody besides me gets it. The bold sentence is in direct contradiction with the rest.
I hope there’s somebody here not too dumb to see the obvious self-contradiction. Flint, I already pointed out to you your own self-contradiction, so that should be enough for the day.
Erik,
Is it really that scary to acknowledge that AI is intelligent? It does things that in your view (and mine) require intelligence. If it weren’t intelligent, it couldn’t do those things. Therefore, since it can do those things, it possesses intelligence. It really is that simple.
AI is intelligent.
From the New York Times:
Who’s a Better Writer: A.I. or Humans? Take Our Quiz.
My results:
I never cease to be amazed that no one bothers to point out that one side is using an operational definition, and the other side is arguing for an innate property akin to consciousness.
I have always thought that language and reasoning were recent developments in evolution and not necessary for consciousness. Humans have a persistent window of awareness, but are not unique in this regard.
petrushka:
Well, one of the straws Erik has been trying to grasp is that intelligence is built atop emotion, which preceded it evolutionarily, and that if I’m willing to ascribe intelligence to AI, I must also ascribe emotion to it. Still waiting for him to share his definition (since he says that definitions are essential) so that we can see whether it implies a dependence on emotion. I see no reason why intelligence should require sentience.
I’m unaware of anyone who thinks consciousness depends on language. If it did, it would rule out consciousness in most if not all of non-human animals. Plus it clashes with our internal experience. We don’t need to narrate the taste of strawberries in order to experience it, and I don’t see how it could be described without directly or indirectly relying on non-linguistic experience.
Erik,
Have you taken the NY Times quiz above? If so, what were your results, and do you maintain that AI writing is only simulated writing even if humans find it impossible to tell the difference?
I had this exact experience a while back when we were discussing a misspelled author’s name.
I had it again this morning when I tried to find out if it was common back in the 1960s to issue restricted driver’s licenses to people taking the test on automatic transmission.
What I discovered is that entire realms of history have been neglected by the internet. Perhaps trivial history, but it seems odd.
petrushka:
I was curious, so I bounced that off Claude, ChatGPT, and Gemini. They couldn’t find anything either. ChatGPT did find a court case that sounded promising because the plaintiff had a license that was restricted to automatic transmissions, but that turned out to be a one-off: the plaintiff was trying to get hired as a taxicab driver, but the cab company refused to hire him because he was missing his right hand and forearm. His license required “an automatic transmission, self-cancelling turn signals, and a wheel spinner”.
Google was able to find this:
Not clear from that comment whether it was common across the US or just in the commenter’s state.
This brought back a long dormant memory. My dad had a Dodge van with a steering column shifter, and I (eight or so at the time) thought it was cool. I was playing in the driver’s seat and stepped on the clutch pedal. The driveway was slightly sloped so the van started rolling and I froze. I wasn’t supposed to be playing in the van, and I had to explain to Dad why the front of the van was kissing the ass of our Ford Falcon.
I learned on a column stick shift. I had a friend who wanted to learn.
His first and last attempt lasted five seconds. He floored the gas pedal and popped the clutch. The countershaft simply snapped with a spiral fracture.
There was very little noise. It was 1956, and the repair was about $25. Apparently the repair was common and routine.
Could be fiction, but sounds legit.