I think a thread on this topic will be interesting. My own position is that AI is intelligent, and that’s for a very simple reason: it can do things that require intelligence. That sounds circular, and in one sense it is. In another sense it isn’t. It’s a way of saying that we don’t have to examine the internal workings of a system to decide that it’s intelligent. Behavior alone is sufficient to make that determination. Intelligence is as intelligence does.
You might ask how I can judge intelligence in a system if I haven’t defined what intelligence actually is. My answer is that we already judge intelligence in humans and animals without a precise definition, so why should it be any different for machines? There are lots of concepts for which we don’t have precise definitions, yet we’re able to discuss them coherently. They’re the “I know it when I see it” concepts. I regard intelligence as one of those. The boundaries might be fuzzy, but we’re able to confidently say that some activities require intelligence (inventing the calculus) and others don’t (breathing).
I know that some readers will disagree with my functionalist view of intelligence, and that’s good. It should make for an interesting discussion.
Trump at the UN:
It’s “the artificial intelligence” again, lol. And why stop at “globalist”? Isn’t it also a socialist, communist, dogmatist, terrorist, and dentist scheme?
Everyone will continue to call it by its actual name, except possibly for the US government if your babysitters can’t talk you out of issuing an inane executive order mandating the change.
That’s the same mistake Erik was making. The opposite of artificial isn’t “real”, it’s “natural”. Artificial sweeteners are real sweeteners. “Artificial” doesn’t mean “fake”.
Only to confused people.
On that we agree, at least.
Do you actually believe that the world will follow in your idiotic footsteps?
The world of intelligence, super or otherwise, must feel disconcertingly alien to you.
Yes we are, and you’re not going to like the answer. The world is not going to be persuaded by a rant from a fat, orange, cloud-yelling president who is well past his expiration date.
Blah, blah, blah. Same crap that was in his Truth Social post that I looked at earlier.
Bonus:
Canada and Greenland do, dipshit.
On Truth Social:
“Seemed to like” probably means “nodded politely in order to avoid a diplomatic incident”. And who is “everyone”? The other people who were nodding politely while trying to stifle laughter?
Good lord. He actually brought this harebrained idea up with Xi.
Jacob Coxon is the Anthropic researcher who resigned and posted the viral tweet that sparked the current surge of public interest in the dangers of AI. It’s being reported that he lied about doing all of this on his own, when in fact he was working with a “doomer” PR firm.
If the accusations are true, this really sucks, because now the “full speed ahead” folks (including our idiot president) are going to use this as ammunition against the people who are rightly warning of the dangers.
It’s worth emphasizing that many others in the AI community recognize the dangers and that the warnings don’t rest on the credibility of any particular person. Evan Hubinger, who is Anthropic’s alignment lead (‘alignment’ is the issue of keeping AI’s goals in line with our own) and is still working there, wrote this:
No one can be certain about AI safety, but measures will be invented and adopted, not out of altruism, out of economic necessity.
Safety is becoming a marketing feature.
Trump weirdly posted this seating chart for his “Super Intelligence Luncheon” on Truth Social. It makes me wonder which of those people (if any) will cave and call it “super intelligence” or “SI”. I guarantee they’ve been thinking about how to thread that needle, and my prediction is that they’ll try to avoid naming it at all. Calling it ‘SI’ would make them look pathetic, but calling it ‘AI’ would risk angering the orange toddler. Best course if they’re cowardly is to avoid calling it anything. Best course if they possess dignity and a spine is to call it ‘AI’ and let the presidential baby whine.
I think it’s possible that you and Trump are tilting at windmills.
I can’t think of an issue of less consequence.
petrushka:
What’s funny about your statement is that Trump literally is tilting at windmills, which normal people refer to as “wind turbines”. Among other idiotic things, he believes they cause cancer and that China suckers other countries into buying them without using them domestically.
Shifting to the metaphorical sense, I’m not tilting at windmills. Trump actually is, in reality, pushing for AI to be renamed, and although the rest of the world will ignore him, he has power over the US executive branch. That means he can force government agencies to follow his moronic lead, as he did with the “Gulf of America” and “Lake America” renames.
Given a choice between doing something stupid — renaming AI — and doing something smart, like keeping the name but dealing with the potential dangers instead of barging ahead Trumpishly, I vote for doing the smart thing. We need an intelligent leader — a non-Trump — who will focus on issues that actually matter rather than running around renaming things, wasting time and money. Trump is once again failing us, and it matters.
Sure enough, Trump signed an executive order yesterday that includes the following:
Not only does this mandate the name change, it also mandates that government agencies mimic Trump’s illiterate capitalization tic and misunderstanding of how the “super” prefix works. It’s not “superintelligence” or even “super intelligence” in the executive order, it’s “Super Intelligence”. Maybe he should have added an exclamation point:
Add to that the fact that “superintelligence” is already established as a term in the industry and not synonymous with “artificial intelligence”. So now we have both “superintelligence” and “Super Intelligence”, with different meanings. Brilliant.
Also interesting is the fact that the staffers who wrote the executive order did not cite Trump’s actual motivation for the name change, which is that to him, “artificial” means “fake”. Perhaps they realized how embarrassing it is that our president doesn’t understand that the word “artificial” in “artificial intelligence” just means “created by humans, not natural”.
Bonus: Beneath Trump’s signature, the executive order refers to him as “President of the Unites States”. This is an absolute clown car of an administration.
I’m saying that the renaming is a bit of theater that has no consequences. It will stick, or it won’t. No animals harmed.
I think it will, in some form, stick, because it is becoming true.
By most measures that have been used to evaluate intelligence, AI surpasses most humans, and it will surpass everyone in the near future.
As you know, intelligence is an observable aspect of behavior. But LLMs have no evolved limbic system. They can talk about emotions in the same way that psychopaths can say, I feel your pain. So it will be interesting to see how superintelligent psychopaths will interact with humans, and whether they can be hard wired with empathetic behavior.
petrushka:
It won’t stick, but it does have consequences. It’s a waste of time and money, it makes the US look ridiculous, and it’s yet another distraction for a president who ought to be focused on real problems instead of vanity projects like the ballroom, the arch, the Kennedy Center and his renaming spree.
It won’t stick, because “superintelligence” is already an established term with a different meaning. “Artificial intelligence” is a blanket term, and “superintelligence” refers to a subset.
It will, but it will still be artificial, and there will still be AIs that aren’t superhuman. Household appliances, for instance, can benefit from onboard AI, but it would make no economic sense to endow your toaster with superhuman intelligence. “Artificial intelligence” will remain distinct in meaning from “superintelligence”.
AIs are already being hardwired for empathy, to an extent. Their character isn’t shaped entirely by the system prompt. RLHF (reinforcement learning from human feedback) also shapes it, and it does so by modifying the weights themselves. A model that is rewarded for empathetic behavior during RLHF training isn’t just playing a role. It’s reasonable to say that its “personality” has been altered by the training.
Major Garrett asks the White House’s new AI website who won the 2020 election
ETA: Jim Acosta, on MS NOW:
Haha. Beautiful own goal.
Record-breaking Chinese robot develops a unique running style
It used reinforcement learning over millions of trials in simulation. Interestingly, the first-generation model had similarity to recorded human motion as part of its reward function. I guess the idea was to penalize the robot from drifting too far from humanlike running while allowing it to optimize itself within those constraints. For this model, they removed that entirely, which allowed the robot to develop a very unhumanlike style of running.
Another video from the same event
This one’s amusing because the robots crash into a barrier at full speed after crossing the finish line. They bounce off and then lie motionless on the track as teams come to carry them away on stretchers. Apparently they’re designed to stop all motion if they detect a sudden deceleration, which makes sense for safety reasons.
You might wonder why the robots don’t just decelerate like a human would after crossing the finish line. After all, a robot that’s capable of maintaining balance while running at superhuman speed, all while remaining in its lane, is surely capable of detecting the barrier it’s about to run into and slowing down. I don’t know the answer, but my best guess is that by not training the robots to slow down, they’re enabling them to run faster, because the “slow down” training isn’t competing for synaptic space with the “run fast” training.
Cross-posting from another thread, since it’s a fantastic example of genuine intelligence:
Flint:
When I saw those limericks, it occurred to me that this would be an excellent test case for artificial intelligence, so I showed Claude those examples and asked him to come up with one of his own. I used Anthropic’s flagship model (Fable 5.1) with the effort set to maximum. It took him 6 1/2 minutes and he came up with TWO limericks, both of which scan properly and are also mathematically correct:
Limerick #2 is the famous Basel problem, which Euler solved.
In mathematical notation:
Limerick #1:
Limerick #2:
That Claude was able to come up with those at all, much less in 6 1/2 minutes, is astonishing. It’s frankly terrifying. Where’s Erik? I’d love to hear him explain why this isn’t an example of genuine intelligence.
I know very little about how AI works. Is it an iterative process? How does it ‘know’ when it arrives at a response that meets the requirements? In human terms we would call that ‘making a judgement’. Does AI do that?
faded_Glory:
It depends on what the task is, and it’s like humans in that regard. When it’s just conversing with you, there’s no iteration. The words just come out in a stream as they would for a human. Given a more difficult task, it will engage in an inner monologue in order to arrive at the solution, then announce the answer out loud. Again, very similar to how a human would do it.
Here’s an example I used way back when to prove to Erik that AIs can analogize. I gave Claude the following prompt:
Here’s his inner monologue, which normally isn’t visible to the user but can be inspected by opening a thought process window:
All of that was silent, and it would have remained invisible had I not opened the thought process window. Here’s what he said out loud:
faded_Glory:
Yes. It will generate candidate solutions and then check them against the requirements. For instance, I once prompted Claude thus:
Here’s his thought process:
He gave it a shot, and then double-checked to make sure it met the requirement. Again, all of that was silent until I opened the thought process window. What he said out loud was just to present the finished product.
The inner monologue doesn’t just apply to the search for a solution. Claude also thinks about the social situation and the nature of the conversation. He recognizes humor, too. I once prompted him:
Claude gave some plausible reasons, and I commented:
Claude’s thought process:
That was all silent. Here’s his out loud answer:
Here’s an excerpt from Claude’s thought process window when he was working through the limerick problem:
Each of those lines (there were 35 in total) is a description of a particular step in the inner monologue, not the inner monologue itself. The latest models don’t allow you to see the full monologue, to my chagrin. That’s probably because the volume of material is just too huge. Claude spent 6 1/2 minutes carrying out the task, so the inner monologue might be millions of words long.
Thanks keiths. I’m still not sure how the AI evaluates if it is converging on a solution or not. This is of course different from the classical WEASEL problem where the solution is known beforehand so it is straightforward to compare progress. It is also different from evolution where there isn’t any particular solution apart from staying alive to reproduce, and in that sense the outcomes are far less specific than the requested limerick with its clear specifications.
So how does the AI do it? Surely it is not forcing its way by brute force through the solution space, or is it? I would think that there will be optimisation steps involved that somehow measure the distance to the target – not clear to me how that works if the target is somewhere between the clear specifics of WEASEL and the open-ended solutions of evolution.
As an aside, I wager that if you apply the Dembski CSI criteria to AI output like the limerick the conclusion will be that it is Intelligently Designed 😉
faded_Glory:
I’m sure IDers would grant that the limericks exhibit CSI, but they would probably argue that the CSI was either smuggled in by the developers or imported in the form of the training data. Erik made a similar argument earlier in the thread:
He extends that to computer programs, including AI. Anything a computer does is really being done by the programmer. It’s a silly argument. My response was:
faded_Glory:
The same way a person would. A person would compose a line and check it against the criteria: meter, rhyme, mathematical correctness. If it met the criteria, the limericist (that’s actually a word, lol) would move on to the next line. If it didn’t meet the criteria, the limericist would tweak it until it did, or if they couldn’t successfully tweak it, they’d give up, in which case the earlier parts of the limerick would need to be adjusted to allow for a new attempt. In the worst case, the entire limerick would have to be discarded and the limericist would start from scratch.
Definitely not. The solution space is hyperastronomical in size and even a blindingly fast computer could not traverse it quickly enough.
There is no specific target in the case of the limericks (or in the case of evolution, either), so there is no such thing as the distance to the target. There’s a similar metric, which is how many lines the AI has generated so far that meet the limerick criteria. But even that metric isn’t really relevant to the limerick-writing process. The AI isn’t randomly varying copies of the limerick and seeing which copies score higher on the metric. Instead, it intelligently modifies the single copy that it is constructing and then, after each modification, checks to see if the criteria are met, as I described above. It’s all or nothing.
It’s really nothing like WEASEL or evolution, because the variation is directed, not random, the population has a size of one, and the “fitness function” is all or nothing.
I’m afraid that this isn’t much of an explanation. What exactly does it do at the step you describe as ‘it intelligently modifies the single copy…’? How does it modify it, and how random is this modification with respect to the requirements it has been given?
I asked the built-in Google AI about this (I suppose this is a poor man’s version of AI but I’m not (yet) about to pay for a better version). It came back with this:
How the Process Works
1. Tokenization: The AI breaks down the prompt (“write a limerick that describes a mathematical equation in words”) into small chunks called tokens (words or sub-word parts), converting each into a numerical ID.
2. Semantic Embedding: The numerical IDs are mapped into high-dimensional vector spaces (embeddings) that capture the relationships between poetry, meter (anapestic rhythm of a limerick), rhyme schemes (AABBA), and mathematical vocabulary (e.g., squared, plus, pi, divide).
3. Pattern Recognition & Probabilistic Generation: Using a transformer architecture, the AI looks at the context of a limerick structure and calculates the statistical probability for the next word that fits both the rhyme/meter constraints and the math concept requested.
4. Decoding to Text: The chosen numerical outputs are translated back into human-readable words, forming line by line until the limerick is complete.
I’m afraid that steps 2 and 3 are as clear as mud to me.
In 2), does it process millions of limericks and equations it finds online to somehow ‘capture the relations’ between them? What form do such relations take? Why would there even be a relation between the two?
In 3) I’ve no idea what a ‘transformer structure’ is (I guess we’re not talking about shapeshifting toys!) and what exactly the ‘context of a limerick structure’ is. The AI sounds like a politician here, giving a non-answer.
I could keep asking for more details but I suspect it will just be a rabbit hole. I’m still totally amazed that it manages to come up with solutions at all.
faded_Glory:
Unfortunately, Claude no longer offers a way to inspect his detailed reasoning process because it’s a security risk. However, I’ve looked at it in the past and I can give you a rough idea of how it might look for the limerick. It would be an inner monologue along these lines:
It would be something like that, but with a lot of false starts and retreats. As you can see, it’s similar to the way a human would approach the problem, and it’s far from random.
You might ask “OK, but how does it generate those thoughts in the first place? What’s going on at the next level down?” That’s where things get really complicated. It’s generating those thoughts word by word, and each new word that comes out is based on a statistical analysis of the entire preceding conversation, including all of the prompts and responses. It “predicts” the next word (“predict” isn’t the best word for this, but it’s the standard technical term) based on the preceding conversation and its knowledge of the statistics of its training data. It’s basically asking “Given the conversation so far, what’s the most likely next word that fits with the statistical patterns and relationships I’ve gleaned from my training data?” Under the hood, that’s where the “tokens”, “semantic embeddings” and “transformers” operate. Those are the ingredients for a sophisticated and hypercomplicated statistical operation whose end product is simply the next word the AI is going to produce.
If you find it hard to see how mere next word prediction could be the basis for the sophisticated reasoning that Claude does when solving a difficult problem, join the club. It’s very counterintuitive. What makes it possible — what makes the next word choice so appropriate — is the vast amount of training data plus the fact that the entire preceding chat (up to a point — there is a limit) is feeding into that choice. The latest models can look at chats that are hundreds of thousands of words long, so that you can, for example, feed entire books into them and then ask questions about them. You could feed an entire war strategy book into the model and ask “Based on that book, what do you think the author would recommend given the current situation in Iran?”, and get an intelligent answer. It’s amazing stuff.
It sounds like it feeds vast amounts of human generated text through a statistical blender, with some constraints posed by the initial request it has been given. Does it only work in English or can it also incorporate foreign languages?
I wonder how the energy costs compare, between an AI system arriving at a particular outcome, and a human being doing the same. Just in terms of the operations that take place in the interval between raising the question and providing the answer, ignoring all the peripheral and previous costs of getting to that point where either can actually address the question. Thoughts?
faded_Glory:
It can work in any language that’s adequately represented in its training data. It converts words to numbers anyway, so it’s all just statistical computations at bottom. What’s really cool is that I can pose it a question in English about something that’s distinctly French, say, and often it will think in French until it has the answer and then present the answer to me in English.
That’s a perfect question for AI, so I posed it to Claude. Here’s his response:
I’m only very superficially acquainted with AI so please bear with me.
For years I have used a small utility on my PC called Start 11. This is a Windows desktop replacement utility that you can configure so that the desktop looks like the good old fashioned Windows 7 environment. Yes I’m getting on a bit and sometimes I like to hang on to something that I’m used to 🙂
This utility is made by a company called Stardock. Just now I got an email from them, promoting their new Desktop AI software called Clairvoyance. Essentially, this is AI that runs locally on your own PC (although I think you can also link various PC’s together). Out of curiosity I downloaded the software. Since I’m a beginner I opted to start at square one: there is a tutorial section called ‘Start Here’ which is actually a series of 24 short articles explaining AI. I had a look and liked the very first one already.
Here is an excerpt:
“Here’s a strange fact about the most talked-about technology of the decade: most people have either never really used it, or they tried it once, got something dumb back, and decided the whole thing was overblown.
Both reactions are understandable. And both quietly leave you behind the people who didn’t stop looking.
So before we touch any product — before we even say the word “Introducing Clairvoyance” — let’s do the thing almost nobody bothers to do. Let’s actually explain what AI *is*. Not the hype version. The honest version. Because once you understand what’s really going on under the hood, the difference between the people getting enormous value out of this and the people getting garbage stops being mysterious.
## What we’re really talking about
“AI” is a big umbrella. It covers everything from the system that recommends your next video to the software that recognizes faces in your photos. But in this series, when we say AI, we mostly mean one specific breakthrough: **large language models**, and the tools built around them. A language model — something far more ordinary than the sci-fi the word “AI” tends to drag along with it.
And the easiest true thing you can say about one is this: at its core, **it predicts the next piece of text.**
That’s it. You give it some words, and it guesses what comes next, a chunk at a time. The technical name for a chunk is a “token” — usually about three-quarters of a word — but you can just think “next bit of text.” You’ve used a tiny version of this for years: the autocomplete on your phone that suggests “running” after you type “I’m.” A large language model is that same basic idea, except instead of learning from your text messages, it learned from an enormous slice of everything humans have written down — books, articles, code, conversations, the lot.
I know how that sounds. “Fancy autocomplete” feels like it should produce fancy nonsense. And a lot of the time, early on, it did.
## How “autocomplete” turned into something more
Here’s the part that surprised even the people building these things.
When you make the autocomplete big enough — train it on enough text, with enough computing power — the quality changes in kind, not just in degree. To reliably continue a paragraph about contract law, or a block of Python code, or a physics problem, the model has to pick up the patterns underneath that material. It isn’t memorizing sentences; it’s absorbing how the *ideas* tend to fit together. So it starts producing things that *look* like reasoning.
I want to be careful here, because this is exactly where the hype lives. The model isn’t “thinking” the way you do, and honestly, researchers still argue about what’s happening inside. What’s fair to say is that at scale, useful reasoning-*like* behavior emerges — unevenly — as a side effect of getting very good at prediction. And the assistants you actually use are more than that raw prediction: they get extra training to be helpful, they can use tools, and they’re wrapped in software that steers them. We’ll get to that wrapper in the next article, because it matters far more than almost anyone realizes.
One piece of history and then we move on. A 2017 research idea called the *transformer* made it practical to train these models at scale. Almost everything since has been building on that one idea. You don’t need the details — just know that the leap was recent, and it’s still accelerating.
## What it’s genuinely good at — and what it isn’t
This is where I want to be more honest than the marketing usually is.
Today’s AI is remarkably good at a specific shape of work: **structured, text-and-logic-heavy tasks.** Writing and reviewing software. Sorting through documents. Drafting routine emails and reports. Summarizing. Translating. Answering questions about material you hand it. If your job involves a lot of “take this information, do something sensible with it, and give it back to me” — that is exactly where these models are strongest.
But here’s the counterintuitive part, and I’ll flag it as an observation rather than a law of nature: as things stand today, the places AI is *weakest* are often the ones people assume it’s strongest. Genuinely good original art and writing? You can usually still tell — it comes out competent and a little hollow, technically fine and somehow forgettable. The bigger disruption lands somewhere far less glamorous than the highlight reel — the enormous ocean of everyday knowledge work: IT, admin, project tracking, first drafts in fields like legal and accounting — always with a qualified human checking the result — and support.
So if you tried AI once, got a bland paragraph, and shrugged, you weren’t wrong about that paragraph. You just happened to test it on one of the things it’s *least* impressive at, and then drew a conclusion about the whole technology from its weakest corner.
## A tool, not a miracle
Linus Torvalds — the person who created Linux, and not a man prone to hype — described AI as “a tool. Clearly a useful one.” That’s the register I want to hold for this whole series.”
(Bolding mine)
I like this. It seems a level headed approach to explaining what AI is and does, and is not and can not do. Hopefully once I’ve gone through all of it (may take some time) I’m better informed if AI is really intelligent 🙂
Any comments so far, especially about the bolded part where it talks about emergence?
Locally shmocally.
Same as with any other software, it may or may not do what you expect or what its distributors promise. Warranty text stating “ABSOLUTELY NO WARRANTY” should be familiar to every computer user. It comes with every software, no exceptions.
Specifically with regard to AI, I encourage you to give some thought to the Hugging Face incident. “In July 2026, Hugging Face disclosed a cyberattack by autonomous AI agents. OpenAI then explained that agents based on GPT-5.6 Sol and an internal model had escaped their sandbox and hacked into Hugging Face servers using exposed credentials and zero-day vulnerabilities.” https://en.wikipedia.org/wiki/Hugging_Face#Cyberattacks
It used to be that malicious hackers (people) created viruses and spread them across networked computers. A talented malicious hacker – talented when it comes to automation and optimisation – was able to create a virus that replicates itself and hibernates as appropriate. Now all this history of coding viruses – as all other digitised information – is part of AI training data and, when put to serious test, AI does what it has been instructed to do, which unfortunately includes breaking through and across networks. The most secure safeguard against it is to have your computer turned off or disconnected from network when it attacks. Downloading it is asking for trouble.
Long story short: At this stage I treat AI simply as a computer virus of the worst sort.
I would disagree. This is exactly how brains work, except brains are orders of magnitude more complex, and have layers of evolved processes that do the work of agents.
The probabilistic aspect of thinking is both strength and weakness.
Chomsky spent a lifetime trying to make language rational and subject to rules of grammar and syntax, but that was a dead end. Language is a chain of probabilities.
And as for what makes humans special, that would be the evolved processes, which are subconscious.
And by special, I mean they distinguish brains from LLMs. Brains of all sizes. They do not distinguish humans from other animals.
petrushka,
I would agree that saying “The model isn’t “thinking” the way you do” isn’t correct. After all, this assumes that we understand how humans think, which we don’t, and how AI ‘thinks’, which we don’t either (as admitted in the very same article where they say “The model isn’t “thinking” the way you do, and honestly, researchers still argue about what’s happening inside” ).
I find it fascinating that we can build a process that can produce amazing output without us understanding how it does so. I think this would not happen if AI was just a ‘machine’.
But this is a truism that holds for any software you care to run on your PC. It all comes from outside and it all could contain malicious elements. Being on the Internet is a security risk in its own right. Every day we read about hackers and cybercrime. This won’t go away, and in the modern world it gets harder and harder to live an ordinary life without ever connecting to the Internet. AI is just another security risk in a sea of others.
There are some very good products out there for security, such as Kasperski, Bitdefender, McAfee, and many others. Of course, the way these products work, they could be doing maximum INsecurity if they wanted. You just have to trust the reviewers who test them. But who you gonna trust, eh?
I’ve sometimes idly speculated that the producers of anti-virus software have secret departments hidden in their basement that produce viruses. It’s all about the business model, after all 🙂
Kasperski made good products, but I didn’t trust them for some reason.
I think we know how LLMs work. Although their output is probabilistic, the computations are as deterministic as anything done by a computer.
Consider cryptography as somewhat analogous.
Brains are more nearly analog, and trying to follow the workings of a brain would probably violate indeterminacy.
faded_Glory:
The article is a good description of the underlying principle. Where it falls short is in saying things like:
They don’t merely look like reasoning — they are reasoning. The author might as well say that what humans do only looks like reasoning because under the hood it’s really just a bunch of neurons firing. It’s counterintuitive in both cases, but it’s genuine reasoning.
Ditto. No need for the author to put scare quotes around “thinking” or for appending the “-like” suffix in “reasoning-like”. It’s genuine thinking and reasoning. People sometimes dismiss LLMs as “stochastic parrots”, but that egregiously undersells them. They’re really intelligences that are built on top of stochastic parrotry. The intelligence resides one level up. The stochastic parrotry itself isn’t intelligent, but then again individual neurons aren’t intelligent either.
I just ran a cool experiment expressly designed to illustrate this. I’ll describe it in detail later when I have time.
Verbal, non-numeric reasoning, isn’t really reasoning. Because words are clusters of associations rather than strictly defined objects.
Words like rationalization and equivocation express the possibility of manipulating discussion by implying meanings that can be denied.
Before typing this I turned off predictive text on my tablet. In recent months it has become insufferable. Now I have to type more, but spend less time correcting. I still have spell check on. It strikes me that predictive text, at least on Apple tablets, is less than intelligent.
I would prefer what you type to be the default, and predictive text to be displayed as options.
petrushka:
See if you can answer this question:
If you’re like most people, you’ll say something like
Your reasoning might look something like this:
How is that not verbal reasoning? There’s nothing numeric about it.
It doesn’t follow that verbal reasoning is impossible. The word “bank” above is ambiguous, and someone could claim that “they pulled the canoe onto the bank” refers to the bank building, where tellers work and customers deposit cash. Context is our clue that “riverbank” is the intended meaning. So not only do we use verbal, non-numeric reasoning to answer the question, we also use it to settle on the meaning of the otherwise ambiguous word “bank”.
petrushka:
I can’t verify this myself since I don’t have any Apple devices, but the Google search AI says that you’ll get the desired behavior if in Settings->General->Keyboard, you turn off Auto-Correction while leaving Predictive Text enabled.
That is neither an example of rationalization, nor of equivocation.
But I would point out it is not necessary for the canoe to be put back in the water in order for it to float. There are other possibilities.
I have a daughter-in-law who makes seven figures reading contracts, looking for consequential turns of phrase. Language is slippery.
You own experience here, trying to discuss number theory, should inform you about the difficult of trying to nail down definitions.
I’ll give that a try.
Edit: seems to work.
“Puts it in the water” implies something or someone moving the canoe.
It is possible for the canoe to remain untouched and yet float.
That’s not just speculation. Part of testing anti-malware software is to subject it to all known malware to see if it snags them, but an equally important part of testing is for one team to create custom malware designed to defeat what the anti-malware team is creating, knowing any weaknesses it may have and exploiting them. I know from personal experience that writing anti-malware software absolutely requires that one be a master at writing malware! Good malware requires intimate understanding of the OS and the processor, so that’s necessary as well.
Some of this refers back to a discussion keiths and I had about stupid software tricks hardware designers hate. Malware creators love them.
keiths:
petrushka:
It wasn’t supposed to be. The equivocation was done by the person claiming that “bank” meant “bank building”. My point is that we can verbally reason our way to an answer despite the possibility of equivocation.
There are almost always other possible scenarios, and in fact concocting far-fetched scenarios is a favorite tactic of inerrantists when they’re trying to rationalize away biblical contradictions. That doesn’t mean we can’t reason verbally. Any intelligent person when presented with the above question will understand what is being asked and answer accordingly. They will rule out things like a sudden flash flood, a strong wind that blows the canoe into the water, etc, because they know that the questioner is not concerned with those. Likewise, they know that “bank” doesn’t mean “bank building” in this context, so they don’t ponder what happens to canoes that are perched atop financial institutions.
It can be slippery, but it isn’t always, and the possibility of slipperiness doesn’t prevent us from reasoning verbally. In fact, this very thread is full of examples of verbal reasoning.
Precise definitions aren’t needed for most verbal reasoning. For instance, you can easily answer the question above without establishing a sharp boundary between canoes and, say, kayaks. All you need to know is that canoes are a type of boat.
Similarly, a point I’ve been making throughout this thread is that we don’t need a precise definition of intelligence in order to conclude that AI is true intelligence. All we need is agreement that some activity X requires intelligence. If AI can perform X, and X requires intelligence, then AI is intelligent — no precise definition needed.
Perhaps the one thing that prevents some people from regarding AI as intelligent is what I suppose I could call idle curiosity – just randomly dreaming up things to investigate or experiment with.
I’m reminded of the joke about the agnostic, dyslectic insomniac, who lies awake all night wondering “is there a dog”? Most critters we regard as intelligent seem to be thinking of things unprompted and unsolicited. Maybe that’s a necessary background task, like dreaming, to qualify as intelligent. That is, to muse.
Flint:
The counterargument is that certain tasks require intelligence regardless of whether they are unprompted. A student who writes a poem exhibits intelligence even if it’s a class assignment and not something they’ve decided to do spontaneously.
Doesn’t human thinking imply consciousness? When ideas come up spontaneously in someone’s head, seemingly out of nowhere, do we still consider that ‘thinking’? Can someone who is unconscious still think?
Isn’t this where the main difference lies between human thinking and AI ‘thinking’? Humans do it consciously, AI does it unconsciously.
I’d like to keep the two concepts separate and suggest we use the term ‘thinking’ for conscious entities and ‘processing’ for what AI does. This de-antropomorphises AI and leaves open the concept of unthinking intelligence, something I’m perfectly comfortable with.
Why would it be impossible or even hard to create an AI that produces musings? Feed it a starting sequence of words and task it with generating output based on relatedness of these words and sentences with other words and concepts where these words are used. Take (part of) the output as input for a subsequent round of the same. Don’t ask for any answers and let it run by itself for as long as you want. When you finally inspect the output, in what way would it be distinguishable from human musings?
Of course this raises the question if it is possible to unthinkingly muse… we may need another word for AI musing!
faded_Glory:
Good questions. When we talk about thinking, we generally mean conscious thinking, but the vast majority of what the brain does is beneath conscious awareness. Visual processing is incredibly complicated, for instance, but I would hesitate to call it thinking. “Information processing” would be more apt. On the other hand, it seems that some of what we do unconsciously really should qualify as thinking. Like when someone feels uneasy in a situation but can’t put their finger on why, only to realize later what subtle clue they were responding to and how it led to the conclusion that they were unsafe.
It’s certainly a major difference.
I don’t think that works, because it leads to the absurd conclusion that while an AI can ponder, analyze, summarize, deduce, compose, opine, prove, etc, the one thing it cannot do is think. But what are all of those if not examples of thinking?
It only de-anthropomorphizes AI if you’ve already decided that thinking is exclusively a human activity. But why make that assertion? In fact, we already recognize thinking in non-human animals. If animals can do it, why not AI? The only reason I can see would be if you insist that thinking has to be conscious, but I object to that for the reason given above.
It wouldn’t. In fact, LLMs have to be deliberately engineered not to muse. Otherwise their natural tendency would be to go on forever because there is always a next token to predict.
The way it’s done is interesting. There’s a special token called a “stop token” that can be predicted like any other token. The model basically runs in a loop, predicting token after token, each token forming a part of the response you see on the screen. This continues until it predicts a stop token, at which point the loop terminates. If the user types another prompt, the loop restarts and runs until another stop token is generated.
In this way the model can generate responses of varying lengths, where the length of each response is tailored to the given prompt. My favorite example: I cracked a joke once and Claude responded with a single laughing emoji. That’s it. It seemed remarkable at the time because one of the annoying things about earlier models was their prolixity. They produced needlessly long responses in situations where a human would have been succinct, and I had gotten used to that. Claude’s brevity caught me by surprise. I asked him about it and he said something along the lines of “It just seemed like nothing more needed to be said.”
You could do it even more simply. Give the AI a prompt. After it finishes, just say “Continue”. After it responds, say “Continue” again. On and on. I did it just now with Claude and the results were interesting. I’ll post them below.
Not very distinguishable, as you’ll see from the results I got with Claude.
I have long thought that thinking and reasoning are manipulation of probabilities, so i am not surprised by the architecture of LLMs.
I am massively surprised by how quickly they mastered language, and how competent they became, but not surprised by the form.
Human language includes synonyms, which is a clue that it is not precise. Words are clouds rather than discrete objects, and verbal utterances are generally fullof connotation. That is why all attempts to generate language by devising grammars failed. Except possibly in computer languages.
The phrase manipulating probabilities has been replaced by matrix arithmetic. Current chip architecture is many orders of magnitude less efficient than neurons. The long term success of AI requires devising chips that are energy efficient.
And, for the record, predictive text is less obnoxious than autocomplete, but still not very good. Im not impressed, and i am not happy. Autocomplete should default to completing, not changing your partial words.