I think a thread on this topic will be interesting. My own position is that AI is intelligent, and that’s for a very simple reason: it can do things that require intelligence. That sounds circular, and in one sense it is. In another sense it isn’t. It’s a way of saying that we don’t have to examine the internal workings of a system to decide that it’s intelligent. Behavior alone is sufficient to make that determination. Intelligence is as intelligence does.
You might ask how I can judge intelligence in a system if I haven’t defined what intelligence actually is. My answer is that we already judge intelligence in humans and animals without a precise definition, so why should it be any different for machines? There are lots of concepts for which we don’t have precise definitions, yet we’re able to discuss them coherently. They’re the “I know it when I see it” concepts. I regard intelligence as one of those. The boundaries might be fuzzy, but we’re able to confidently say that some activities require intelligence (inventing the calculus) and others don’t (breathing).
I know that some readers will disagree with my functionalist view of intelligence, and that’s good. It should make for an interesting discussion.
Cross-posting this from the “An AI loses it” thread since it’s relevant:
petrushka:
I ran across an article yesterday that quoted the cofounder of Hugging Face, Thomas Wolf, who shares your skepticism. He says
He’s referring to the fact that LLMs like ChatGPT are, at base, just glorified next word predictors. They don’t understand the meanings of the words they emit – they just choose them based on the statistical relationships they’ve observed among the trillions of words of text in their training datasets. They might as well just be emitting numbers or meaningless symbols. If they were sentient, they’d be bored out of their minds.
Any seeming intelligence arises from the interaction between them and their training data. It’s as if they’re channeling the collective wisdom of all the humans who have contributed to their training corpus.
Despite the mundane concept on which they’re based, they’re astonishingly capable (but also frustratingly stupid at times). I commented earlier:
I’m not as pessimistic as Wolf. I think he’s underestimating the creativity of LLMs. It’s true that everything they produce is implicit in their training data, but not in a mere copy-and-paste sense. Earlier I wrote that
ChatGPT has made me laugh out loud at some of its jokes (and cringe at others), and the jokes were context-dependent and specific enough that I can guarantee they never occurred anywhere in its training data. That counts as creativity in my book, and I don’t see any reason in principle why the same sort of creativity can’t lead to scientific discovery.
Kekulé famously dreamed of a snake swallowing its tail (or so the story goes), which triggered his realization that benzene was a ring of carbon and not a straight chain. That was a major breakthrough in organic chemistry, but it wasn’t a bolt from the blue. It was the combination of existing ideas: the ouroboros, which had been around since at least the time of the ancient Egyptians, and the idea of carbon chains. The combination was novel, but the concepts were not, and I’ve seen LLMs come up with similar novel combinations cobbled together from pre-existing ideas.
I was sloppy in that comment. I wrote:
I think it’s genuine intelligence, not merely “seeming intelligence”. Here’s the subtlety I failed to convey: The “predict the next word” function is mindless. It’s just a lot of statistical number crunching, and as far as that function is concerned, the words might as well be numbers or meaningless symbols. It doesn’t care. It’s just doing statistics.
But the AI isn’t just the “predict the next word” function by itself. That’s the principle upon which it operates, but it isn’t where the intelligence resides. If you build an LLM but don’t train it, it will still run, crunching away at the numbers, but nothing useful will come out. The intelligence resides in the system after it has been trained. It’s encoded in the synaptic weights and biases of the artificial neural network. The training process has added a layer (metaphorically speaking, not literally) to the next word predictor, and it’s the combination that’s intelligent, not the predictor function itself.
By analogy, human neurons are mindless little electrochemical machines. They aren’t intelligent. Their operation is just as mindless as the next word predictor of the AI. But if you hook them together in the right way, forming a brain, and allow them to learn over time, you’ll have something that’s intelligent. It’s intelligent despite being based on a mindless substrate. The same holds true of the AI.
Is AI intelligent?
As keiths admits, we do not have a good definition of “intelligent”, so we really cannot give factual answers. At best we can express our opinions.
In my opinion: no, AI systems are not intelligent. The LLMs are plagiarism devices. And yes, you can appear to be intelligent by using plagiarism, but the intelligence isn’t real.
I thought Cathy O’Neil had a good response to this at We should not describe LRM’s as “thinking”. She used the example of the Towers of Hanoi problem. Her comment was that the AI systems are not actually analyzing the problem. Rather, they are predicting what will appear in future blog posts about the problem.
Neil:
I said that we don’t have a precise definition. There are good definitions. After all, if you look in the dictionary, the entry for ‘intelligent’ isn’t blank, and the definitions provided make sense to us and correspond to how we use the word. However, there’s no definition (or set of definitions) that captures the concept so perfectly that for any given example, we can confidently classify it as either intelligent or not intelligent.
This isn’t unusual. If you gradually add water to a damp cloth, when precisely does it become wet? If you slowly shift the frequency of a light beam, at what point does it change from green to blue? If you pluck out your hairs one by one, is there a specific moment at which you suddenly become bald? We deal with this sort of fuzziness all the time. (I may do an OP on this later.)
Everyone is free to draw their own boundaries, but there is a point at which it becomes perverse. If someone claims that Newton and Leibniz weren’t intelligent, I think it’s safe to say that they’re wrong. Edge cases are debatable, and AI is one of them, but that doesn’t mean that it’s just a matter of opinion whether those guys were intelligent.
If that were true, then everything they generated could be found somewhere in their training data. It definitely can’t. To make that point, I gave Claude the following instructions:
His response:
That ain’t plagiarism. I guarantee that those instructions appear nowhere in Claude’s training data. In fact, I guarantee that they have never appeared anywhere at any time before Claude did his magic.
This is my favorite part:
That made me laugh out loud. I’m trying to imagine what burnt cardamom mixed with copper would smell like.
In response, you might argue that Claude isn’t doing anything original here. He’s just following a pattern he’s observed in technical documentation and mixing in some elements from elsewhere in his training data, plus novel nonsense names for the part. To which I say: yes, precisely! And that is exactly what a human would do in the same situation.
If you were a creative writing teacher and you gave that assignment to your students, and one of them turned in something identical to what Claude wrote, would you accuse them of plagiarism? I wouldn’t. I’d give them an A.
Those assembly instructions aren’t plagiarism. They’re highly original and creative, and I would argue that they are the product of an intelligent entity — Claude.
Neil:
They aren’t predicting the future. In fact, they aren’t really predicting at all, though that’s the word that is most commonly used to describe how they work. What they really are is statistical text synthesizers. Here’s how I think of it. LLMs are really asking themselves the following question:
They aren’t predicting the future. They’re “predicting” the next word in a retroactive counterfactual that fits with their training data.
(That’s a bit of an oversimplification. The LLM is really sampling from what it thinks is the most likely distribution, not predicting the most likely word. That’s because you don’t want it to produce identical output if you run it again on the same prompt.)
It’s generating a plausible candidate text that fits well with the rest of its training data. This is analogous to how AI image generation works (via diffusion models). There, the question is:
(This is oversimplified too, because I’m not taking the prompt into account. The principle remains the same, however.)
Just as the LLM is trying to generate text that fits well with the statistical patterns it observed in its training data, the diffusion model is trying to generate an image that fits well with the statistical patterns in its training data. The text and the image are novel. They aren’t plagiarisms of what’s in their training data, but they do share statistical characteristics with the training corpus.
It’s humans projecting their own activities on AI. Humans assume that the behaviour of AI qualifies close enough as analogizing, understanding etc. but it’s not.
In fact, rather few *people* have a proper understanding of what analogising even is in the first place. My degree is in linguistics and literature, so I happen to be in a better than average position to assess this. I am almost daily appalled by politicians, economists and physicists (and biologists too) using what they think are analogies and metaphors but are often actually false analogies, a mistake of logic that they fail to recognise. As to AI analogising, sorry, just no.
Does ChatGPT ever laugh or cringe at *your* jokes? (Or at its own jokes – because this is how AI would supposedly train its own sense of humour.) There is probably plenty of “Oh I see haha” type of material in its training data, so it should…