If “It’s just prediction” tells us that a language model is doing less than we might think, the next sentence usually arrives soon afterwards:
“It doesn’t understand.”
Perhaps.
But what, exactly, is being claimed?
The difficulty is that understand is one of those words that appears to explain something while concealing a remarkable amount of variation.
We say that a child understands a joke.
We say that a scientist understands a theory.
We say that someone understands how to repair a bicycle.
We say that a dog understands that it is time for a walk.
We say that a person understands another person.
And sometimes we say that a computer understands a command.
These cannot all mean precisely the same thing.
So when we say that a language model does not understand, we have already smuggled in a theory of what understanding is supposed to be.
Usually, the missing ingredient is something like meaning.
The model manipulates linguistic forms, we are told, but it does not possess the meanings those forms express. It has learned statistical relationships among words and tokens, but it has no genuine grasp of what they are about.
There is a real question here.
But there is also a curious asymmetry.
We are quite happy to infer understanding in other beings from what they do.
A child answers a question appropriately, modifies its behaviour when corrected, makes connections between things, uses a word in a new situation, and eventually demonstrates that it can do something with what it has learned.
We take such behaviour as evidence of an underlying capacity.
Yet when a language model does some remarkably similar things, the behaviour is frequently redescribed as evidence of something else: statistical pattern matching.
That may be the correct interpretation.
But notice what has happened.
The behaviour has not disappeared.
We have changed the vocabulary in which we describe its cause.
And the vocabulary matters.
If “understanding” means human conscious comprehension grounded in embodied experience, then we have a perfectly respectable reason to be cautious about applying the term to an LLM.
But if understanding means something more general — being able to discriminate relevant distinctions, preserve relationships across contexts, respond appropriately to what has been said, and use what has been learned in subsequent activity — then the question becomes much less obvious.
The disagreement is therefore not simply about machines.
It is about the category of understanding itself.
There is an even deeper problem.
Suppose we insist that genuine understanding requires some particular thing that language models lack: a body, sensory experience, biological needs, consciousness, intentionality, or a particular kind of causal connection to the world.
That may turn out to be right.
But then we need to explain why that thing is necessary.
Otherwise we have merely replaced:
“The machine doesn't understand.”
with:
“The machine doesn't understand because it isn't the kind of thing that understands.”
That is not yet an explanation. It is a boundary drawn around the word.
And boundaries can be useful.
But they should not be mistaken for discoveries about what lies on either side of them.
There is another possibility worth considering.
Perhaps understanding is not a single capacity at all.
Perhaps there are different kinds and degrees of understanding, emerging from different kinds of organisation.
A thermostat can respond to temperature without understanding weather.
A bacterium can discriminate chemical gradients without understanding chemistry.
An organism can learn from its environment without possessing a theory of learning.
A human can understand a sentence without being able to explain its grammar.
The existence of these distinctions does not make all these systems equivalent.
It makes the opposite point.
Different kinds of organisation can support different kinds of capacities.
So the interesting question about an LLM may not be:
“Does it understand?”
That question is too compressed.
We might instead ask:
What does the system have to be capable of doing before the word understanding becomes a useful description of its behaviour?
That question leaves open the possibility that the first answer is no.
It also leaves open the possibility that our existing concept of understanding is too closely tied to the particular way in which humans happen to understand.
Either way, simply saying “It doesn't understand” settles nothing.
It tells us what conclusion to draw only after we have already decided what understanding must be.
And that is the problem.
The sentence sounds like a description of the machine.
But perhaps it is really a description of our boundary around a word.
No comments:
Post a Comment