We have now reached a claim that sounds considerably more substantial:
“It has no experience.”
Unlike just, only and really, there is no obviously dismissive little word here.
It sounds like a straightforward fact.
A language model has no childhood.
It has never been hungry.
It has never been cold.
It has never fallen in love.
It has never stubbed a toe.
It has never stood beneath a thunderstorm and watched the sky change.
Whatever else a language model may do, it has not lived a human life.
That seems important.
But then we encounter a deceptively simple question:
What is experience?
If by experience we mean conscious subjective experience — there being something it is like to be the system — then the claim becomes closely related to the problem of consciousness.
And that is precisely where we need to be careful.
We do not have direct access to anyone else's experience.
We infer it.
We infer that another human being experiences pain because they behave in ways associated with pain, because they have bodies and nervous systems like ours, because they tell us they are in pain, and because we understand something about the biological processes involved.
With other animals, the inference becomes more difficult but does not disappear.
We observe behaviour, physiology, learning, communication and environmental interaction.
We construct an account of what kind of experience those systems might have.
The question of artificial systems is therefore not settled merely by observing that their experience would be unlike ours.
Of course it would be unlike ours.
If there were artificial experience, we should expect it to belong to the organisation of an artificial system.
But there is a second meaning of experience that complicates the matter.
We also use experience to mean what a system has been through.
A person has experience of teaching because they have taught.
A doctor has experience of treating patients.
A machine-learning system has training experience in the sense that its parameters have been changed by exposure to data.
The two meanings are obviously not equivalent.
Training on text is not the same thing as living a life.
But neither is the second meaning irrelevant.
A language model does have a history.
Its present state is the result of what has happened to it during training and, in some systems, during subsequent interaction.
The system at one point in training is not numerically identical in organisation to the system after billions of parameter updates.
Its history has changed what it can subsequently do.
That is not subjective experience.
But it is a form of historical organisation.
And this distinction matters because some of the strongest claims about machine experience quietly slide between the two meanings.
We begin with:
“It has no lived experience.”
which is plainly true in the ordinary human sense.
We then move to:
“It therefore has no experience.”
which is a much stronger claim.
And finally:
“Therefore nothing that happens to the system matters to it.”
That last step does not follow at all.
Something can matter to a system without being consciously experienced.
A thermostat can be affected by temperature.
An immune system can distinguish one molecular pattern from another.
A plant can respond differently depending on its history of light exposure.
A nervous system can be altered by learning.
In each case, the system's subsequent possibilities depend upon what has happened to it.
The system has, in that sense, a history that becomes part of its organisation.
That is particularly relevant to artificial neural networks.
Training is not simply the delivery of information to a passive container.
Training changes the system.
The resulting organisation then determines how the system responds to subsequent inputs.
Its history becomes, in part, a resource for its future.
But this still leaves the deepest question untouched.
A system can have a history without having an experience of its history.
A rock has a history.
A glacier has a history.
A genome has a history.
A language has a history.
None of those facts tells us that there is something it is like to be the thing whose history we are describing.
So perhaps we need to distinguish three questions.
Has the system been affected by what has happened to it?
For a trained model, obviously yes.
Does what has happened to it alter what it can subsequently do?
Again, yes.
Is there something it is like for the system to undergo those changes?
That is a very different question.
And that third question is the one that matters if by experience we mean subjective experience.
The first two do not establish the third.
But neither does the third become impossible simply because the first two can be described mechanically.
That would be another version of the argument we have encountered throughout this series:
The system can be described in terms of its mechanism.
Therefore nothing more interesting is happening.
But description and exclusion are not the same thing.
Perhaps current language models have no subjective experience.
Perhaps some future artificial systems will.
Perhaps subjective experience requires forms of embodiment, self-maintenance, affect, agency or ongoing interaction with an environment that current LLMs lack.
Those are serious possibilities.
They are also hypotheses to be investigated rather than conclusions supplied by the sentence “It has no experience.”
The most interesting part of the problem may therefore be the distinction between having a history and experiencing a history.
An LLM certainly has the first.
Whether it has the second is precisely what we do not know how to determine.
And that leaves us with a rather uncomfortable reformulation.
Instead of asking:
“Does the machine have experience?”
we might first ask:
What would a system have to be like for its history to become something it could experience?
That question takes us beyond the text it produces.
It takes us into organisation, embodiment, persistence, memory, environment and the possibility of a point of view.
And perhaps that is where the real question begins.
No comments:
Post a Comment