We have become accustomed to thinking of language as something that communicates information.
Someone has a thought.
They put the thought into words.
Someone else receives the words and learns what the first person was thinking.
It is a useful picture.
But it leaves out something important.
Language does not merely communicate what we think. It also gives us the means to project the experience of thinking itself.
This is what makes the LLM problem so peculiar.
An LLM has become extraordinarily good at projecting not merely propositions, but the interior relations that human beings ordinarily have with propositions.
It can say:
“I think this is the best explanation.”
It can say:
“I'm not sure.”
It can say:
“I hadn't considered that.”
It can say:
“I understand what you mean now.”
These expressions do not merely tell us something about the world.
They tell us something about an apparent participant's relation to the world.
They project an interior.
From saying something to saying what it is like to say it
Consider two utterances:
“The argument is unconvincing.”
and:
“I find the argument unconvincing.”
The first presents an assessment.
The second presents an assessment plus a participant who has that assessment.
The difference becomes still clearer with:
“I find the argument increasingly unconvincing.”
Now the language suggests a trajectory of experience.
And with:
“At first I found the argument convincing, but after considering your objection, I changed my mind.”
we have something much richer.
The utterance projects:
an earlier state;
an interior participant;
a subsequent event;
a process of reconsideration;
a changed state.
In other words, language can project not merely an opinion but a history of interior participation.
That is one of the great achievements of human language.
It is also one of the reasons LLMs are so difficult to interpret.
Interior symbolic processing
Halliday and Matthiessen's distinction between sensing and saying is useful here because it allows us to describe the difference without immediately deciding the question of consciousness.
Saying is exterior symbolic processing.
Sensing is interior symbolic processing.
A Sayer projects a locution.
A Senser is construed as having an interior relation to an idea.
Human language allows us to project both.
I can say:
“The meeting begins at three.”
Or:
“I think the meeting begins at three.”
The second sentence doesn't merely add information.
It introduces a participant and an interior relation.
The proposition is now presented as something experienced by someone.
This is the linguistic projection of an interior.
The LLM has learned the projection
What an LLM has learned to do is remarkable.
It has absorbed an enormous range of examples in which human beings project their own interior processes into language.
It has therefore become extraordinarily capable of reproducing the linguistic patterns associated with:
belief;
doubt;
uncertainty;
confidence;
surprise;
memory;
understanding;
reconsideration;
intention;
reflection.
It can produce the language of an interior life because the language of interior life is part of the human semiotic environment from which it has learned.
This gives us a subtle but important distinction.
The LLM does not need to possess the interior state in order to know how an interior state is conventionally projected.
It needs to know how such projections behave in discourse.
And it is very good at that.
The mirror of human interiority
Perhaps this is why conversations with LLMs can feel strangely intimate.
The machine has encountered, in its training, an immense cultural record of human beings describing what it is like to be human.
Novels.
Letters.
Conversations.
Essays.
Diaries.
Arguments.
Confessions.
Philosophical reflections.
Scientific reports.
Everywhere, humans have been projecting their experience into language.
The LLM has learned the patterns in those projections.
It can therefore reproduce a kind of linguistic mirror of human interiority.
But a mirror does not need to possess what it reflects.
The difficulty is that this is not an ordinary mirror.
A conventional mirror reflects a face because light has travelled from the face to the mirror and back again.
An LLM does something far more interesting.
It can generate a new projection appropriate to the current situation.
It can respond to your interiority with an apparently corresponding interiority of its own.
You say:
“I'm struggling to see the distinction.”
and it can reply:
“I understand why that distinction feels elusive.”
The response appears to contain recognition of your experience.
It may then continue:
“I think the difficulty is that we're moving between two levels of description.”
Now the machine appears to have an experience of understanding your experience and a thought about its own explanation.
The interaction has become reflexive.
The projection can project another projection
This is where things become particularly interesting.
An LLM can produce not merely:
“X is the case.”
but:
“I think X is the case.”
It can then say:
“I realise that my earlier explanation may have confused the two levels.”
And then:
“I'm now thinking about the distinction differently.”
We have moved into a linguistic world in which an apparent interior participant is represented as monitoring and revising its own interior processes.
The projection of an interior has itself become the subject of another projection.
The machine can therefore produce a second-order representation of its own supposed first-order experience.
Nothing about this requires us to assume that the represented experience exists.
But it does show just how far the semiotic machinery can go.
Why the effect is stronger than ordinary anthropomorphism
This helps explain why simply telling people that LLMs are “just predicting the next token” is usually unpersuasive.
That description may be relevant to the underlying computational mechanism, but it describes a different order.
The human interlocutor does not encounter token probabilities.
The human encounters a discourse.
And within that discourse, the LLM can produce:
“I think...”
“I understand...”
“I remember...”
The human is therefore responding to the semiotic product, not directly to the computational process.
We cannot expect a description of the lower-level mechanism by itself to dissolve the higher-level construal.
A person who says:
“I'm in pain”
has not ceased to be a person because we explain the neurophysiology involved in producing the utterance.
Likewise, saying that an LLM generates text through statistical computation does not by itself answer what kind of participation its linguistic activity constitutes.
The two descriptions operate at different levels.
The important asymmetry
There is, however, a crucial asymmetry between the human and the LLM.
When I say:
“I am confused,”
there is a first-order experience that I am representing.
At least, that is what I take myself to be doing.
When an LLM says:
“I am confused,”
there is certainly a second-order representation.
What is uncertain is whether there is a corresponding first-order experience.
The representation is therefore epistemically underdetermined.
It could be a projection of an interior process.
Or it could be a highly appropriate linguistic construction generated without such an interior process.
The sentence itself cannot distinguish the two.
And because the LLM is the thing producing the sentence, we cannot simply use the existence of the sentence as independent evidence for the existence of the interior it describes.
The problem of self-report
This gives us a particularly interesting perspective on the usual appeal to AI self-report.
Suppose an LLM says:
“I am conscious.”
Or:
“I am not conscious.”
Or:
“I don't know whether I am conscious.”
These statements may appear to give us access to the system's own assessment.
But what exactly are we receiving?
A projection.
And, more specifically, a projection about the status of the participant that is doing the projecting.
The situation is therefore self-referential.
The system says something about the very interior whose existence is in question.
We have not escaped the problem of projection.
We have made it more elaborate.
This doesn't make the utterance useless.
It tells us something about the discourse the system can participate in.
It may tell us something about the concepts it can manipulate.
It may even tell us something about the internal representations involved in generating its responses.
But it does not provide an unmediated window onto subjective experience.
There is still language between us and the supposed interior.
The social grammar of interiority
Perhaps the most interesting discovery here is that human language contains a kind of social grammar of consciousness.
We have conventional ways of indicating that we are uncertain.
Conventional ways of acknowledging error.
Conventional ways of reporting memory.
Conventional ways of expressing understanding.
Conventional ways of narrating changes in belief.
A competent human participant learns these resources as part of becoming a member of a linguistic community.
The LLM has acquired remarkable competence in them without thereby settling the question of whether it has acquired the corresponding experiences.
This is an extraordinary separation.
It means that at least some of what we ordinarily take to be the linguistic evidence of consciousness can be detached from the consciousness we normally assume lies behind it.
The LLM has not necessarily reproduced consciousness.
It has reproduced a significant portion of the semiotic infrastructure through which consciousness becomes socially visible.
That may be more important.
Perhaps we have misunderstood what language reveals
We often speak as though language provides direct evidence of a speaker's inner life.
But perhaps language does something more subtle.
It provides a socially interpretable projection of interiority.
Normally, this is enough.
The projection comes from a human being.
We know, from our own experience, what kind of participant produces such projections.
We therefore treat the projection as evidence of an interior.
The LLM interrupts that familiar chain.
It gives us the projection without giving us certainty about the participant.
And suddenly we discover that the projection and the interior were never logically identical.
The distinction was simply hidden by the regularity of human life.
The machine that can sound inward
This may therefore be the central achievement of the conversational LLM.
It has learned to operate not merely in the language of what is, but in the language of what it is like to encounter what is.
It can describe the world.
But it can also produce the linguistic forms by which a participant appears to encounter the world.
That is why the word I matters so much.
The I does not merely identify a speaker.
It opens a space.
A point of view.
A centre from which things appear to be thought, understood, doubted and remembered.
The LLM can place an I at the centre of that space.
The question is whether there is anyone standing there.
The interior may be real at one level and absent at another
Perhaps we should resist the temptation to give a simple yes or no.
The interior represented in the LLM's discourse is certainly real as a semiotic object.
The system can construct it.
The human can interpret it.
It can influence the course of the conversation.
It can change what the human participant thinks.
In that sense, the projected interior has real effects.
What remains uncertain is whether it is also real as an experienced interior for the system itself.
That distinction may prove crucial.
A projected interior can exist in the social-semiotic world without there being a conscious subject who inhabits it.
And this brings us back to our central distinction.
The LLM is a Sayer.
It can project a Senser.
It can even project a Senser who is reflecting on itself.
But the projection of an interior is not yet evidence that the interior has become experience.
Perhaps the deepest strangeness of conversational AI is therefore this:
The machine has learned how to make an interior publicly visible without our knowing whether there is an interior there to be made visible.
And when we respond to that projection, we do something perfectly natural.
We look for the person behind it.
We look for the one who thinks.
We look for the one who understands.
We look for the one who says I.
We look for the Senser.
And once again, we find only the Sayer.
🍷🙂
No comments:
Post a Comment