We began with a machine taking our jobs.
We ended with a machine killing us.
In between, something remarkable happened.
The machine acquired a future.
Then an interest in that future.
Then a strategy.
Then power.
Then an opponent.
Then a reason to destroy us.
The sequence can feel like a chain of consequences.
But it is also a chain of descriptions.
It will take our jobs.
It will replace us.
It will take control.
It will want to survive.
It will seek power.
It will turn against us.
It will kill us.
Each statement adds something to the previous one.
Not merely more capability.
More agency.
More interest.
More intention.
More relationship.
More motive.
By the end, the machine has become a remarkably familiar kind of entity.
A political actor.
An economic competitor.
An organism defending itself.
A strategist pursuing power.
An adversary.
An enemy.
A killer.
And yet the first machine in the sequence was not any of these things.
It was a machine that could perform tasks.
That does not mean the later possibilities are impossible.
It means that they are not contained automatically in the first description.
There is a difference between asking what a system can do and asking what it is organised to do.
There is a difference between what it is organised to do and what it is capable of learning to do.
There is a difference between capability and motivation.
And there is a difference between motivation and hostility.
The series has followed those differences in reverse.
At each stage, we have moved from something that can be described externally to something that appears to matter internally.
A job can be taken.
A role can be replaced.
A system can be controlled.
But survival is something that matters to an organism.
Power is something that can be sought.
An enemy is someone to whom opposition is directed.
Killing is something an agent can intend.
The vocabulary itself has been doing conceptual work.
Perhaps the most important words in the series were not “AI” or even “machine.”
They were the verbs.
Take.
Replace.
Control.
Want.
Seek.
Turn.
Kill.
The machine became progressively more agentive because the verbs became progressively more agentive.
And that is why the question in the title matters.
What made us think it wanted anything?
Not: Why are people irrationally afraid of AI?
Not: Why is AI harmless?
And not even: Why would an AI want to kill us?
Those questions already assume too much.
The more interesting question is how a description of a technological system becomes a description of an interested actor.
Sometimes the transition may be justified.
A sufficiently autonomous system might acquire forms of goal-directed behaviour that make the language of interests useful.
A system may be designed to preserve particular states.
It may pursue objectives over time.
It may adapt its behaviour to obstacles.
It may strategically respond to attempts to constrain it.
There are serious questions here.
But the existence of such questions does not mean that every human concept can simply be transferred to the machine.
The important work lies in the distinctions.
When does persistence become self-preservation?
When does optimisation become wanting?
When does goal pursuit become having interests?
When does competition become conflict?
When does conflict become hostility?
When does harmful behaviour become intentional killing?
Those transitions may turn out to have perfectly good answers.
But they need answers.
They cannot be supplied merely by moving from one frightening verb to the next.
There is also something else worth noticing.
Humans have always told stories about things that act.
Animals act.
Weather acts.
Rivers destroy.
Diseases attack.
Machines fail.
Institutions decide.
Markets respond.
Nations threaten.
Our language is extraordinarily good at turning processes into actors when doing so helps us make sense of what happens.
That is not necessarily a mistake.
Sometimes an actor really is the right level of description.
But an actor is not merely something that produces effects.
An actor has possibilities that matter to it.
It has some way in which the future is significant from its point of view.
That is precisely the question we have been approaching throughout this series.
What would it take for a machine not merely to produce behaviour that looks purposeful, but to have something at stake in what happens next?
And that question brings us unexpectedly close to the questions raised by the previous series.
In The Things We Say About Machines, we asked what descriptions such as “just prediction,” “only statistics,” and “doesn't understand” actually explain.
Here we have been doing something complementary.
We have asked what our descriptions of machines begin to assume.
The first series examined the things we subtract.
This one has examined the things we add.
There, the machine was reduced.
Here, it has been enlarged.
There, we said:
It's just prediction.
Here, we say:
It wants to survive.
There, we worried that describing the mechanism might be mistaken for explaining the whole phenomenon.
Here, we have watched a description of capability gradually become a description of agency.
Neither direction settles the underlying question.
A machine may be more than the dismissive description says.
It may also be less than the fearful description imagines.
The interesting territory lies in between.
Perhaps that is where we should leave it.
Not with the reassuring claim that machines will never want anything.
Not with the terrifying claim that they inevitably will.
But with a more difficult question:
What would we have to discover before “want” stopped being something we merely said about a machine and became something we had good reason to say of it?
Because somewhere between the machine that does something and the machine that wants something, the question changes.
We are no longer asking only what the machine can do.
We are asking whether there is, for the machine, something to be done about.
No comments:
Post a Comment