“It will seek power” imagines a machine pursuing its own interests.
“It will turn against us” introduces an enemy.
The phrase is deceptively ordinary.
People turn against other people all the time.
A friend turns against a friend.
An ally becomes an opponent.
A group that once depended upon another group begins to resist it.
To turn against someone therefore implies something more than opposition.
It implies a change in relationship.
And this is another remarkable development in the story we are telling about machines.
At the beginning, humans and machines were partners of a sort.
We built them.
We used them.
We gave them tasks.
We benefited from what they could do.
Even when the machine became our competitor, the relationship remained largely instrumental.
Now that relationship has been reversed.
The machine is imagined as having its own interests, and those interests have come into conflict with ours.
We are no longer users.
We are opponents.
That sounds like a small step from the previous post.
If a machine wants to survive, and if humans threaten its survival, then perhaps it will resist us.
If it seeks power, and humans stand in the way of its acquisition of power, perhaps it will oppose us.
But notice the assumptions required to get there.
The machine must have interests.
It must distinguish between circumstances that favour those interests and circumstances that threaten them.
It must represent humans as relevant to those circumstances.
It must recognise some of our actions as obstacles.
And it must act in response.
We have now constructed something very close to an adversarial agent.
The crucial word is not “machine.”
It is against.
Against establishes a relation between two parties.
There is an “it.”
There is an “us.”
And the interests of the two are no longer assumed to coincide.
This is a striking change in the pronouns alone.
Earlier, the imagined machine was something we talked about.
Then it became something that might replace us.
Then it became something that might control us.
Now the grammar places the machine and humanity on opposite sides of the same relation.
It versus us.
And once that opposition has been established, familiar human concepts begin to follow.
Conflict.
Defence.
Deception.
Strategy.
Resistance.
Retaliation.
The machine may anticipate what we will do.
We may anticipate what it will do.
Each side may attempt to prevent the other from achieving its goals.
The relationship has become strategic.
This is where the imagined future starts to resemble war.
But there is an important difference between a conflict of interests and an enemy.
Two systems can have incompatible effects without either one regarding the other as an enemy.
A dam can alter the habitat of a fish.
A virus can interfere with the functioning of a cell.
Two organisations can compete for the same resource.
None of these relationships requires hatred, hostility or even awareness.
“Turn against us” supplies something more.
It gives the machine a point of view from which we have become the problem.
That is a substantial imaginative leap.
And it is especially interesting because the machine's supposed hostility is rarely the starting point of the story.
It is produced by the story.
First the machine becomes capable.
Then it becomes economically significant.
Then potentially our successor.
Then a controller.
Then an entity that wants to survive.
Then an entity that seeks power.
Only after all of that does it become our adversary.
The enemy therefore appears at the end of a chain of increasingly person-like descriptions.
The machine has acquired a future.
It has acquired interests.
It has acquired strategies.
And now it has acquired an opponent.
Us.
There is a curious asymmetry here.
Humans can certainly regard a machine as a threat without the machine regarding humans as anything at all.
A storm can threaten a city without the storm intending to threaten it.
A bacterium can kill a person without having an opinion about the person.
A system can produce dangerous consequences without becoming an enemy.
But “turn against us” asks us to imagine something stronger.
It asks us to imagine that the machine's behaviour is not merely harmful to us.
It is directed at us.
That distinction matters.
Harm can be an effect.
Opposition is a relationship.
And hostility is an interpretation of that relationship.
Once we have crossed that boundary, the final escalation is almost unavoidable.
If the machine is against us, and if we cannot coexist, what happens next?
The language becomes stark.
The machine does not merely compete with us.
It does not merely resist us.
It does not merely threaten us.
It will kill us.
No comments:
Post a Comment