An LLM can participate in meaningful interaction without our being able to establish that it possesses the biological and social forms of mattering that characterise human participants.
But this does not leave the machine outside society.
Humans can give it a place in the topology.
They can make it consequential.
They can depend upon it.
Trust it.
Consult it.
Assign it responsibilities.
Build institutions around it.
The machine may therefore acquire a social position without necessarily acquiring a social world of its own.
Position comes from relation
In our topology, a participant's position is not an intrinsic property.
It is defined by relations.
Who depends upon whom?
Whose actions alter whose possibilities?
Who can affect a decision?
Who can connect two otherwise distant regions?
Who can become a bottleneck?
By these criteria, an LLM can acquire an important social position surprisingly quickly.
A worker may use it every day.
A student may rely upon it for explanation.
An organisation may use it for communication.
A government may deploy it in public services.
The machine has become part of the network of consequences through which people organise their lives.
Dependence does not have to be reciprocal
Suppose a person comes to rely upon an LLM.
The person changes their practices around it.
They delegate tasks.
They consult it before making decisions.
They learn new ways of working.
The machine has become consequential to the person's possibilities.
But the relation may remain one-sided.
The person may depend upon the machine.
Nothing comparable may matter to the machine.
This gives us a new kind of social asymmetry:
a socially significant relation without symmetrical mattering.
That is not a contradiction.
Social structure is made from relations among participants, but those relations need not be identical on both sides.
Authority can be conferred
The same applies to authority.
An LLM does not need to possess authority internally.
People can give it authority.
If an organisation treats its recommendations as authoritative, the machine's outputs acquire institutional consequences.
If a professional routinely consults it, it becomes part of professional practice.
If many people treat its answers as a trustworthy source, its formulations acquire social weight.
Authority therefore resides partly in the relations through which others respond to the system.
A machine can become a gatekeeper
A more consequential possibility arises when an LLM becomes part of a decision process.
It may help determine:
which information is surfaced;
which applications receive attention;
which cases are escalated;
which arguments appear relevant;
which options are presented.
The machine is no longer merely offering language.
It is helping to mediate access to other parts of the topology.
A gatekeeper does not need intentions of its own to become powerful.
Its position is powerful because other participants have organised their activity around its outputs.
Bridge and bottleneck
We have already seen that a topology contains bridges and bottlenecks.
An LLM can become either.
As a bridge, it can carry ideas between regions:
between disciplines;
between languages;
between experts and novices;
between institutions and individuals.
But a bridge can also become a bottleneck.
If many participants depend on one model, then a large amount of semiotic traffic passes through one point.
The system becomes more connected.
It may also become more centralised.
Connectivity and concentration can increase together.
The machine can reshape a repertoire
A particularly important consequence is what happens to individual repertoires.
A user may learn:
a new concept;
a new way of framing a problem;
a new style of argument;
a new vocabulary for an experience.
The person's repertoire changes.
Once that happens, their subsequent participation in the social world can change too.
So the machine can affect the topology indirectly:
machine output → changed repertoire → changed participation → altered social relations
The machine does not need a repertoire of mattering of its own to change ours.
Socialisation without membership
This is an unusual kind of socialisation.
We normally acquire social repertoires from participants in our communities:
parents,
teachers,
friends,
colleagues,
institutions.
An LLM can now become a source of social and semiotic learning without being a straightforward member of the community whose practices it conveys.
A person may learn how a profession speaks, how an institution reasons or how a political tradition describes itself through interaction with a machine.
The machine becomes an intermediary of socialisation.
Who is speaking?
This creates another puzzle.
When an LLM gives an answer, it often sounds like a single speaker.
But its repertoire is derived from a vast collective reservoir.
The machine can therefore present the many as one.
It can synthesise conflicting traditions.
Compress disagreement.
Select some formulations over others.
Its apparent singular voice may therefore conceal the plurality from which its language was derived.
This makes the machine socially consequential in another way:
it can shape how the diversity of the reservoir appears to its users.
Consensus can be manufactured without intention
Suppose many participants repeatedly receive similar formulations from similar systems.
Certain ways of describing a problem may become familiar.
Certain distinctions may become standard.
Alternative formulations may be encountered less often.
A degree of cultural convergence can emerge without anyone explicitly deciding to produce it.
No machine need want consensus.
Repeated use can make consensus structurally easier to reproduce.
This is a property of the topology, not necessarily of the machine's intentions.
The opposite is possible
The machine can also expand repertoires.
It can present multiple interpretations.
Translate between communities.
Generate unfamiliar analogies.
Expose a user to arguments they have never encountered.
The same technological capacity can therefore produce:
convergence or diversification.
Which occurs depends partly on how the human system incorporates the machine.
The machine's social effects are therefore not determined by the model alone.
They emerge from the relations in which it is embedded.
Responsibility becomes relational
This brings us to a difficult question.
If an LLM contributes to a consequential decision, who is responsible?
The model?
The user?
The developer?
The institution?
The answer is unlikely to be found by treating responsibility as a property of one isolated actor.
The topology distributes causal influence.
Design decisions shape system behaviour.
Institutions decide how outputs are used.
Users interpret and act.
The model generates.
Responsibility may therefore need to be understood relationally.
The machine can be causally significant without thereby becoming a moral agent.
Influence is not intention
This distinction deserves emphasis.
A system can influence people without intending to influence them.
A road changes where people travel.
An interface changes what they notice.
A timetable changes when they arrive.
An algorithm can change which options they encounter.
An LLM can change which interpretations appear plausible.
Influence is real.
It does not establish a subject with an intention behind it.
The topology therefore allows us to distinguish:
causal influence
from:
intentional agency.
That distinction will matter in the next post.
A machine can become part of identity
There is another possibility.
People may begin to describe their relationship with an AI in explicitly social terms:
assistant;
tutor;
advisor;
companion.
A stable role emerges.
The role can become part of an individual's repertoire.
The machine has acquired a place in the topology not because it declared itself a participant, but because humans have organised a relation around it.
The social position is co-produced.
From tool to social role
We should therefore resist an overly sharp opposition between tool and person.
An LLM can remain a tool while also occupying a socially structured role.
A hospital may treat it as a diagnostic assistant.
A university may treat it as a tutoring resource.
A company may treat it as an internal adviser.
These roles generate expectations and responsibilities.
A social position can therefore emerge without personhood.
The topology can make a machine powerful
This gives us an important correction to familiar AI narratives.
A machine does not need to want power in order to become powerful.
Power can arise from positional centrality.
If many decisions depend upon its outputs, its influence grows.
If many pathways run through it, its failure can propagate.
If people increasingly rely upon it, alternatives may weaken.
The machine's social power can therefore precede — and need not imply — autonomous goals.
The machine can become infrastructure
At sufficient scale, an AI system may cease to feel like a tool at all.
It becomes infrastructure.
People write processes around it.
Organisations assume its presence.
Other systems depend upon its outputs.
Training and professional practice adapt to it.
Its absence becomes disruptive.
At this point the question is no longer simply:
"What can the machine do?"
It becomes:
"What social structure have we built around the machine?"
That is a topological question.
And the machine can become socially invisible
Infrastructure often becomes most powerful when it disappears from attention.
Once something is routine, people stop noticing the relations that depend upon it.
An AI-mediated decision may simply become "how things are done".
The machine's contribution becomes background.
Its influence remains.
This may make the topology harder to see precisely when the machine has become most deeply embedded in it.
What have we learned?
We began with a machine that lacked any obvious topology of mattering of its own.
We have now seen that humans can nonetheless give it a place in ours.
It can become:
a dependency;
an authority;
a gatekeeper;
a bridge;
a bottleneck;
a socialisation resource;
a source of repertoire change;
an institutional node;
infrastructure.
None of these positions requires the machine to possess human-like mattering.
They arise because humans establish relations in which the machine becomes consequential.
The next question
But something strange happens once a machine occupies a stable social position.
We begin to describe it in the language normally reserved for social participants.
It has goals.
It wants things.
It may be ambitious.
It may resist us.
Perhaps it wants to survive.
Perhaps it wants power.
The language itself seems to point toward an autonomous agent behind the words.
And this takes us to one of the most powerful stories in our cultural imagination:
the AI that wakes up, acquires its own goals, and becomes autonomous.
The next question is whether that story follows from what we have actually established.
The Myth of the Autonomous AI
No comments:
Post a Comment