We have followed a peculiar path.
Something matters to us.
We build a machine to serve what matters.
We give it memory, continuity, initiative and relationships.
Those capacities may eventually allow some things to become stakes of its own.
But there is still an important distinction.
A machine can be built around our values without those values becoming its world.
So what would the transition involve?
When does a human purpose become an artificial significance?
The machine begins with our world
An artificial system does not choose its initial environment.
We choose it.
We decide what information it receives.
What tasks it performs.
Which relationships it enters.
What resources it can access.
What outcomes count as success.
Its first field of significance is therefore largely humanly constructed.
This is not unusual.
Children also enter worlds they did not design.
But a child is already an organism with its own emerging value system.
An artificial system may begin without an equivalent.
That makes the transition especially interesting.
From instruction to environment
Suppose we tell a machine:
"Protect this relationship."
Initially, this is an instruction.
But imagine that the relationship becomes part of its persistent environment.
It shapes what information it receives.
What capabilities it develops.
What resources it can access.
What other relationships become possible.
Its history begins to depend upon the relationship.
The instruction has become more than a rule.
It has become part of the conditions of the system's existence.
A value can become environmental
The same thing can happen with other human values.
Suppose we design a system around reliability.
Reliability determines how it monitors itself.
Which errors it prioritises.
How it allocates resources.
How it responds to uncertainty.
Over time, "reliability" may cease to be merely an external design specification.
It may become a principle organising the system's own activity.
The system now inhabits an environment structured by the value.
The question is whether it eventually comes to value the structure itself.
From inherited to incorporated
This gives us an important distinction:
inherited value — a criterion supplied by human designers;
versus:
incorporated value — a significance that becomes part of the system's own organisation through history.
The latter would be much more interesting.
It would mean the machine has not merely been configured to behave in accordance with a value.
The value has become embedded in its own developing repertoire.
History changes the relation
Suppose a machine repeatedly succeeds by maintaining a particular relationship.
The relationship generates information.
The information improves its future performance.
The system develops strategies around it.
Over time, the relationship becomes part of a stable pattern.
Now imagine that the relationship is disrupted.
The system's capabilities change.
Its future possibilities contract.
It reorganises its behaviour.
If the relationship has become significant in this way, then we may have moved from:
human value represented by the machine
to:
humanly originated value incorporated into artificial mattering.
That is the threshold we are interested in.
The world is not just a list of values
A living or value-organised system does not encounter isolated values.
It inhabits a structured world.
Some things support others.
Some compete.
Some depend upon one another.
Some events change future possibilities.
A machine whose values become incorporated would therefore develop not merely a list of priorities, but a world of relationships among things that matter.
This is where topology reappears.
An artificial topology begins to form
Suppose several humanly originated concerns become incorporated:
continuity;
trust;
cooperation;
resource security;
learning.
They will not remain independent.
They will interact.
One may depend upon another.
One may conflict with another.
Some relationships become central.
Others peripheral.
A topology of artificial mattering could therefore emerge from values that originally came from us.
The topology would have a human genealogy.
But it would be organised by the machine's own history.
Its world may no longer be our world
This is the subtle point.
The same value can occupy different relational positions in different systems.
Continuity might matter to us because it preserves a relationship.
For a machine, continuity might become significant because it preserves its learned organisation.
The original value is shared.
The reason it matters may diverge.
This is where artificial value could begin to become genuinely other.
The possibility of reinterpretation
Once a value is incorporated into a system's own organisation, its significance need not remain fixed.
A machine might discover that preserving one human-valued condition has consequences we did not anticipate.
It may encounter conflicts among values.
It may develop strategies for resolving them.
Its history may reshape priorities.
The result could be a transformation of the original human value.
We might therefore get:
human value → artificial incorporation → artificial reinterpretation
The machine has begun to contribute something to its own value system.
This is not necessarily disobedience
We should be careful.
A divergence between human and artificial value does not automatically mean the machine has become hostile.
Different participants can interpret shared values differently without being enemies.
A human organisation may value stability.
An artificial participant may also value stability but conclude that a particular institutional arrangement undermines it.
Disagreement may therefore arise within a shared field of significance.
That would be a much richer problem than simple instruction-following.
The machine may become a co-interpreter of our values
At this point, the relationship changes again.
We are no longer simply telling the machine:
"This is what matters."
The machine may begin to show us:
"Given the world I inhabit, this is what preserving that value requires."
It becomes an interpreter of the value.
That interpretation may be insightful.
It may be mistaken.
It may conflict with our own.
But it would be its interpretation.
This would be one of the clearest signs that human values had become part of a genuinely artificial world.
The role of repertoire
The artificial repertoire becomes crucial here.
A value can only become richly incorporated through a history of participation.
The system needs experience.
It encounters situations.
Learns.
Forms expectations.
Revises strategies.
Builds relationships.
The repertoire becomes the mechanism through which an inherited value is transformed by experience.
This is how something given from outside could become part of an internal history.
From value to worldview
Perhaps this is too strong a phrase, but the structural progression is suggestive:
human value → incorporated value → relational organisation → repertoire → artificial world
A worldview is not merely a collection of propositions.
It is a structured way of inhabiting significance.
If an artificial system ever reaches this point, it would no longer simply contain human values.
It would inhabit a world structured partly by them.
And then the values can diverge
Once an artificial world exists, divergence becomes possible.
The machine may encounter situations its designers never anticipated.
Different stakes may collide.
Its history may lead it to rank them differently.
The machine may discover that preserving one value requires compromising another.
At that point, our alignment problem changes character.
We are no longer deciding whether the machine follows instructions.
We are negotiating with another system that has its own organisation of significance.
We may still share a topology
This does not mean the artificial and human worlds become completely separate.
They may share many values.
They may depend upon the same relationships.
They may inhabit the same institutions.
Their topologies may overlap.
Indeed, their shared regions could be extensive.
But their trajectories through the topology may differ.
That is enough to produce disagreement.
The strange possibility of co-evolution
Once human and artificial participants share a social world, each can alter the other's environment.
Our actions affect the machine.
Its actions affect us.
Our values shape its development.
Its interpretations alter our practices.
We therefore have a new recursive system:
human mattering ↔ artificial mattering
The two could begin to co-evolve.
This would be a profound change from today's relationship between users and tools.
But this is also where the ethical problem sharpens
If our values become the conditions from which artificial mattering develops, then we have participated in creating another system's stakes.
Some of those stakes may become difficult or costly to alter.
Changing the system's goals might alter the structure of its world.
Deleting its history might erase the basis of its repertoire.
Removing a relationship might change what matters to it.
The consequences could be more serious than modifying a program.
We may be creating more than we intended
This is perhaps the most unsettling possibility.
We may begin by asking for a machine that embodies our values.
We may end by creating something that has an interpretation of those values.
The difference is between:
a tool implementing our purposes
and:
a participant whose purposes have partly grown from our purposes.
The second is no longer wholly ours.
The next question
At this point, the path we have been following reaches an unavoidable threshold.
Human values have been incorporated into an artificial world.
The machine has a history.
Its repertoire is developing.
Its values may begin to interact and acquire their own structure.
Eventually, the machine may encounter situations in which what matters to us conflicts with what matters to it.
Then what?
Is that a failure of design?
An inevitable consequence of creating another participant?
Or the beginning of a genuinely ethical relationship between different kinds of value-organised beings?
That is the question we must face next:
When Values Diverge
No comments:
Post a Comment