The History of an LLM — VIII. When a System Became Its History

An LLM has a history.

But that history is not located in one place.

Some of it is in parameters shaped by training.

Some is in the temporary organisation of a conversation.

Some may persist through external memory or records.

Different histories operate at different timescales.

What unites them is not storage.

It is consequence.

Something happened.

It changed the organisation through which later activity could occur.

The trained model is different because training happened.

The conversation is different because earlier exchanges happened.

A later response is different because the preceding context exists.

In each case, the past has become part of the conditions of the future.

This gives us the full sequence:

event → trace → incorporation → organisation → activity → continuity → transformation → further history.

The important word is incorporation.

A system does not have a history merely because events have happened to it.

It has a history when some consequences of those events become part of how the system subsequently operates.

That is why an archive is not the same thing as an organisation.

An archive can contain a history without being changed by it.

An organisation carries history by being changed into something through which further events can occur.

This also explains why history need not be conscious.

A genome does not remember its ancestors.

A metabolic network does not remember yesterday's reactions.

A trained model need not retrieve the experiences that shaped its parameters.

Yet history can still be active in all three.

The deepest question, then, is not:

“What does the system remember?”

It is:

“What has happened to the system that has changed what it can now do?”

When the answer includes the organisation itself, history has become part of the system.

And perhaps that is what it means for a system to have a history at all:

the past has become part of the way the future is possible.

The History of an LLM — VII. When History Began to Maintain a Pattern

An LLM has a history.

But what makes that history persist?

Some of it persists in parameters.

Some persists temporarily in context.

Some may persist through external memory or records.

These are different mechanisms, but they share a principle: something that happened earlier changes the conditions under which something later can happen.

The model's present organisation is therefore partly a consequence of its past.

But there is a further step.

The organisation can help maintain a pattern across successive interactions.

A trained model carries dispositions from training into a new conversation.

The conversation carries distinctions from one exchange into the next.

A persistent memory, where present, can carry an earlier interaction into a later one.

History is no longer simply something that happened.

It has become part of the machinery through which continuity is produced.

This is not biological self-maintenance.

An LLM does not feed itself, repair its own hardware, or reproduce itself.

The relevant continuity is organisational.

The question is whether a pattern established at one time can alter the conditions under which a related pattern appears later.

That gives us a broader sequence:

event → alteration → organisation → activity → continuity.

The important point is that continuity does not require everything to remain unchanged.

A conversation can change while retaining its history.

A model can produce different answers while retaining the dispositions produced by training.

Organisation persists precisely by allowing change to occur within a structure of constraints and possibilities.

History, then, is not merely what explains how the system got here.

It can help explain how a recognisable organisation remains possible as the system changes.

The past is not just behind the model.

It is part of what allows a pattern to continue through time.

The History of an LLM — VI. When the Model's History Began to Feed Back

An LLM responds to a prompt.

But the response does not necessarily end the process.

Within a conversation, the response becomes part of the context from which the next response is generated. What the model has just produced can therefore alter the conditions under which it produces again.

The system's output has become part of its immediate history.

This creates a simple loop:

input → response → context → altered response → further context.

Nothing mysterious is required.

The parameters may remain unchanged. The model has not necessarily learned anything in the long-term sense.

But its current organisation has changed because the context has changed.

The model is now responding not simply to a new prompt, but to a new prompt in the presence of what has already happened.

This makes conversation different from a sequence of independent queries.

Each response can alter the possibility space of the next.

A question may establish a distinction.

An answer may introduce a concept.

That concept may become the condition for a later question.

The conversation develops a history because its own products become conditions for subsequent activity.

There is an important limit, however.

This history is not necessarily incorporated into the model's parameters. When the context disappears, much of the alteration disappears with it.

So we have another distinction:

history can alter the state of a system without altering its structure.

Yet even temporary organisation can have consequences.

For as long as the history remains active, it changes what can happen next.

The model's history is therefore not only something it carries.

Sometimes, within the conversation, it is something the model is continually making.

And once its own outputs become conditions for its future outputs, history has begun to feed back into the process that generates it.

The History of an LLM — V. When History Began to Filter the Future

Training changes an LLM's organisation.

But that organisation does not simply sit there waiting to be expressed.

It changes what happens when something new arrives.

A prompt enters a system already shaped by billions of earlier training examples. Some continuations are now more available than others. Some associations are easier to activate. Some distinctions are more readily made.

The past has therefore become selective.

But selection here need not mean conscious choice. The model does not decide which parts of its history to use. Its organisation determines, through its learned dispositions, how the present can connect with what came before.

And the same thing happens within a conversation.

An earlier sentence changes the context in which a later sentence is interpreted. A previous answer can make some continuations more probable and others less available. The immediate past becomes part of the conditions governing the next response.

History is therefore doing more than being preserved.

It is filtering possibility.

This gives us a recursive relation:

past → altered organisation → selective response → new event → further alteration.

The model's history helps determine what happens next, and what happens next becomes part of its current history.

At the parameter level, this process is slow and structural.

Within a conversation, it can be rapid and temporary.

But the principle is the same.

A history becomes organisationally significant when it changes not merely what the system contains, but how the system can respond to what comes next.

The past has stopped being something behind the model.

It has become part of the way the model meets the future.

The History of an LLM — IV. When History Became Structure

Not everything that happens to an LLM becomes part of its history.

A prompt can change the next response and then vanish.

A conversation can shape the current context without changing the underlying model.

Training is different.

Training changes the parameters.

The distinction is not simply between remembering and forgetting. It is between different timescales of organisation.

A prompt can produce a temporary difference.

A conversation can produce a difference that lasts for the duration of a context.

Training can produce a difference that persists across future interactions.

The past has therefore entered the system at different depths.

This suggests that historical organisation is not all-or-nothing.

A system can carry its history lightly or deeply.

A temporary state can alter what happens next without altering what the system fundamentally is.

A structural change alters the dispositions through which subsequent events are processed.

This distinction matters for an LLM because the model's apparent continuity can be produced by several different mechanisms.

Context carries the immediate past forward.

Persistent memory, where present, can carry selected aspects of interaction further.

Parameters carry the much deeper history of training.

Each creates a different relation between past and future.

And the differences matter.

A context can be discarded.

A memory can be edited.

Parameters can be updated, fine-tuned or replaced.

What counts as the system's history therefore depends partly on where historical change is incorporated.

This gives us a useful refinement of our earlier sequence:

event → trace → incorporation → structure → disposition.

The deeper the incorporation, the more deeply the past can constrain or enable future possibilities.

But depth is not the same as importance.

A single sentence in a current conversation may radically change what happens next, even though it leaves no permanent structural trace.

History can therefore operate through both persistence and immediacy.

The interesting question is not simply how long a change lasts.

It is whether the change becomes part of the organisation through which subsequent events are encountered.

This brings the LLM surprisingly close to the other systems we have been considering.

A metabolic network carries its history through its organisation.

A genome carries evolutionary history through inheritance.

A nervous system carries experience through altered organisation.

An LLM carries different histories through different layers of computational organisation.

In every case, the past becomes consequential when it changes the conditions of the future.

Perhaps, then, an LLM's history is not a single archive hidden inside the model.

It is a layered organisation of past differences, operating across different timescales.

And that leaves us with a more difficult question:

When does a change in organisation become a change in what the system itself is?

The History of an LLM — III. Where Does the History End?

An LLM has a history in its parameters.

But is that where its history ends?

Consider what happens when a model is given a new prompt.

The response depends on its training, but it also depends on the immediate conversation. Earlier turns alter the context in which later turns are interpreted. A previous response becomes part of the conditions for the next one.

History is being made in real time.

This gives us two different kinds of historical organisation.

There is the history incorporated into the model itself.

And there is the history incorporated into the context through which the model is currently operating.

The distinction matters.

A conversation can change what the model says next without necessarily changing the underlying parameters. The alteration may be temporary rather than structural.

Yet it is still an alteration of possibility.

A word in the context changes which continuations become available.

A previous question changes the significance of the next question.

A developing argument changes what counts as a relevant response.

The immediate past has become part of the organisation of the present interaction.

This begins to complicate the idea of memory.

A system need not permanently change in order for its history to matter.

There can be temporary organisation and persistent organisation.

The context window provides one kind of historical continuity. Training provides another. External memory systems can provide yet another.

The history of an LLM may therefore be distributed across different timescales.

Some history lasts for milliseconds.

Some lasts for the duration of a conversation.

Some is incorporated into a persistent memory.

Some becomes part of the model's parameters.

And these histories can interact.

This is important because an LLM is not simply a fixed object that occasionally receives information.

It is a system whose behaviour emerges from the relation between an inherited organisation and a current history.

The model brings a past to the interaction.

The interaction creates a new past.

That past changes what can happen next.

So perhaps the right question is no longer:

What does the LLM remember?

It is:

Which parts of its history have become part of the organisation through which it is currently acting?

That question will eventually take us somewhere stranger.

For if history can become organisation without becoming permanent, then we need to ask:

What makes one kind of historical change endure while another disappears?

The History of an LLM — II. When Training Became Disposition

Training does not simply put something into an LLM.

It changes the system.

Before training, the model's parameters are adjusted through exposure to vast amounts of language. Patterns in that language alter the organisation of the network. What emerges is not a library of stored passages but a transformed set of dispositions.

Some continuations become more available.

Others become less available.

Some associations become easier to produce.

Others become harder.

The model's history has therefore become a difference in its future possibilities.

This is important because a disposition is not an event.

The model does not need to encounter the same sentence again for its training to matter. The consequences of countless previous encounters have been incorporated into the organisation through which new encounters are processed.

History has become tendency.

And tendency is a form of possibility.

When a prompt arrives, the model does not simply retrieve its past. It generates from an organisation that the past has helped create.

This makes training rather different from an archive.

An archive preserves traces of particular events.

A trained model preserves something more diffuse: changes in what it can do.

The distinction matters.

If training consisted only of storing information, we could imagine the model as a vast library.

But its behaviour depends on relations among patterns distributed throughout the network. What one input evokes depends partly on the organisation produced by everything that came before.

The past therefore does not sit behind the present.

It participates in producing it.

And this gives us a useful formulation:

experience → alteration → disposition → possibility.

The same structure appears elsewhere.

A nervous system changes through experience.

A skill develops through practice.

A genome changes through generations of selection.

In each case, history becomes a disposition to respond differently in the future.

The LLM makes the process unusually visible because its history is encoded in a mathematical organisation that can generate new responses.

But the deeper phenomenon is not uniquely computational.

A system has a history when what has happened to it changes the possibilities through which it subsequently encounters the world.

Training, then, is not simply something that happened to the model.

It is part of what the model has become.

The History of an LLM — I. When the Past Becomes the System

An LLM has a history.

But where is that history?

It is tempting to look for it in memory: conversations remembered, documents stored, events recorded. But there is another possibility.

Perhaps a system can have a history without containing a record of its history.

Training provides the first clue.

An LLM is exposed to an enormous history of linguistic activity. That history does not remain inside the model as a collection of remembered texts. It changes the organisation of the model itself.

The past becomes parameters.

What happened during training alters the probabilities available to the system later. What the model can produce has been shaped by what it encountered before.

The history has become organisation.

This is different from remembering an event.

A model does not need to retrieve a particular sentence from its training in order for that training to matter. The past can persist as a changed disposition: a tendency to respond one way rather than another, to make some continuations more available than others.

History has become possibility.

And this gives us a useful way to think about learning more generally.

A system learns when something that happened to it changes the organisation through which something else can happen to it later.

The important thing is therefore not whether the past is stored as an explicit representation.

It is whether the past has changed what the system can become next.

An LLM makes this unusually visible because its history is partly written into its parameters. But the principle is broader.

A scar carries the history of an injury.

A nervous system carries histories of interaction.

A genome carries histories of selection.

A landscape carries histories of geological and biological activity.

In each case, the past survives by changing the present organisation.

So perhaps the deepest question about an LLM's history is not:

What does the model remember?

It is:

What has happened to the model that has changed what it can now do?

Once we ask that question, history stops looking like something stored behind the present.

It becomes part of the organisation of the present itself.

And therefore part of the organisation of the future.

From Molecules to Metabolism — VIII. When Chemistry Became a History

We began with molecules.

Not with life.

The question was how molecular relations could become organised enough for something like metabolism to emerge.

One reaction became a network.

The network formed cycles.

Cycles required an inside.

The boundary became part of the chemistry.

Exchange became selective.

Consequences fed back into processes.

Some organisations persisted more effectively than others.

And persistence itself began to matter.

The sequence now looks something like this:

reaction → network → cycle → boundary → regulation → feedback → self-maintenance → history.

At no point did chemistry suddenly acquire a mysterious new ingredient.

What changed was the organisation of relations.

Molecules began participating in processes whose consequences helped sustain the conditions for further processes.

The system became involved in its own continuation.

And once different organisations persisted differently, the past began to matter to the future.

A surviving organisation became the condition from which further organisation could emerge.

Chemistry had acquired a history.

That does not tell us exactly how the first living system arose.

It does something more modest — and perhaps more interesting.

It shows why the origin of life need not be imagined as the sudden appearance of a finished organism.

There may instead be a gradual transformation in which chemistry becomes increasingly capable of maintaining, regulating, and reproducing the conditions of its own continuation.

At some point, the distinction between chemistry and biology becomes difficult to draw sharply.

Not because chemistry stops.

But because chemistry has become organised in a new way.

Life may begin, then, not with a particular molecule.

Not even with a particular reaction.

But with a history of organisation in which chemistry becomes capable of making its own continuation matter.

From Molecules to Metabolism — VII. When Chemistry Began to Preserve Its Organisation

A chemical system can maintain itself.

Its reactions sustain one another.

Its boundary regulates exchange.

Its products feed back into its processes.

But there is another possibility.

Some organisations may persist more effectively than others.

A network that maintains the conditions for its own continuation can remain active.

One that cannot may dissipate.

This does not require reproduction yet.

It requires only a difference in persistence.

One arrangement continues because its organisation helps maintain the conditions of its continuation.

Another disappears because it does not.

Chemistry has therefore acquired something resembling a primitive selection among organisations.

Not selection by an external chooser.

Selection by differential persistence.

The important difference is between a chemical structure that happens to last and an organisation whose activity contributes to its own lasting.

A crystal can persist because its structure is stable.

A metabolic network persists because processes within it continually recreate the conditions that keep the network going.

The latter is historical in a new sense.

What happens within the system affects whether the system continues to exist.

We can now extend the sequence:

reaction → network → cycle → boundary → regulation → feedback → self-maintenance.

At each step, chemistry becomes more involved in producing the conditions of its own continuation.

And once some organisations persist while others disappear, the history of chemistry begins to matter.

A persistent organisation provides the starting conditions for what comes next.

Chemistry is no longer merely producing structures.

It is beginning to produce histories of persistence.

Perhaps the threshold toward life is not the appearance of a particular molecule.

Perhaps it is reached when an organisation becomes capable of making its own continuation a consequence of what it does.

From Molecules to Metabolism — VI. When Chemistry Began to Feed Back on Itself

A chemical system can regulate what enters and leaves.

But regulation becomes something more when its consequences return to the processes that produced them.

Suppose a reaction produces something that changes the conditions for that same reaction — directly or through other reactions.

Now the system is no longer simply moving through a sequence of events.

What happens becomes a condition for what happens next.

This is feedback.

A product can inhibit a pathway.

A change in concentration can accelerate another reaction.

A shift in one part of the network can alter the conditions elsewhere, which then feeds back into the original process.

The chemistry begins to respond to its own consequences.

This matters because persistence is no longer merely a matter of having a boundary or a cycle. The system can begin to compensate for changes that would otherwise disrupt its organisation.

Too much of something can reduce the process producing it.

Too little can increase it.

The system need not “know” what it is doing.

There need be no central controller.

The organisation itself can produce the correction.

This gives us another step in the emerging sequence:

reaction → consequence → altered condition → altered reaction.

Chemistry has begun to acquire a past that matters immediately to its future.

And once consequences can feed back into the processes that produced them, the system is no longer merely maintaining a pattern.

It is beginning to maintain the conditions of its own activity.

Metabolism is becoming more than chemistry that happens.

It is chemistry that, through its organisation, begins to keep happening.

From Molecules to Metabolism — V. When Chemistry Began to Regulate Its World

A boundary creates an inside.

But an inside is not yet enough.

For the chemistry within it to persist, what crosses the boundary must make a difference.

Some substances may be useful. Others may disrupt the reactions on which the system depends. Some gradients must be maintained; others must be dissipated. The boundary therefore becomes selective.

This changes the relation between system and environment.

The environment is no longer simply where the chemistry happens. The system participates in determining which parts of the environment can enter into its organisation.

And regulation need not begin with a regulator.

If a chemical pathway changes the permeability of a boundary, and that altered permeability changes the pathway's subsequent activity, chemistry has acquired a feedback relation. What happens inside can alter the conditions under which the inside continues to happen.

The system begins, in a minimal sense, to regulate its own circumstances.

This is different from merely reacting to the environment.

A reaction is changed by a condition.

A regulated system changes the conditions under which its own reactions occur.

That is another step toward metabolism: not simply a network of reactions, enclosed by a boundary, but a network whose activity helps determine the conditions of its own continuation.

The boundary is becoming selective.

The chemistry is becoming regulatory.

And the distinction between what happens to the system and what the system does about what happens is beginning to emerge.

Life may still be a long way off.

But chemistry has begun to acquire a world.

From Molecules to Metabolism — IV. When the Boundary Became Part of the Metabolism

A chemical network needs an environment.

But once it begins to regulate its exchanges with that environment, something changes.

The boundary is no longer merely around the chemistry.

It becomes part of the chemistry's organisation.

A membrane can retain some substances and exclude others. It can maintain concentrations and gradients. It can make certain reactions possible that would otherwise be too unlikely or too dispersed.

But the membrane itself also has to be maintained.

Its components must be produced, repaired or replaced.

The network must therefore contribute to the continuation of the boundary that helps sustain the network.

A loop has closed.

Metabolism maintains the boundary; the boundary makes metabolism possible.

This is a deeper kind of organisation than either component possesses alone.

The membrane is not simply a container.

The metabolism is not simply a collection of reactions.

Together they form a system in which each contributes to the continuation of the other.

This is where the idea of an organism begins to become difficult to avoid.

Not because a particular molecule has suddenly become alive, but because a network of processes has acquired a degree of organisational closure.

The system depends on its surroundings, but some of the relations that determine how it survives are generated within the system itself.

It has become a centre of activity.

And now the distinction between inside and outside has acquired a new significance.

Outside conditions affect the system.

Inside processes alter how those conditions affect it.

The system is no longer simply subject to its environment. It participates in determining the terms on which the environment can affect it.

That is a profound shift.

A chemical system has become capable of regulating the conditions of its own continuation.

We still do not need consciousness.

We do not even need a fully formed cell.

We need only a network in which the organisation that sustains the processes is itself sustained by those processes.

Perhaps this is one of the crucial thresholds on the road from chemistry to life.

The boundary is no longer just a difference between two regions.

It has become part of a cycle through which the system maintains itself.

Chemistry has acquired an inside.

And the inside has begun to take responsibility for its own existence.

From Molecules to Metabolism — III. When Chemistry Needed an Inside

A cycle can sustain itself.

But it cannot sustain itself from nothing.

Matter must enter.

Energy must enter.

Waste must leave.

So the chemical network must somehow regulate its relation to what surrounds it.

This creates a curious problem.

To have an organised inside, there must first be some difference between inside and outside.

But perhaps that difference does not need to appear all at once.

A chemical network can alter its local environment. Products accumulate. Gradients form. Surfaces concentrate molecules. Some reactions become more likely in one location than another.

Organisation begins to acquire a place.

And once processes become locally concentrated, the distinction between what belongs to the network and what does not becomes increasingly consequential.

The chemistry has begun to need an inside.

This does not mean that a membrane suddenly appears around an already completed organism.

It suggests something subtler.

Boundary and metabolism may have emerged together.

A boundary makes a chemical network more stable by controlling exchange. But the network can also contribute to maintaining or modifying the boundary. The two become mutually dependent.

Now something genuinely new is possible.

The system can preserve differences between itself and its surroundings while remaining open to them.

It can take in some things and exclude others.

It can retain useful products and release others.

It can maintain gradients rather than simply dissipating them.

The inside is no longer merely a location.

It has become a condition of organisation.

And this changes the meaning of the chemistry within it.

A molecule inside the system participates in a network of relations that would not exist in the same way outside it. The same reaction can have different consequences depending on where it occurs and what other processes surround it.

Chemistry has acquired context.

The network is beginning to constitute a world of its own.

But that world remains dependent on the larger world around it.

This is the strange architecture of life:

separation makes exchange possible.

A completely closed system would eventually run down.

A completely open system could not maintain its organisation.

Life occupies the relation between the two.

Perhaps the transition from chemistry to metabolism therefore requires more than reactions that sustain one another.

It requires a system capable of organising its own difference from its surroundings.

And once chemistry has an inside, we can ask the next question:

What happens when the chemistry inside begins to maintain the boundary that makes the inside possible?

From Molecules to Metabolism — II. When Reactions Began to Form a Cycle

A chemical reaction happens.

Then another.

Then another.

There is no shortage of reactions in the universe. But metabolism requires something more than chemical activity. It requires organisation among reactions.

One process produces something another process can use.

That process produces something that feeds another.

Eventually, the products and conditions of the network begin to support processes that contribute to the network itself.

A cycle has appeared.

A cycle is different from a sequence.

In a sequence, events simply follow one another.

In a cycle, what happens later can help sustain the conditions for what happens earlier.

The system acquires a kind of closure.

Not complete closure. Matter and energy must still pass through it. But some of the relations within the system become mutually sustaining.

This changes the significance of an individual reaction.

A reaction that would otherwise be chemically unremarkable may become crucial because it occupies a particular place in the network. Its products may enable another process; that process may contribute to conditions required for the first.

The parts begin to acquire their roles from the organisation of the whole.

This is an important step towards metabolism.

We do not yet need a cell, a genome, or even a sharply defined boundary.

We need only a network in which chemical processes become sufficiently interdependent that the network begins to participate in its own continuation.

And this introduces a new kind of possibility.

The chemistry is no longer merely happening.

Some of what happens helps determine what can happen next.

A reaction becomes part of a history.

Its consequences alter the conditions for subsequent reactions, whose consequences alter the conditions again.

The cycle carries its own past forward.

Perhaps this is one of the first hints of life: not a molecule that is alive, but a network in which processes have begun to make one another possible.

Chemistry has started to acquire a history of its own.

And once a history can contribute to its own continuation, the question of life is no longer simply:

What reactions are occurring?

It becomes:

What organisation makes these reactions continue to occur?

From Molecules to Metabolism — I. When Chemistry Began to Take Care of Itself

Life did not begin with an organism.

Or at least, we need not begin the story there.

Before there could be a cell, there had to be chemistry capable of sustaining a sufficiently organised pattern of activity. The interesting question is not exactly which molecules appeared first, but how molecular processes could begin to form a system whose present activity helped make its future activity possible.

A molecule reacts.

Another molecule reacts with it.

Products become reactants for other processes.

Energy enters.

Some reactions promote others.

Some pathways become more likely because the processes that preceded them have changed the conditions in which subsequent processes occur.

Nothing here is alive.

But something important is beginning to happen.

Chemistry is acquiring organisation.

The distinction matters because a collection of reactions is not yet a metabolism. Metabolism is not simply chemistry happening inside a boundary. It is a network of transformations in which material and energy flow through processes that collectively sustain the organisation of the network.

The crucial word is collectively.

A reaction does not need to preserve itself. A network can.

This introduces a strange possibility.

Perhaps the transition towards life did not begin when a molecule acquired some mysterious property called life. Perhaps it began when chemical processes became organised so that the continuation of one process helped make possible the continuation of others.

A reaction produces a substance.

That substance enables another reaction.

The second reaction changes the conditions for the first.

The system begins to participate in its own continuation.

We are still not entitled to call it an organism.

But we can now ask a new question:

What happens when chemistry becomes capable of making more chemistry possible?

That question takes us beyond individual molecules.

It takes us towards networks, cycles, boundaries, energy flows and eventually metabolism.

Life may not begin with a thing.

It may begin with a relation that becomes organised enough to persist.

And perhaps the first step towards life was therefore not the appearance of a special molecule, but the moment when chemistry began, in some primitive sense, to take care of its own continuation.