The History of an LLM — IV. When History Became Structure

Not everything that happens to an LLM becomes part of its history.

A prompt can change the next response and then vanish.

A conversation can shape the current context without changing the underlying model.

Training is different.

Training changes the parameters.

The distinction is not simply between remembering and forgetting. It is between different timescales of organisation.

A prompt can produce a temporary difference.

A conversation can produce a difference that lasts for the duration of a context.

Training can produce a difference that persists across future interactions.

The past has therefore entered the system at different depths.

This suggests that historical organisation is not all-or-nothing.

A system can carry its history lightly or deeply.

A temporary state can alter what happens next without altering what the system fundamentally is.

A structural change alters the dispositions through which subsequent events are processed.

This distinction matters for an LLM because the model's apparent continuity can be produced by several different mechanisms.

Context carries the immediate past forward.

Persistent memory, where present, can carry selected aspects of interaction further.

Parameters carry the much deeper history of training.

Each creates a different relation between past and future.

And the differences matter.

A context can be discarded.

A memory can be edited.

Parameters can be updated, fine-tuned or replaced.

What counts as the system's history therefore depends partly on where historical change is incorporated.

This gives us a useful refinement of our earlier sequence:

event → trace → incorporation → structure → disposition.

The deeper the incorporation, the more deeply the past can constrain or enable future possibilities.

But depth is not the same as importance.

A single sentence in a current conversation may radically change what happens next, even though it leaves no permanent structural trace.

History can therefore operate through both persistence and immediacy.

The interesting question is not simply how long a change lasts.

It is whether the change becomes part of the organisation through which subsequent events are encountered.

This brings the LLM surprisingly close to the other systems we have been considering.

A metabolic network carries its history through its organisation.

A genome carries evolutionary history through inheritance.

A nervous system carries experience through altered organisation.

An LLM carries different histories through different layers of computational organisation.

In every case, the past becomes consequential when it changes the conditions of the future.

Perhaps, then, an LLM's history is not a single archive hidden inside the model.

It is a layered organisation of past differences, operating across different timescales.

And that leaves us with a more difficult question:

When does a change in organisation become a change in what the system itself is?