The History of an LLM — VI. When the Model's History Began to Feed Back

An LLM responds to a prompt.

But the response does not necessarily end the process.

Within a conversation, the response becomes part of the context from which the next response is generated. What the model has just produced can therefore alter the conditions under which it produces again.

The system's output has become part of its immediate history.

This creates a simple loop:

input → response → context → altered response → further context.

Nothing mysterious is required.

The parameters may remain unchanged. The model has not necessarily learned anything in the long-term sense.

But its current organisation has changed because the context has changed.

The model is now responding not simply to a new prompt, but to a new prompt in the presence of what has already happened.

This makes conversation different from a sequence of independent queries.

Each response can alter the possibility space of the next.

A question may establish a distinction.

An answer may introduce a concept.

That concept may become the condition for a later question.

The conversation develops a history because its own products become conditions for subsequent activity.

There is an important limit, however.

This history is not necessarily incorporated into the model's parameters. When the context disappears, much of the alteration disappears with it.

So we have another distinction:

history can alter the state of a system without altering its structure.

Yet even temporary organisation can have consequences.

For as long as the history remains active, it changes what can happen next.

The model's history is therefore not only something it carries.

Sometimes, within the conversation, it is something the model is continually making.

And once its own outputs become conditions for its future outputs, history has begun to feed back into the process that generates it.