In discussions of artificial intelligence, “hallucination” is usually treated as a technical defect: a system invents information that is not true. The natural question that follows is therefore how such errors can be eliminated. Yet this framing already assumes something misleading about how large language models work. It assumes that they are designed primarily to retrieve facts from the world.
In reality, large language models are designed to do something quite different. They generate coherent stretches of language.
Seen from a semiotic perspective, hallucination is therefore not simply a malfunction. It is what happens when a meaning-producing system continues to construct a coherent text despite lacking a stable referential anchor. The system does not retrieve knowledge in the way a database does; it completes patterns of meaning.
A small example illustrates the mechanism. Suppose one asks an AI system about “Wuthering Heist.” If the system fails to recognise the phrase as the title of an episode of Inside No. 9, it may instead interpret the phrase as a distorted reference to Wuthering Heights and proceed to produce a perfectly coherent explanation of the novel. Nothing in the generated text may appear internally inconsistent. The hallucination arises not from a breakdown of coherence, but from the system stabilising the text around the nearest available semantic pattern.
In other words, the error does not lie in the text itself. The text may be entirely coherent. The problem lies in the relation between the text and the situation it is supposed to describe. Hallucination occurs when coherence outruns reference.
To understand why this happens, it is useful to recall a foundational insight from systemic functional linguistics. Language can be understood as a meaning potential: a structured set of possibilities from which particular meanings can be actualised in specific contexts. Speakers and writers do not retrieve sentences from storage. Rather, they select and combine options from this potential in order to produce a meaningful text suited to the situation at hand.
Large language models approximate such a meaning potential in computational form. During training, the system is exposed to vast quantities of text and learns statistical patterns linking words, phrases, and larger semantic configurations. When prompted, the model does not search for a stored answer. Instead, it generates a continuation that is probabilistically consistent with the patterns it has learned.
Under ordinary circumstances this process works remarkably well. The prompt provides enough contextual constraint for the system to converge on an interpretation that aligns with the intended situation. But when those constraints are weak or ambiguous, the system still faces the same imperative: it must produce a coherent continuation of the text. In the absence of clear contextual grounding, it therefore selects the most probable semantic trajectory available within its learned meaning potential.
The result may be a text that is internally coherent yet externally misaligned with the situation. The system has produced a plausible construal, but not the one intended by the user.
From this perspective, hallucination is not a mysterious failure of intelligence. It is a predictable consequence of how generative language systems operate. The model is designed to maintain coherence and to follow high-probability patterns of meaning. When contextual information is insufficient to anchor the interpretation, probability fills the gap.
Humans, it is worth noting, behave in similar ways. In conversation we routinely interpret unfamiliar or ambiguous expressions by assimilating them to patterns we already know. Mishearings, mistaken references, and confident but incorrect interpretations are common features of everyday communication. The difference is that human interlocutors usually have access to richer contextual cues that allow misunderstandings to be corrected quickly.
What generative AI makes visible, in an unusually stark form, is a general property of meaning-making systems. Coherence and reference are not the same thing. A text can be perfectly coherent while still failing to correspond to the situation it purports to describe.
Seen in this light, the term “hallucination” may even be somewhat misleading. Nothing magical or irrational is occurring. The system is doing exactly what it was designed to do: it is producing a coherent construal of meaning from the patterns available to it.
The real question, then, is not why such systems sometimes hallucinate. The real question is why we expected a system designed to generate meaning to behave like a system designed to retrieve facts.
Understanding this distinction opens the door to a deeper analysis. If hallucination arises from the interaction between a probabilistic meaning potential and the need to produce a specific instance of text, then the phenomenon can be examined through one of the central concepts of systemic functional linguistics: the cline of instantiation.
That is where the next post will begin.
No comments:
Post a Comment