The Ideology Test

The afternoon sun filtered through the tall windows of the Institute’s reading room. Papers lay scattered across the table, alongside a laptop displaying a recently watched video.

Miss Elowen Stray leaned forward with interest.

“Apparently,” she said, “several language models were given political compass tests. Most ended up slightly left of centre.”

Mr Blottisham snorted.

“Well there you are then. Bias confirmed. The machines have gone woke.”

Professor Quillibrace, who had been stirring his tea with deliberate slowness, raised an eyebrow.

“My dear Blottisham,” he said mildly, “you appear to have attributed an ideology to an autocomplete system.”

Blottisham waved a hand impatiently.

“If it answers political questions and lands on the left, that means it’s left-leaning. Surely that’s obvious.”

Elowen tilted her head.

“But does answering a question imply holding a belief?”

Blottisham blinked.

“Of course it does.”

Quillibrace placed the spoon carefully beside the saucer.

“Let us test that proposition,” he said. “If I read aloud a line from a cookbook recommending cinnamon in apple pie, does that mean the book believes in cinnamon?”

“That’s different.”

“Indeed,” said Quillibrace. “The cookbook contains patterns of instruction. It does not possess culinary convictions.”

Blottisham frowned.

“Yes, but this is politics, not baking.”

“Quite so,” said Quillibrace. “But the structure of the inference remains identical.”

Elowen looked thoughtful.

“So when the model answers a political questionnaire…”

“…it generates a sentence,” Quillibrace finished gently.

“Yes.”

“And the researchers then treat that sentence as evidence of a belief?”

Quillibrace inclined his head.

“A belief held by an entity which, strictly speaking, does not exist.”

Blottisham folded his arms.

“Well something must be producing the answers.”

“Certainly,” said Quillibrace. “A statistical language system trained on an enormous corpus of human discourse.”

Elowen’s eyes lit slightly.

“So the result might reflect the distribution of discourse in the training data, rather than the ideology of the machine?”

“Precisely.”

Blottisham looked unconvinced.

“But why do they end up slightly left of centre?”

Quillibrace smiled faintly.

“My suspicion is that the tests are detecting the centre of gravity of institutional discourse.”

“Meaning?”

“Journalism, academic writing, policy documents, moderated public discussion.”

Blottisham shrugged.

“So the internet.”

“Not quite,” said Quillibrace. “The portion of the internet that is most heavily represented in curated training corpora.”

Elowen nodded slowly.

“So the compass is mapping the discursive average of those sources.”

“Just so.”

Blottisham leaned back in his chair.

“Well that’s disappointing. I was hoping the machines had opinions.”

“Oh, they do not,” said Quillibrace pleasantly. “But they are excellent mirrors.”

Elowen smiled.

“Mirrors of what?”

Quillibrace gestured lightly at the laptop.

“Of the linguistic patterns we have collectively produced.”

Blottisham thought for a moment.

Then his eyes brightened.

“I saw someone online suggest that maybe intelligence naturally leads to left-wing views.”

Elowen laughed softly.

“That seems like a dangerous claim.”

Quillibrace’s smile deepened.

“Indeed.”

“Why?”

“Because it forces a choice.”

Blottisham leaned forward.

“What choice?”

Quillibrace ticked the options off on his fingers.

“Either intelligence tends toward a particular ideology…”

“…or the test is not actually measuring ideology at all.”

Blottisham stared at him.

“That’s sneaky.”

Quillibrace lifted his teacup.

“Philosophy occasionally is.”

Elowen looked back at the laptop screen.

“So the real mistake is assuming that answers imply beliefs.”

“Exactly,” said Quillibrace.

“And if there is no belief…”

“…there is no ideology.”

Blottisham sighed.

“Which means the whole debate is based on a misunderstanding.”

Quillibrace took a sip of tea.

“My dear Blottisham,” he said gently, “a great many debates are.”