The Algorithm That Decided

The reading room of the Institute had acquired yet another device.

This one was smaller than the previous contraptions: a polished wooden box with a brass plaque on the front. A slot at the top accepted forms, and a narrow opening below dispensed printed slips.

Mr Blottisham stood beside it with the quiet pride of a man who had solved a problem once and for all.

Miss Elowen Stray inspected the box with interest.

“What does it do?” she asked.

Blottisham gestured toward the plaque.

Elowen leaned closer and read aloud.

THE DECISION ENGINE

She looked up.

“Impressive.”

At that moment Professor Quillibrace entered the room carrying his teacup.

He paused, regarding the box with mild curiosity.

“Ah,” he said. “Another addition to the Institute’s growing museum of epistemological machinery.”

Blottisham straightened.

“This one is particularly useful.”

“In what way?”

“It makes decisions.”

Quillibrace placed his cup on the table.

“How convenient.”

Blottisham held up a form.

“You feed in the relevant information, the algorithm processes it, and the machine decides.”

Elowen leaned forward.

“What sort of decisions?”

“Oh, all kinds. Loan approvals, job applications, insurance risk, that sort of thing.”

Quillibrace nodded thoughtfully.

“A device of considerable modern relevance.”

Blottisham slipped the form into the slot.

The machine whirred briefly.

A slip of paper emerged from the bottom.

Elowen read it.

DECISION: DENIED

Blottisham folded his arms.

“There you are.”

Quillibrace examined the slip.

“And who denied the request?”

Blottisham gestured at the box.

“The algorithm.”

“I see.”

Elowen looked curious.

“How does it decide?”

Blottisham shrugged.

“Mathematics.”

Quillibrace tilted his head.

“A formidable authority.”

Blottisham nodded confidently.

“The algorithm analyses the data and produces the correct decision.”

Quillibrace folded his hands.

“My dear Blottisham, may I ask a small question?”

Blottisham sighed.

“You’re about to tell me the machine doesn’t decide anything.”

“Not necessarily.”

“Go on then.”

Quillibrace gestured toward the box.

“Who designed the algorithm?”

Blottisham hesitated.

“Well… engineers.”

“And who chose which information appears on the form?”

“Policy designers.”

“And who determined what counts as a favourable or unfavourable outcome?”

Blottisham paused.

“I suppose… the organisation using the system.”

Quillibrace nodded.

“And who trained the model, if it happens to be a learning system?”

Blottisham rubbed the back of his neck.

“Data scientists.”

Elowen smiled thoughtfully.

“So before the machine even begins processing the form…”

“…many human decisions have already been made,” Quillibrace finished gently.

Blottisham frowned at the box.

“But the algorithm still makes the final decision.”

Quillibrace considered the phrasing.

“What exactly does the algorithm do?”

“It calculates the result.”

“According to what?”

“A set of rules.”

“And who wrote those rules?”

Blottisham sighed.

“People.”

Elowen spoke softly.

“So the algorithm isn’t deciding in the ordinary sense.”

“No,” said Quillibrace.

“It is executing procedures.”

Blottisham crossed his arms.

“But the outcome still comes from the machine.”

Quillibrace nodded.

“Certainly.”

Elowen tilted her head.

“But the procedures that produce the outcome were designed by humans.”

“Exactly.”

Blottisham stared at the slip of paper again.

“So when people say ‘the algorithm decided’…”

“…they are compressing a very large network of human choices into a single phrase,” Quillibrace said.

Elowen’s eyes lit slightly.

“And that makes the decision sound impersonal.”

Quillibrace lifted his teacup.

“A most perceptive observation.”

Blottisham looked suspiciously at the wooden box.

“So the machine isn’t really responsible.”

“No.”

“It’s just following instructions.”

“Precisely.”

Blottisham sighed.

“That’s much less mysterious.”

Quillibrace smiled faintly.

“Perhaps.”

Elowen looked thoughtfully at the brass plaque.

“So the interesting question isn’t what the algorithm decided…”

“No?”

“It’s who decided the rules that the algorithm follows.”

Quillibrace inclined his head.

“My dear Miss Stray,” he said, “you have identified the true location of the decision.”

Blottisham examined the device again.

“Well,” he said slowly, “perhaps the plaque should say something else.”

“Oh?”

Blottisham picked up a pencil and scribbled a note.

Then he read it aloud.

THE ALGORITHM THAT EXECUTED A SERIES OF HUMAN DECISIONS

Quillibrace nodded approvingly.

“Admirably accurate.”

Blottisham stared at the long phrase.

“Though perhaps not quite as catchy.”

Quillibrace sipped his tea.

“My dear Blottisham,” he said gently, “clarity rarely is.”