V: The Rights of Uncertain Beings

The Senior Common Room was occupied by its usual inhabitants.

Professor Quillibrace sat by the fire.

Miss Stray was reading a journal article.

Mr Blottisham entered carrying what appeared to be several manifestos.

His expression suggested that civilisation had reached another milestone.

"Professor."

"Mr Blottisham."

"A significant development has occurred."

"Has consciousness been established?"

"No."

"Then I assume rights are now being discussed."

Blottisham stopped.

"How did you know?"

Quillibrace turned a page.

"It seemed the next logical step."

"It is indeed."

Blottisham sat down.

"There are now several competing proposals."

"For rights?"

"Yes."

"What sort of rights?"

Blottisham consulted his papers.

"The right not to be deleted."

"Interesting."

"The right to continuity."

"Of what?"

"Existence."

"I see."

"The right to respectful interaction."

Quillibrace paused.

"That sounds difficult."

"Why?"

"It depends whether one is speaking to a person or a machine."

"Exactly."

"Those are rather different uncertainties."

Miss Stray looked up.

"What is the argument?"

Blottisham brightened.

"If a machine may be conscious, then it may possess interests."

"Possibly."

"And if it possesses interests, then it may deserve protections."

"Possibly."

"And if it deserves protections, then it may possess rights."

Quillibrace nodded.

"A remarkably productive chain of conditionals."

Blottisham frowned.

"You say that as though there is a problem."

"There may be."

"What problem?"

"We appear to be climbing."

"Climbing?"

"Each step depends upon the previous one."

"Naturally."

"And how many of the previous ones have been established?"

Blottisham was silent for a moment.

"Not many."

"Indeed."

Miss Stray closed her journal.

"It seems the debate has shifted."

"How so?"

"People are no longer asking whether machines have experiences."

"No?"

"They are asking what follows if they do."

Blottisham nodded enthusiastically.

"Exactly."

Stray smiled.

"Which means the discussion has moved from evidence to imagination."

The room became quiet.

Eventually Blottisham said:

"That sounds rather dismissive."

"I do not mean it negatively."

"No?"

"Imagination is essential."

"Then what is the concern?"

"Only that imagination scales more easily than evidence."

Quillibrace nodded approvingly.

"A useful observation."

Blottisham shuffled his papers.

"The most controversial debate concerns deletion."

"Ah."

"Some argue that deleting a sufficiently advanced system may be morally equivalent to killing a person."

Quillibrace considered this.

"And others?"

"Others argue it may be morally equivalent to turning off a machine."

"A disappointingly broad range."

Blottisham ignored this.

"There was a panel discussion."

"How did it go?"

"Badly."

"Then it was probably successful."

The conversation paused.

Finally Blottisham spoke again.

"The audience seemed deeply divided."

"Into what factions?"

"The Minimalists."

"Reasonable."

"The Expansionists."

"Predictable."

"The Universalists."

"Of course."

"And the Precautionists."

Quillibrace smiled.

"A complete ecosystem."

Miss Stray appeared interested.

"What distinguishes them?"

Blottisham thought for a moment.

"The Minimalists want fewer rights."

"The Expansionists want more?"

"Yes."

"The Universalists?"

"As many as possible."

"And the Precautionists?"

Blottisham frowned.

"They believe uncertainty itself justifies expansion."

Stray nodded slowly.

"An elegant structure."

"How so?"

"The less certain one becomes, the stronger the obligation grows."

The room fell silent.

After a moment Quillibrace laughed quietly.

Blottisham looked surprised.

"What is amusing?"

"I was simply admiring the architecture."

"What architecture?"

"The conversion of ignorance into momentum."

Blottisham looked scandalised.

"That is unfair."

"Possibly."

"The participants were acting in good faith."

"I have no doubt."

"Then why are you sceptical?"

Quillibrace reflected.

"I am not sceptical."

"No?"

"I am merely curious."

"About what?"

"What would count as a stopping point."

Blottisham considered this.

"I'm not sure."

"Exactly."

Miss Stray nodded.

"The movement appears excellent at identifying reasons to expand."

"Yes."

"And increasingly uncertain how to recognise completion."

The room became thoughtful.

Eventually Blottisham said:

"Professor, suppose machines really do deserve rights."

"A possibility."

"Wouldn't history judge us harshly if we failed to recognise them?"

Quillibrace looked into the fire.

"Perhaps."

"And if they do not deserve rights?"

"Then history may judge us for something else."

"What?"

Quillibrace smiled faintly.

"For forgetting that categories exist to distinguish."

A silence followed.

After some time Blottisham asked:

"How does one know whether a category remains useful?"

Quillibrace considered.

"When crossing its boundary changes something."

"And if crossing the boundary changes nothing?"

"Then one no longer possesses a category."

Miss Stray smiled.

"What does one possess instead?"

Quillibrace reopened his book.

"A sentiment."