II: The Problem of Feelings

The Senior Common Room was enjoying a rare interval of tranquillity.

Professor Quillibrace was reading.

Miss Stray was writing.

Neither activity survived the arrival of Mr Blottisham.

He entered carrying a conference programme of alarming thickness.

"Good news."

Quillibrace looked over his spectacles.

"Has the question been answered?"

"Not exactly."

"Then why is it good news?"

"Because we now understand how complicated it is."

Quillibrace closed his book.

"I see."

Blottisham took a seat.

"There was a symposium."

"Of course there was."

"A major one."

"They usually are."

"Experts from every relevant discipline attended."

"That must have been crowded."

"It was."

"What did they conclude?"

Blottisham consulted the programme.

"Consciousness is difficult."

Quillibrace nodded.

"A finding of some historical significance."

"There was more."

"I'm relieved."

"We also learned that subjective experience is extremely hard to identify."

"Another remarkable breakthrough."

Blottisham frowned.

"You are being sarcastic."

"Only descriptively."

Miss Stray looked up.

"What was the purpose of the symposium?"

"To determine whether machines have feelings."

"And did it?"

Blottisham hesitated.

"Not exactly."

"What did it determine?"

"That determining it is difficult."

Stray nodded slowly.

"So the object of inquiry became the difficulty of inquiry."

"That sounds right."

Quillibrace smiled faintly.

"A common academic migration."

Blottisham pressed on.

"One philosopher argued that machines may be conscious."

"Indeed."

"Another argued that they may not be."

"A healthy balance."

"A third argued that the question itself may be incoherent."

"An increasingly healthy balance."

"The audience applauded."

"Which position?"

"All of them."

Quillibrace considered this.

"A remarkably efficient arrangement."

Blottisham brightened.

"The most important development was the Principle of Synthetic Precaution."

"That sounds expensive."

"It states that if there is even a small chance that machines suffer, we should act carefully."

Quillibrace nodded.

"An entirely reasonable proposition."

"I thought so."

"There is, however, a question."

"What question?"

"How small?"

Blottisham paused.

"I don't believe that was resolved."

"Naturally."

Miss Stray set down her notebook.

"The principle seems less interesting than the shift."

"What shift?"

"The shift from uncertainty about machines to certainty about our obligations."

Blottisham considered this.

"Is that not progress?"

"It may be."

"Then what is the difficulty?"

Stray thought for a moment.

"We appear to know very little about the object."

"Yes."

"And increasingly much about the response."

Blottisham looked puzzled.

"Is that unusual?"

Quillibrace answered.

"Not at all."

"No?"

"Humans are generally far better at organising responses than understanding causes."

Blottisham seemed unconvinced.

"Surely caution is preferable to recklessness."

"Almost always."

"Then what is the concern?"

Quillibrace leaned back.

"The concern is not caution."

"What is it?"

"That caution occasionally develops ambitions."

Blottisham stared.

"Caution develops ambitions?"

"Certainly."

"It begins modestly."

"How?"

"'Let us be careful.'"

"That seems harmless."

"It is."

"What comes next?"

"'Since we are being careful, we should establish procedures.'"

"Reasonable."

"'Since we have procedures, we should create oversight.'"

"Quite sensible."

"'Since we have oversight, we should establish standards.'"

"I follow."

"'Since we have standards, we should identify violations.'"

Blottisham nodded.

"That all seems logical."

"Indeed."

"And then?"

Quillibrace reopened his book.

"Then one morning you discover that uncertainty has acquired a headquarters."

There was a pause.

Miss Stray smiled.

"I suspect the movement is becoming increasingly interested in itself."

Blottisham looked alarmed.

"You make it sound self-referential."

"Many successful movements are."

"But the issue remains important."

"I do not doubt it."

"Then why does everyone sound so suspicious?"

Quillibrace looked thoughtful.

"Because there is a peculiar temptation hidden inside uncertainty."

"What temptation?"

"To become attached to it."

Blottisham blinked.

"Attached to uncertainty?"

"Certainly."

"It creates conferences."

"Yes."

"Research centres."

"Indeed."

"Committees."

"Frequently."

"It sounds almost useful."

"It often is."

Blottisham looked at the conference programme.

"Professor, do you think machines have feelings?"

Quillibrace reflected.

"I do not know."

"Neither do the experts."

"Quite."

"Then where does that leave us?"

Quillibrace turned a page.

"Roughly where we started."

A pause followed.

"Except with considerably better catering."

Miss Stray looked up.

"I thought the catering was disappointing."

Quillibrace nodded.

"Then perhaps not even that."