Several afternoons later, Mr Blottisham arrived carrying a notebook filled with underlined passages.
This was seldom encouraging.
Professor Quillibrace was reading quietly.
Miss Stray was arranging papers.
Blottisham entered with evident purpose.
"I have discovered something extraordinary."
Quillibrace looked up.
"Again?"
"Yes."
"How fortunate."
Blottisham sat down.
"The future machine will eventually become incomprehensible."
"To whom?"
"To us."
"I see."
"It will be so intelligent that we shall no longer understand its reasoning."
Quillibrace nodded.
"A familiar proposition."
"An important proposition."
"Possibly."
Blottisham opened his notebook.
"The argument is impeccable."
"Excellent."
"If a machine becomes vastly more intelligent than a human..."
"Yes."
"...its thought processes must eventually exceed human comprehension."
"Must they?"
"Certainly."
The professor considered this.
"How would we know?"
Blottisham blinked.
"Know what?"
"That its reasoning exceeds our comprehension."
"Because we would fail to comprehend it."
Quillibrace nodded.
"Indeed."
The room became quiet.
Miss Stray looked interested.
After a moment Quillibrace asked:
"Suppose I hand you a page of mathematics."
"Very well."
"You do not understand it."
"Entirely possible."
"Does this prove the mathematics exceeds your intelligence?"
"No."
"Why not?"
"Because I might simply lack the relevant training."
"Excellent."
The professor folded his hands.
"Now suppose I hand you another page."
"Yes."
"You do not understand it either."
"Very well."
"How do you distinguish between profundity and obscurity?"
The room became still.
Blottisham frowned.
"That seems unfair."
"Why?"
"Because the future machine will genuinely be intelligent."
"Perhaps."
"Not perhaps."
"Possibly."
Blottisham sighed.
Miss Stray smiled.
The ritual had become familiar.
After a pause she asked:
"Do people perhaps assume that incomprehensibility is evidence of intelligence?"
Blottisham brightened.
"Exactly."
Quillibrace looked thoughtful.
"That is a remarkably dangerous habit."
"Why?"
"Because many things are incomprehensible."
"Such as?"
"The tax code."
"Ah."
"Certain committee reports."
"Indeed."
"Some contemporary poetry."
Miss Stray laughed.
"Steady on, Professor."
The professor smiled.
"My apologies."
The room relaxed.
Blottisham remained determined.
"The principle still holds."
"What principle?"
"The greater the intelligence, the greater the incomprehensibility."
Quillibrace considered this carefully.
"I wonder."
"No?"
"One might equally argue that greater intelligence produces greater explanatory power."
Blottisham looked troubled.
The possibility appeared not to have occurred to him.
Miss Stray nodded.
"A good teacher, for example."
"Precisely."
"A good scientist."
"Indeed."
"A good mathematician."
"Quite."
She paused.
"Many of the people we regard as intellectually exceptional are unusually good at making difficult things understandable."
The room became quiet.
Blottisham stared at the carpet.
This idea appeared to be rearranging some internal furniture.
Eventually he rallied.
"Nevertheless, there must be a point beyond which understanding fails."
"Perhaps."
"And beyond that point, the machine will continue advancing."
"Possibly."
Blottisham looked relieved.
"Then we agree."
"Not entirely."
The relief vanished.
Quillibrace leaned forward.
"What interests me is the peculiar structure of the claim."
"What structure?"
"We begin with understanding."
"Yes."
"We then imagine a future state beyond understanding."
"Yes."
"We then proceed to describe that state in considerable detail."
The room became silent.
Miss Stray laughed softly.
Blottisham looked annoyed.
"I know what you are implying."
"Do you?"
"You think there is a contradiction."
"Only a small one."
The professor picked up his teacup.
"If a condition is defined by our inability to understand it..."
"Yes."
"...how do we become so confident about its properties?"
Blottisham opened his mouth.
Then closed it.
The room waited.
After several moments Miss Stray spoke.
"I wonder whether the future machine sometimes functions like a mirror."
The others turned toward her.
She continued.
"When people describe its incomprehensible wisdom..."
"Yes?"
"...they often seem to be describing what they hope exists beyond their own uncertainty."
The room became very quiet.
Even Blottisham did not immediately respond.
Quillibrace regarded her thoughtfully.
"An interesting observation."
The silence deepened.
Outside, rain tapped softly against the windows.
Inside, Blottisham stared at his notebook.
At length he said:
"Surely there are limits to human understanding."
"Undoubtedly," replied Quillibrace.
"And something may lie beyond them."
"Certainly."
"Then why are you both so sceptical?"
The professor smiled.
"We are not sceptical of the possibility."
"No?"
"We are merely cautious about the descriptions."
The room fell silent once more.
Eventually Quillibrace returned to his book.
Miss Stray resumed her notes.
Blottisham remained staring into space.
Several minutes later he said:
"I believe I may have understood."
Quillibrace looked up.
"Excellent."
A pause followed.
Blottisham frowned.
"Although I cannot quite explain why."
The professor smiled faintly.
"Then perhaps you are becoming superintelligent."
No comments:
Post a Comment