When Would We Know? — A Natural History of Consciousness: III. The Problem of the Perfect Imitation

Suppose we build a machine that behaves exactly as a conscious person would.

It reports pain when damaged. It says that colours look different. It remembers yesterday, anticipates tomorrow, complains about boredom and asks whether anyone else understands what it is like to be it.

Suppose its behaviour is not merely convincing.

Suppose it is perfect.

Would that tell us that the machine is conscious?

Not necessarily.

This is the problem of the perfect imitation. We can imagine a system producing all the right behaviour while, supposedly, having no experience at all. There would be nothing it was like to be that system.

The thought experiment creates an uncomfortable gap between behaviour and experience.

But there is another question.

What would make the imitation an imitation?

If two systems were behaviourally indistinguishable but one was conscious and the other was not, we would need some difference between them that mattered. Perhaps their internal organisation would differ. Perhaps their causal dynamics would differ. Perhaps consciousness would depend on some property that behaviour alone does not reveal.

But then the question returns:

What property?

If there is a real difference, we need an account of what the difference is and why it matters. If there is no difference anywhere in the organisation of the two systems, it becomes difficult to say what an empirical test could possibly detect.

The perfect imitation therefore does not merely show that behaviour can deceive us.

It exposes a deeper problem.

A test can tell us what a system does. It can sometimes tell us how the system is organised. But consciousness concerns whether there is something it is like to be that organisation.

So perhaps the question is not whether a machine can imitate consciousness perfectly.

Perhaps the question is whether, at some point, calling its behaviour an imitation leaves out something about the system itself.

That takes us beyond the test.

It takes us to the harder question:

When was there something to be like?