Pertinent to my interests, and tonight’s Jeopardy!, here is a thought experiment by American philosopher John Searle.

It addresses the question: if a machine can convincingly simulate an intelligent conversation, does it necessarily understand? In the experiment, Searle imagines himself in a room acting as a computer by manually executing a program that convincingly simulates the behavior of a native Chinese speaker. People outside the room slide Chinese characters under the door and Searle, to whom “Chinese writing is just so many meaningless squiggles”, is able to create sensible replies, in Chinese, by following the instructions of the program; that is, by moving papers around. The question arises whether Searle can be said to understand Chinese in the same way that, as Searle says, “according to strong AI, … the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to understand and have other cognitive states.”

Searle concludes that since he does not understand Chinese, the system created by Searle + computer + room, the Chinese room, does not understand Chinese. If you don’t follow, the Chinese room is supposed to represent AI. Therefore, AI is excluded from achieving human understanding.