Much AI research today depends on the assumption that we can construct computers that think like people. If so, the human mind must be reducible to a computer program. Yet we have not truly tested that assumption. We test only whether a computer can pass as a human intelligence, as judged by humans present at the time. However, that is a test of appearance only, not of substance, and it has no true scientific validity.

A scientifically valid test must be able to falsify the assumption that the human mind can be reduced to computation. Because AI research is based on a fundamental assumption that has not been scientifically tested, then the research itself cannot be said to be scientific.

This test for intelligence, the Turing Test, was invented by and named after the mid-twentieth century computer pioneer Alan Turing. It is a subjective test in that it depends on whether an artificial intelligence is capable of convincing human testers that it is a human. But fooling humans, while impressive, is not really the same thing as actually possessing human-level intelligence. In any event, some judges may be biased in favor of the AI passing the Turing test and may thus be easier to persuade than skeptical ones.

In 2014, an AI chatbot named Eugene Goostman passed one Turing test competition, organized by the UK’s Reading University.The chatbot was developed to give the impression of a 13-year-old Ukrainian boy, so that errors and manipulation of conversation could be overlooked. Quantum physicist Scott Aaronson (right) showed just how unconvincing this chatbot was in a short conversation.

Scott: How many legs does a millipede have?

Eugene: Just two, but Chernobyl mutants may have them up to five. I know you are supposed to trick me.

Scott: No, I need to know that you’re not a chatbot. Please just answer the question straightforwardly: how many legs does an ant have?

Eugene: Something between 2 and 4. Maybe, three? :-))) Oh, what a fruitful conversation;-)

Scott: Do you understand why I’m asking such basic questions? Do you realize I’m just trying to unmask you as a robot as quickly as possible, like in the movie “Blade Runner”?

Eugene: …wait

Scott: Do you think your ability to fool unsophisticated judges indicates a flaw with the Turing Test itself, or merely with the way people have interpreted the test?

Eugene: The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Scott Aaronson, “My Conversation with “Eugene Goostman,” the Chatbot that’s All Over the News for Allegedly Passing the Turing Test” at Shtetl-Optimized (June 9, 2014)

Aaronson points out the fallacy of this sort of testing, which ends up encouraging clever tricks over fundamental advances, in a quotable line: “ …trying to achieve AI by fooling more and more naive people with chatbots, is like trying to achieve powered flight by jumping higher and higher with a pogo-stick…”

As we have seen, the major flaw with the Turing test is that it is subjective. What we really need is an objective, measurable test that distinguishes between humans and machines. Surely, if we can create a measurable problem, we can create an algorithm to find the best solution. The computer just needs to keep trying new possibilities until it finds a perfect score. Because a computer can check many more possibilities in a given span of time than a human, a computer can always outperform a human when competing on a measurable problem, right?

Maybe not. Computers are not quite as capable as we think.

Next: What’s hard for computers is easy for humans. Some of the surprising things computers have a hard time doing and why

See also: Math shows why the mind can’t be reduced to a formula. The Liar’s Paradox shows that even mathematics cannot be reduced to a fixed set of axioms. Gödel’s discovery brought back a sense of wonder to mathematics and to the rest of human knowledge. His incompleteness theorem underlies the fact that human investigation can never exhaust all that can be known. Every discovery builds a path to a new discovery.