The recent triumph of an AI pokerbot over five human opponents at once is hailed at science journal Nature as an event that “brings bots closer to solving complicated real-world problems”:

The team behind Pluribus had already built an AI, called Libratus, that had beaten professionals at two-player poker. It built Pluribus by updating Libratus and created a bot that needs much less computing power to play matches. In a 12-day session with more than 10,000 hands, it beat 15 top human players. “A lot of AI researchers didn’t think it was possible to do this using [our] techniques,” says Noam Brown at Carnegie Mellon University in Pittsburgh, Philadelphia, and Facebook AI Research in New York, who developed Pluribus with his Carnegie colleague Tuomas Sandholm.

Douglas Heaven, “No limit: AI poker bot is first to beat professionals at multiplayer game ” at Nature

Pluribus played trillions of hands against itself, without human input, settling on a few strategies not often used by human players.

The move from beating a single player to beating five is seen as an advance because “game theory is less helpful for scenarios involving multiple parties with competing interests and no clear win–lose conditions — which reflect most real-life challenges.” (Nature)

Is the pokerbot really a step on the road to getting AI to do our complex thinking for us, as the article goes on to suggest? Computer engineer Eric Holloway, a Walter Bradley Center fellow, offers some thoughts:

That’s pretty neat, but these game-playing AIs are an apples-to-oranges comparison with human intelligence. They all take extraordinarily more attempts to learn these strategies than it takes a human. So, whatever AI is doing, it is not comparable to human intelligence.

Someone might say, “We don’t care if AI is like human intelligence, just as we don’t care if airplanes fly like birds!”

That’s fair enough but it misses the point. If AI is phenomenally inefficient compared to human intelligence, as all these gaming AIs show, then AI will never be able to match human performance. There is always a line past which humans will outperform AI. Greater processing power and more storage will move the line somewhat, but the line will always remain.

The fundamental problem is the decision tree. At each point where the AI must make a decision, the tree splits. So, for instance, if there are always 2 decisions, and there are 10 decision points, then there are 2 to the power of 10 different paths the AI must consider. This means that for any moderately long sequence of decisions the number of paths the AI must search is greater than the lifespan of the universe.

So what sets humans apart? It is our ability to learn and apply general principles. We can identify trends and patterns in these games that we can extrapolate for many moves into the future, allowing us to establish long-term strategies. In the short term, the AI can explore all the paths, and so it can develop very effective tactics.

However, in the long term, there are just too many paths for the AI to explore. It will end up dramatically miscalculating human strategies that extend beyond its tactics horizon. So, the inability of AI to generalize makes it inferior to humans in the gameplaying domain and any other.

In the bigger picture, this is the problem with modern AI. The most widely used AI techniques, such as deep neural networks and decisions trees, all suffer from the problem of overfitting, where they are unable to generalize from the dataset. They end up memorizing the noise and irrelevant features. That is why Google’s deep learning model is fooled into thinking a kitten is an avocado by adding a single green pixel or neural networks learn to recognize criminals by the white background of the mug shot, and so on. All of modern AI is crippled by its inability to generalize.

One final analogy, modern AI is like a whiz student that can pass every test but does so by using a cheat sheet. It can get all the right answers in the restricted domain of the test environment. The machine learning researches give the AI all the answers during the training phase and the AI is just learning the relationship between question and answers. However, the AI is not figuring out anything regarding the principles that lead to the right answers in the first place. This is why whenever modern AI is taken outside of its finely tuned test environment it falls flat on its face.

There is a security issue here: Once modern AI hits widespread production and use, there will be so many different ways for hackers to exploit the systems. If you think our modern internet infrastructure is at risk for hacking, even though it is fairly well understood, then just imagine what will happen once we run our systems with very poorly understood black boxes that are full of holes.

Software developer Brendan Dixon, who follows the news around self-driving cars, adds, “researcher Noam Brown ‘thinks that their success is a step towards applications such as automated negotiations, better fraud detection and self-driving cars.’ But according to Nature: ‘To tackle six-player poker, Brown and Sandholm radically overhauled Libratus’s search algorithm. Most game-playing AIs search forwards through decision trees for the best move to make in a given situation. Libratus searched to the end of a game before choosing an action.'”

“The trouble is, any technique that works by searching ‘to the end of the game’ will not help self-driving cars (as an example) one bit…unless they have also mastered predicting the future. There is no ‘end of the game’ for nearly all decisions we make.”

So when we hear that AI will soon be able to solve complex real-world problems because it wins at Texas hold ‘em, you can be sure of one thing: It isn’t the machine that’s doing the bluffing this time.

Historical note: Bradley Center director Robert J. Marks notes that Bernie Widrow was using a computer to win at the card game of Black Jack at Stanford in 1960. He also forecast weather and translated spoken language to type. Not as sophisticated as today, but very impressive.

See also: Why AI can’t win wars as if wars were chess games. (Bradley A. Alaniz and Jed Macosko)

and

Fake news thrives on fears of a robot takeover (Brendan Dixon)