Walter Bradley Center’s director Robert J. Marks was joined for this week’s podcast by economics professor Gary Smith. This episode, When I Nod My Head, Hit It! And Other Commands that Confuse AI, explores the fact that computers don’t have common sense.

Which means that as data gets larger and larger, nonsensical coincidences become more probable, not less.

Smith, the Fletcher Jones Professor of Economics at Pomona College, is the author of The AI Delusion (Oxford University Press, 2018). His other books include estimable textbooks like Essential Statistics, Regression, and Econometrics (2015). But he has also written serious popular discussions like Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics (2014). The latter’s basic thesis is, “As Nobel Prize-winning economist Ronald Coase once cynically observed, ‘If you torture data long enough, it will confess.’”

One example Smith offered is false correlations; we think a statistic means something but it doesn’t. A famous example is the idea that football wins correlate with the stock market. The idea got started as a sportswriter’s joke in 1978 but it took on a life of its own:

The Super Bowl Indicator, at one point in time, boasted a more than 90% success rate in predicting the up-or-down outcome of the S&P 500 the following year. However, the old maxim applies: correlation does not imply causation…

From 2007 to 2017, the Super Bowl Indicator went 50-50 in predicting the up-down performance of the market, the same as a coin flip. It failed to predict a down market in both 2016 and 2017, when the Denver Broncos and New England Patriots, both original AFC teams, won Super Bowls. Also of note, in 2008, despite the New York Giants (NFC) winning the Super Bowl, which supposedly indicated a bull market, the stock market suffered one of the largest downturns since the Great Depression.

Will Kenton, “Super Bowl Indicator” at Investopedia

Folklore dies hard:

Smith: “I have friends on Wall Street who are rooting for the Rams because they think it’s good for the stock market. That’s just nuts but its part of human nature to see patterns and think they’re meaningful. And what we have with computers nowadays, AI in particular, is just that kind of data mining—coincidental correlations on steroids because computers are so good at finding patterns and correlations but absolutely useless at deciding whether they make sense or not or they’re some kind of cosmic joke.”

What difference has AI made to the stock market? Smith tells us, “A third of all trades now are made by computers with no human supervision.” However, he notes, many computer trades are just a legal form of “front-running,” in this case, taking advantage of rapid electronic turnaround times to earn small amounts on shares before more analytical trading kicks in for bigger gains.

After a brief discussion of AI and the US 2016 election, the discussion turned to IBM’s Watson, which beat human contestants at Jeopardy in 2011:

Marks: I heard that the programmers of Watson had certain types of questions that they did not want Watson to be asked.

Smith: There were two types of games going on. For the human contestants, they weren’t allowed to buzz in until the light went on. It took them a fraction of a second to see the light and respond. And Watson couldn’t see lights and it was sent an electronic signal about when it was okay to buzz in and that signal got there faster and was processed faster so Watson was able to buzz in repeatedly faster than humans were. It wasn’t that humans didn’t know the answer; it’s that they didn’t have the reflexes.

The other gaming was that computers don’t really understand words. You could say a word like “Abraham Lincoln” was the sixteenth president of the United States” and the computer doesn’t know what “sixteenth” and “President of the United States” means but it can go rummage through its Wikipedia-like sources and find those words and match them to a president, Abraham Lincoln. But then you put anything in that’s like a puzzle or a joke or a riddle or sarcasm that you can’t look up in Wikipedia and the computers are helpless.

So the IBM team did not want Watson asked any questions that involved ambiguity.

On that note, talk turned to Watson’s interaction with Nobel Peace Prize-winning folk singer Bob Dylan:

As Smith recalls, the cult around Watson provoked one of his colleagues, Roger Schank, to harsh words, “Watson is a fraud. I am not saying that it can’t crunch words, and there may well be value in that to some people. But the ads are fraudulent.”

Schank told his readers in 2016:

I am a child of the 60s’ and I remember Dylan’s songs well enough. Ask anyone from that era about Bob Dylan and no one will tell you his main theme was “love fades”. He was a protest singer, and a singer about the hard knocks of life. He was part of the anti-war movement. Love fades? That would be a dumb computer counting words. How would Watson see that many of Dylan’s songs were part of the anti-war movement? Does he say anti-war a lot? He probably never said it in a song.

Roger Schank, “They are not doing “cognitive computing” no matter how many times they say they are” at RogerShank.com

This inability to “understand” could help us see why Watson has not really lived up to its hype as a brave new tool for medicine.

Next week: Why IBM Watson is Going Toes Up

Here’s a list of podcasts from Mind Matters News

Further reading on AI and Big Data:

Big Data can lie: Simpson’s Paradox The Paradox illustrates the importance of human interpretation of the results of data mining

Study shows eating raisins causes plantar warts Sure. Because, if you torture a Big Data enough, it will confess to anything

and

IBM’s Watson is NOT our new computer overlord