It has been 10 years since the Great Recession bottomed out. The unemployment rate has fallen from a peak of 10 percent in October 2009 to below 4 percent today. And now might be a good time to remind ourselves of an expensive lesson: inscrutable math and astonishing computers are not infallible.

During the Great Depression in the 1930s, when millions of people lost their jobs, homes, farms, and businesses, the great British economist John Maynard Keynes (1883–1946) dismissed the wishful thinking of the prevailing economic models:

Professional economists, after Malthus, were apparently unmoved by the lack of correspondence between the results of their theory and the facts of observation… It may well be that the classical theory represents the way in which we should like our Economy to behave. But to assume that it actually does so is to assume our difficulties away.”

J.M. Keynes (1993) The General Theory of Employment, Interest and Money (Macmillan Cambridge University Press), p. 33

Déjà vu. Déjà vu. We are still seduced by elegant models that misrepresent the messy reality we live in.

The roots of the Great Recession began in the 1970s when mortgage lending was revolutionized by three government-sponsored agencies, nicknamed Fannie Mae, Ginnie Mae, and Freddie Mac, that bought mortgages from banks and S&Ls. Soon private mortgage funds joined the revolution and more than half of all mortgages were sold to mortgage funds.

In theory, this was win-win. Investors could buy mortgages as well as stocks and bonds and banks that sold their mortgages could make more loans. In practice, this new system created perverse incentives. Banks had traditionally been cautious lenders because, if borrowers default, banks were stuck with the bad loans. However, when a bank sells its mortgages to someone else, it has little reason to be prudent. Its profits depend on the number of loans it approves, not on whether the mortgage payments are made on time.

This problem was exacerbated by the growth of national mortgage brokers who made money by issuing mortgages, including subprime “NINJA” loans—that is, loans to homebuyers with No Income, No Job, and no Assets who, at the first sign of trouble, disappear like ninjas in the night. The website of one mortgage broker boasted, “We don’t get paid unless we say YES.” With no incentive to say NO, it is not surprising that lots and lots of questionable loans were approved.

In addition, clever financial engineers discovered that they could make ridiculous fees by dividing mortgage pools into sub-pools, called tranches, based on the default risk. Investors who bought the lowest (“junior”) tranches received more income from the mortgages but were hit the hardest if borrowers defaulted. These sliced-and-diced mortgage pools were called collateralized debt obligations (CDOs).

As if that wasn’t complicated enough, investment banks created CDOs-squared, synthetic CDOs, and credit default swaps (CDS). Don’t ask. One thing that financial engineers are very good at is thinking up exotic securities that sound good, but almost no one understands them.

Who was dumb enough to invest in risky pools of subprime mortgages and related securities that they didn’t understand? Incredibly, mortgage pools of subprime loans were rated triple-A and bought by pension funds, insurance companies, and other supposedly prudent investors. How could a pool of mortgages be rated triple-A when almost every mortgage in the pool was junk?

Professors at prestigious business schools and financial engineers with advanced degrees built mathematical models to assess the risks, but their models were deeply flawed.

One problem was that the models often focused on daily price fluctuations, rather than the chances that mortgage defaults would make the pools, tranches, and swaps worthless. Citigroup’s chief financial officer later lamented, “We had a market-risk lens looking at those products, not the credit-risk lens.”

A second problem was that the model builders underestimated the chances of extreme events because they had been seduced by the mathematical convenience of the normal distribution which assigns low probabilities to extreme events.

A third problem was that default probabilities were estimated from historical data but, historically, subprime borrowers didn’t have mortgages. AIG claimed that, based on historical data, the chances of CDO defaults were “remote, even in severe recessionary market scenarios.” In 2007, they boasted, “It is hard for us, without being flippant, to even see a scenario within any kind of realm of reason that would see us losing one dollar in any of those transactions.” Like those who thought black swans were impossible, they thought that a wave of mortgage defaults was impossible.

A fourth problem was that the models often relied on an elegant mathematical theorem that states that a bundle of risky assets is safer than any asset in the bundle. The crucial assumption is that the returns are independent but mortgage defaults may not be. If Jane loses her job and can’t make her mortgage payments, that doesn’t necessarily mean that Joe will lose his job and not make his mortgage payments. However, in a recession, lots of Janes and Joes lose their jobs, especially Janes and Joes who were bad credit risks because of their unstable employment history.

Some suspected that the risks were real and enormous, but they didn’t want to miss out on the money that was being made. Shortly before the party ended, Citigroup’s CEO said that “as long as the music is playing, you’ve got to get up and dance. We’re still dancing.”

Underlying many of the mistakes was incredible hubris. Too many people thought that, because they were making so much money, they must really be too smart to make mistakes.

You know how it ended. The music stopped and the dancers ran for the exits. Home prices stopped rising and dominos started falling. Subprime borrowers couldn’t make their mortgage payments and couldn’t sell their homes for enough money to pay off their mortgages. Many sold their homes at distress prices or walked away from their underwater loans, which pushed home prices down further and encouraged more subprime borrowers to walk away.

Homebuilders stopped building homes and ripples from unemployed construction workers spread through the economy, as spending, income, and stock prices collapsed. Household wealth fell by $12 trillion and the unemployment rate doubled in 18 months, from 5 percent in April 2008 to 10 percent in October 2009, the highest since the Great Depression. The U.S. economy was on the brink of a second great depression.

The mortgage defaults triggered CDO defaults, which in turn triggered swap defaults. Bear Stearns, Lehman Brothers, Merrill Lynch, J.P. Morgan, AIG, and other financial institutions were in trouble and Congress was forced to create a $700 billion bailout fund.

Hundreds had fallen into deep financial pits. Millions paid the price.

I love math and I love computers, but we shouldn’t be blinded by love. Math and computers are tools, nothing more. They are certainly not masters that should be blindly obeyed. As Warren Buffett once warned, “Beware of geeks bearing formulas.”

Here’s more statistical wisdom from Gary Smith:

A BABY, a GEEK, and a COW … all walk into a bar looking for some BEER and VINO… What happens next? They all beat the market. Were their ticker names a factor? I decided to study that.

We see the pattern! But is it real? It’s natural to imagine that a deep significance underlies coincidences. Unfortunately, patterns are not always a source of information. Often, they are a meaningless coincidence like the 7-11 babies this summer.

Computers’ stupidity makes them dangerous. The real danger today is not that computers are smarter than us, but that we think computers are smarter than us. Many marketing decisions, medical diagnoses, and stock trades, loan and job applications, and election strategies are evaluated by computers. But, as my little experiment shows, the computer does not know whether a pattern is information or noise.

The paradox of luck and skill Why did Shane Lowry win the British Open golf championship? Because someone had to. In any competition including academic tests, athletic events, and company management where there is an element of luck that causes performances to be an imperfect measure of ability, there is an important difference between competitions among people with high ability and competitions among people of lesser ability.

and

The US 2016 election: Why Big Data failed. Economics professor Gary Smith sheds light on the surprise result.